语言模型中的实体跟踪（英文版）

英文标题：Entity Tracking in Language Models中文摘要：本文探讨了大语言模型在跟踪实体状态和关系变化方面的能力，发现只有预训练于大量代码的 GPT-3.5 模型具有此能力，而使用预训练于文本的较小模型进行微调后也可以完成一定程度的实体追踪。但这种能力不仅取决于模型的大小，大文本库的预训练也不是必要条件。英文摘要：Keeping track of how states and relations of entities change as a text ordialog unfolds is a key prerequisite to discourse understanding. Despite thisfact, there have been few systematic investigations into the ability of largelanguage models (LLMs) to track discourse entities. In this work, we present atask to probe to what