KEPLET：具备主题实体意识的知识增强预训练语言模型（英文版）

英文标题：KEPLET: Knowledge-Enhanced Pretrained Language Model with Topic Entity Awareness中文摘要：本文提出了一种新型的 KEPLMs - KEPLET，它不仅考虑了维基百科丰富的语料库结构，还具有主题实体感知。实验证明，KEPLET 应用于两种典型的 KEPLMs，在四项实体中心任务中取得了显著的改进。英文摘要：In recent years, Pre-trained Language Models (PLMs) have shown theirsuperiority by pre-training on unstructured text corpus and then fine-tuning ondownstream tasks. On entity-rich textual resources like Wikipedia,Knowledge-Enhanced PLMs (KEPLMs) incorporate the interactions between tokensand mentione