基于伪目标训练的语言生成知识蒸馏系统研究（英文版）

英文标题：A Systematic Study of Knowledge Distillation for Natural Language Generation with Pseudo-Target Training中文摘要：研究自然语言生成（NLG）中的知识蒸馏技术优化模型并生成具体的文本任务，提出伪目标（PT）数据增强方法并应用于双方产生的多个 PT 的词级 KD，从而有效地压缩模型。英文摘要：Modern Natural Language Generation (NLG) models come with massivecomputational and storage requirements. In this work, we study the potential ofcompressing them, which is crucial for real-world applications serving millionsof users. We focus on Knowledge Distillation (KD) techniques, in which a s