VECO 2.0: 多粒度对比学习的跨语言语言模型预训练（英文版）

英文标题：VECO 2.0: Cross-lingual Language Model Pre-training with Multi-granularity Contrastive Learning中文摘要：本文提出了一种基于对比学习的多粒度对齐的跨语言预训练模型 VECO~2.0，将序列到序列对齐和标记到标记对齐相结合，以支持跨语言模型预训练，通过在 XTREME 基准上的实验证明了该方法的有效性。英文摘要：Recent studies have demonstrated the potential of cross-lingualtransferability by training a unified Transformer encoder for multiplelanguages. In addition to involving the masked language model objective,existing cross-lingual pre-training works leverage sentence-level contrastivelearning