Sabiá：葡萄牙语大语言模型（英文版）

英文标题：Sabiá: Portuguese Large Language Models中文摘要：在这篇论文中，我们证明了在目标语言上进行单语言预训练可以显著提高已经广泛训练于多个语料库的模型，并在 14 个葡萄牙语数据集上表现优于基于英语和多语言模型的模型。我们的结果表明，从单语预训练获得的大多数收益来自于领域特定知识。英文摘要：As the capabilities of language models continue to advance, it is conceivablethat "one-size-fits-all" model will remain as the main paradigm. For instance,given the vast number of languages worldwide, many of which are low-resource,the prevalent practice is to pretrain a single model on multiple languages. Inthis paper, we add to t