预训练与搜索：利用预训练神经成本模型实现高效的嵌入表分片（英文版）

英文标题：Pre-train and Search: Efficient Embedding Table Sharding with Pre-trained Neural Cost Models中文摘要：本文提出了一种有效的分片方法 ——“预训练和搜索”，使用预训练的神经成本模型作为高效的分片模拟器，并在此基础上使用在线搜索来识别最佳分片方案，实验表明，NeuroShard 可以在分片数据集上以多项改进表现，同时在实际应用中，可以取得 6.6% 的端到端训练吞吐量改进。英文摘要：Sharding a large machine learning model across multiple devices to balancethe costs is important in distributed training. This is challenging becausepartitioning is NP-hard, and estimating the costs accurately and efficiently isdifficult. In this work, we e