RAFT: 用于生成式基础模型对齐的奖励排序微调方法（英文版）

英文标题：RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment中文摘要：本文提出了一种新的框架 RAFT，它利用奖励模型和足够数量的样本将生成模型对齐，选择高质量的样本并去除那些表现不良的样本。该算法在大型语言模型和扩散模型的情况下表现良好。英文摘要：Generative foundation models are susceptible to implicit biases that canarise from extensive unsupervised training data. Such biases can producesuboptimal samples, skewed outcomes, and unfairness, with potentiallysignificant repercussions. Consequently, aligning these models with humanethics and preferences is an essential ste