人类和机器学习模型的标记可追踪性：一项注释研究（英文版）

英文标题：Tokenization Tractability for Human and Machine Learning Model: An Annotation Study中文摘要：本研究比较了六种不同的 tokenization 方法在日语常识问答数据集上人工注释和机器学习模型的表现，分析了 tokenization 的合适性和响应时间等因素和表现的关系，结果表明人类和机器学习模型的 tokenization 不完全一致。英文摘要：Is tractable tokenization for humans also tractable for machine learningmodels? This study investigates relations between tractable tokenization forhumans (e.g., appropriateness and readability) and one for models of machinelearning (e.g., performance on an NLP task). We compar