论文标题
对话响应的多次参考培训
Multi-Referenced Training for Dialogue Response Generation
论文作者
论文摘要
在开放域对话响应的一代中，可以通过各种响应来继续对话环境，对话模型应捕获这种一对一的关系。在这项工作中，我们首先从Kullback-Leibler Divergence（KLD）的角度分析对话模型的训练目标，并表明现实世界概率分布与单一参考数据的概率分布之间的差距可阻止该模型有效地学习一对多关系。然后，我们探讨了两个方面的多参考培训方法。从数据角度来看，我们从一个强大的预验证模型中生成了不同的伪参考，以构建多参考数据，从而更好地近似现实世界分布。在模型方面，我们建议将变异模型配备有表现力的先验，名为Linear Gaussian模型（LGM）。自动化评估和人类评估的实验结果表明，这些方法比基准产生了显着改善。我们将在https://github.com/zhaoting/dialog-processing中发布代码和数据。
In open-domain dialogue response generation, a dialogue context can be continued with diverse responses, and the dialogue models should capture such one-to-many relations. In this work, we first analyze the training objective of dialogue models from the view of Kullback-Leibler divergence (KLD) and show that the gap between the real world probability distribution and the single-referenced data's probability distribution prevents the model from learning the one-to-many relations efficiently. Then we explore approaches to multi-referenced training in two aspects. Data-wise, we generate diverse pseudo references from a powerful pretrained model to build multi-referenced data that provides a better approximation of the real-world distribution. Model-wise, we propose to equip variational models with an expressive prior, named linear Gaussian model (LGM). Experimental results of automated evaluation and human evaluation show that the methods yield significant improvements over baselines. We will release our code and data in https://github.com/ZHAOTING/dialog-processing.
