论文标题
常识性故事创造的知识增强的预处理模型
A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation
论文作者
论文摘要
故事产生，即从领先的环境中产生合理的故事，是一项重要但具有挑战性的任务。尽管成功地建模流利和局部连贯性，但现有的神经语言产生模型（例如GPT-2）仍然遭受重复，逻辑冲突以及生成故事中缺乏远距离连贯性的困扰。我们猜想这是因为很难将相关的常识性知识，理解因果关系以及计划和事件的适当时间顺序秩序相关联。在本文中，我们为常识性故事创造了一种知识增强的预处理模型。我们建议利用来自外部知识基础的常识知识来产生合理的故事。为了进一步捕获合理故事中句子之间的因果关系和时间依赖性，我们采用了多任务学习，该学习结合了一个歧视性目标，以在微调过程中区分真实和虚假的故事。自动和手动评估表明，与最先进的基线相比，我们的模型可以产生更合理的故事，尤其是在逻辑和全球连贯性方面。
Story generation, namely generating a reasonable story from a leading context, is an important but challenging task. In spite of the success in modeling fluency and local coherence, existing neural language generation models (e.g., GPT-2) still suffer from repetition, logic conflicts, and lack of long-range coherence in generated stories. We conjecture that this is because of the difficulty of associating relevant commonsense knowledge, understanding the causal relationships, and planning entities and events with proper temporal order. In this paper, we devise a knowledge-enhanced pretraining model for commonsense story generation. We propose to utilize commonsense knowledge from external knowledge bases to generate reasonable stories. To further capture the causal and temporal dependencies between the sentences in a reasonable story, we employ multi-task learning which combines a discriminative objective to distinguish true and fake stories during fine-tuning. Automatic and manual evaluation shows that our model can generate more reasonable stories than state-of-the-art baselines, particularly in terms of logic and global coherence.
