论文标题
可控文本生成，用于开放域的创造力和公平性
Controllable Text Generation for Open-Domain Creativity and Fairness
论文作者
论文摘要
大型预训练的语言模型的最新进展表明，在生成自然语言以及许多自然语言生成（NLG）应用（例如机器翻译和文本摘要）的表演方面取得了良好的成果。但是，当一代任务更开放并且内容不足时，现有的技术难以生成长期连贯和创造性的内容。此外，这些模型表现出甚至扩大了从培训语料库中学到的社会偏见。之所以发生这种情况，是因为训练了生成模型以捕获表面模式（即单词序列），而不是捕获基本的语义和话语结构以及包括社会规范在内的背景知识。在本文中，我介绍了我们最近的有关可控文本生成的著作，以增强语言生成模型的创造力和公平性。我们探索层次结构的生成和限制解码，并将其应用于创意语言生成，包括故事，诗歌和比喻语言以及对生成模型的偏见缓解。
Recent advances in large pre-trained language models have demonstrated strong results in generating natural languages and significantly improved performances for many natural language generation (NLG) applications such as machine translation and text summarization. However, when the generation tasks are more open-ended and the content is under-specified, existing techniques struggle to generate long-term coherent and creative content. Moreover, the models exhibit and even amplify social biases that are learned from the training corpora. This happens because the generation models are trained to capture the surface patterns (i.e. sequences of words), instead of capturing underlying semantics and discourse structures, as well as background knowledge including social norms. In this paper, I introduce our recent works on controllable text generation to enhance the creativity and fairness of language generation models. We explore hierarchical generation and constrained decoding, with applications to creative language generation including story, poetry, and figurative languages, and bias mitigation for generation models.
