论文标题
用潜在单词袋的释义产生
Paraphrase Generation with Latent Bag of Words
论文作者
论文摘要
释义生成是自然语言处理中的长期重要问题。 此外，深层生成模型的最新进展显示了用于文本生成的离散潜在变量的有希望的结果。 受到带有离散潜在结构的变异自动编码器的启发，在这项工作中，我们提出了一个潜在的单词（BOW）模型，用于释义。 我们将弓从目标句子中扎根于离散潜在变量的语义。 我们使用此潜在变量来构建一个完全可区分的内容计划和表面实现模型。 具体而言，我们使用源词来预测他们的邻居并用软马克斯的混合物对目标弓进行建模。 我们使用Gumbel Top-K重新聚集化来对预测的弓分布进行可区分的子集采样。 我们检索采样的单词嵌入，并使用它们来增强解码器并指导其生成搜索空间。 我们的潜在弓模型不仅增强了解码器，而且还具有明显的解释性。 我们显示了与\ emph {（i）}无监督的词邻居\ emph {（ii）}逐步生成过程的无监督学习的模型。 广泛的实验证明了该模型的透明和有效的生成过程。
Paraphrase generation is a longstanding important problem in natural language processing. In addition, recent progress in deep generative models has shown promising results on discrete latent variables for text generation. Inspired by variational autoencoders with discrete latent structures, in this work, we propose a latent bag of words (BOW) model for paraphrase generation. We ground the semantics of a discrete latent variable by the BOW from the target sentences. We use this latent variable to build a fully differentiable content planning and surface realization model. Specifically, we use source words to predict their neighbors and model the target BOW with a mixture of softmax. We use Gumbel top-k reparameterization to perform differentiable subset sampling from the predicted BOW distribution. We retrieve the sampled word embeddings and use them to augment the decoder and guide its generation search space. Our latent BOW model not only enhances the decoder, but also exhibits clear interpretability. We show the model interpretability with regard to \emph{(i)} unsupervised learning of word neighbors \emph{(ii)} the step-by-step generation procedure. Extensive experiments demonstrate the transparent and effective generation process of this model.\footnote{Our code can be found at \url{https://github.com/FranxYao/dgm_latent_bow}}
