论文标题
学习重组和重新采样数据以进行组成概括
Learning to Recombine and Resample Data for Compositional Generalization
论文作者
论文摘要
灵活的神经序列模型在各种任务上都优于基于语法和自动机的同行。但是，神经模型在需要培训数据之外的组成概括的环境中表现较差，尤其是在罕见或看不见的子序列上。过去的工作发现这些环境中必不可少的象征性脚手架（例如语法或自动机）。我们描述了R＆R，这是一种学习的数据增强方案，可实现大量的组成概括，而无需吸引潜在的符号结构。 R＆R有两个组成部分：通过基于原型的生成模型重组原始培训示例，并重新采样生成的例子以鼓励外推。训练与重组和重新采样示例增强的数据集上的普通神经序列模型可显着改善两个语言处理问题的概括 - 以下教学（SCAN）和形态学分析（Sigmorphon 2018） - R＆R可以从八个初始示例中学习新的结构和时态。
Flexible neural sequence models outperform grammar- and automaton-based counterparts on a variety of tasks. However, neural models perform poorly in settings requiring compositional generalization beyond the training data -- particularly to rare or unseen subsequences. Past work has found symbolic scaffolding (e.g. grammars or automata) essential in these settings. We describe R&R, a learned data augmentation scheme that enables a large category of compositional generalizations without appeal to latent symbolic structure. R&R has two components: recombination of original training examples via a prototype-based generative model and resampling of generated examples to encourage extrapolation. Training an ordinary neural sequence model on a dataset augmented with recombined and resampled examples significantly improves generalization in two language processing problems -- instruction following (SCAN) and morphological analysis (SIGMORPHON 2018) -- where R&R enables learning of new constructions and tenses from as few as eight initial examples.
