论文标题
半监督的终身语言学习
Semi-Supervised Lifelong Language Learning
论文作者
论文摘要
终身学习旨在依次学习任务时积累知识并减轻灾难性遗忘。但是，现有的终身语言学习方法仅着眼于监督学习环境。在现实情况下可以轻松访问的未标记数据未经忽视。在本文中，我们探讨了一种新颖的设置，半监督的终身语言学习（SSLL），其中模型通过标记和未标记的数据学习顺序到达语言任务。我们提出了一个未标记的数据增强的终身学习者来探索SSLL。特别是，我们专用于特定任务的模块来减轻灾难性的遗忘和设计两个模块以利用未标记的数据：（1）虚拟监督增强的任务求解器是在教师学生框架上构建的，以从未经标记的数据中挖掘基本知识； （2）构建了一个向后的增强学习者，以鼓励从新来的未标记数据转移到先前任务的知识转移。各种语言任务的实验结果证明了我们的模型在新设置SSLL下的有效性和优越性。
Lifelong learning aims to accumulate knowledge and alleviate catastrophic forgetting when learning tasks sequentially. However, existing lifelong language learning methods only focus on the supervised learning setting. Unlabeled data, which can be easily accessed in real-world scenarios, are underexplored. In this paper, we explore a novel setting, semi-supervised lifelong language learning (SSLL), where a model learns sequentially arriving language tasks with both labeled and unlabeled data. We propose an unlabeled data enhanced lifelong learner to explore SSLL. Specially, we dedicate task-specific modules to alleviate catastrophic forgetting and design two modules to exploit unlabeled data: (1) a virtual supervision enhanced task solver is constructed on a teacher-student framework to mine the underlying knowledge from unlabeled data; and (2) a backward augmented learner is built to encourage knowledge transfer from newly arrived unlabeled data to previous tasks. Experimental results on various language tasks demonstrate our model's effectiveness and superiority over competitive baselines under the new setting SSLL.
