论文标题
不忘记的转移
Transfer without Forgetting
论文作者
论文摘要
这项工作调查了持续学习（CL）与转移学习（TL）之间的纠缠。特别是，我们阐明了网络预处理的广泛应用，强调它本身受到灾难性遗忘的影响。不幸的是，这个问题导致在以后任务期间知识转移的解释不足。在此基础上，我们提出了转移而不忘记（TWF），这是在固定的识别同胞网络上建立的混合方法，该方法通过层次损失项不断地传播源域中固有的知识。我们的实验表明，TWF在各种设置上稳步优于其他CL方法，在各种数据集和不同的缓冲尺寸上，平均销售精度的增益为4.81％。
This work investigates the entanglement between Continual Learning (CL) and Transfer Learning (TL). In particular, we shed light on the widespread application of network pretraining, highlighting that it is itself subject to catastrophic forgetting. Unfortunately, this issue leads to the under-exploitation of knowledge transfer during later tasks. On this ground, we propose Transfer without Forgetting (TwF), a hybrid approach building upon a fixed pretrained sibling network, which continuously propagates the knowledge inherent in the source domain through a layer-wise loss term. Our experiments indicate that TwF steadily outperforms other CL methods across a variety of settings, averaging a 4.81% gain in Class-Incremental accuracy over a variety of datasets and different buffer sizes.
