论文标题
注意模型的正规化前回答解码器
Regularized Forward-Backward Decoder for Attention Models
论文作者
论文摘要
如今，注意模型是言语识别的流行候选人之一。到目前为止，许多研究主要集中在编码器结构或注意模块上，以增强这些模型的性能。但是，主要忽略解码器。在本文中，我们提出了一种新颖的正则化技术，该技术在训练阶段结合了第二个解码器。该解码器会事先对定期的目标标签进行优化，并通过增加未来上下文的知识来支持培训期间的标准解码器。由于仅在培训期间添加它，因此我们不会更改网络的基本结构或在解码过程中添加复杂性。我们在较小的Tedliumv2和较大的Librispeech数据集上评估了我们的方法，从而对它们都取得了一致的改进。
Nowadays, attention models are one of the popular candidates for speech recognition. So far, many studies mainly focus on the encoder structure or the attention module to enhance the performance of these models. However, mostly ignore the decoder. In this paper, we propose a novel regularization technique incorporating a second decoder during the training phase. This decoder is optimized on time-reversed target labels beforehand and supports the standard decoder during training by adding knowledge from future context. Since it is only added during training, we are not changing the basic structure of the network or adding complexity during decoding. We evaluate our approach on the smaller TEDLIUMv2 and the larger LibriSpeech dataset, achieving consistent improvements on both of them.
