论文标题
增强SBERT：用于改善双向句子评分任务的双重编码器的数据增强方法
Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks
论文作者
论文摘要
成对句子评分有两种方法：对输入对进行全面注意的跨编码器，而双重编码器则独立地映射每个输入到密集的向量空间。尽管跨编码器通常会取得更高的性能，但对于许多实际用例，它们太慢了。另一方面，双重编码器需要大量的培训数据，并对目标任务进行微调以实现竞争性能。我们提出了一种简单而有效的数据增强策略，称为增强Sbert，在这里我们使用跨编码器标记了一组更大的输入对，以增强双重编码器的训练数据。我们表明，在此过程中，选择句子对对于该方法的成功而言是非平凡的，至关重要。我们在多个任务（内域）以及域适应任务上评估我们的方法。与原始的双重编码器相比，增强的Sbert可在域适应任务中获得多达6分，最多可获得37分。
There are two approaches for pairwise sentence scoring: Cross-encoders, which perform full-attention over the input pair, and Bi-encoders, which map each input independently to a dense vector space. While cross-encoders often achieve higher performance, they are too slow for many practical use cases. Bi-encoders, on the other hand, require substantial training data and fine-tuning over the target task to achieve competitive performance. We present a simple yet efficient data augmentation strategy called Augmented SBERT, where we use the cross-encoder to label a larger set of input pairs to augment the training data for the bi-encoder. We show that, in this process, selecting the sentence pairs is non-trivial and crucial for the success of the method. We evaluate our approach on multiple tasks (in-domain) as well as on a domain adaptation task. Augmented SBERT achieves an improvement of up to 6 points for in-domain and of up to 37 points for domain adaptation tasks compared to the original bi-encoder performance.
