论文标题
在嘈杂的设置中概括相似性：dibs现象
Generalizing similarity in noisy setups: the DIBS phenomenon
论文作者
论文摘要
这项工作揭示了数据密度，噪声和相似性学习中的概括能力之间的相互作用。我们认为暹罗神经网络（SNN）是对比度学习的基本形式，并探索两种类型的噪声，这些噪声可能影响SNN，配对标签噪声（PLN）和单标签噪声（SLN）。我们的研究表明，无论训练设置如何，SNNS都表现出双重下降行为，并且噪声进一步加剧了它。我们证明了数据对的密度对于概括至关重要。当SNN在具有相同数量的PLN或SLN的稀疏数据集上训练时，它们具有可比的概括属性。但是，当使用密集的数据集时，PLN病例比过度隔离区域中的SLN概述更糟，导致一种现象，我们称之为密度引起的相似性突破（DIB）。在此制度中，PLN相似性违规变为宏观，将数据集损坏到无法实现完整插值的点，无论模型参数的数量如何。我们的分析还深入研究了相似性学习中在线优化和离线概括之间的对应关系。结果表明，在所有考虑的情况下，在存在标签噪声的情况下，这种等价性失败。
This work uncovers an interplay among data density, noise, and the generalization ability in similarity learning. We consider Siamese Neural Networks (SNNs), which are the basic form of contrastive learning, and explore two types of noise that can impact SNNs, Pair Label Noise (PLN) and Single Label Noise (SLN). Our investigation reveals that SNNs exhibit double descent behaviour regardless of the training setup and that it is further exacerbated by noise. We demonstrate that the density of data pairs is crucial for generalization. When SNNs are trained on sparse datasets with the same amount of PLN or SLN, they exhibit comparable generalization properties. However, when using dense datasets, PLN cases generalize worse than SLN ones in the overparametrized region, leading to a phenomenon we call Density-Induced Break of Similarity (DIBS). In this regime, PLN similarity violation becomes macroscopical, corrupting the dataset to the point where complete interpolation cannot be achieved, regardless of the number of model parameters. Our analysis also delves into the correspondence between online optimization and offline generalization in similarity learning. The results show that this equivalence fails in the presence of label noise in all the scenarios considered.
