论文标题
将神经崩溃和L2归一化与深度神经网络中的分布外检测的改善联系起来
Linking Neural Collapse and L2 Normalization with Improved Out-of-Distribution Detection in Deep Neural Networks
论文作者
论文摘要
我们提出了对标准重新系统体系结构的简单修改-L2在特征空间上的归一化 - 它在先前提出的深层确定性不确定性（DDU）基准上显着改善了分布外（OOD）的性能。我们表明，这种变化还引起了早期神经崩溃（NC），这与更好的OOD性能有关。我们的方法在基准的一小部分训练时间中实现了可比或优越的OOD检测得分和分类精度。此外，它基本上改善了多个随机初始化模型的最坏情况。尽管我们不建议NC是深神经网络（DNN）中OOD行为的唯一机制或全面的解释，但我们认为NC的简单数学和几何结构可以为对未来工作中这种复杂现象的分析提供一个框架。
We propose a simple modification to standard ResNet architectures--L2 normalization over feature space--that substantially improves out-of-distribution (OoD) performance on the previously proposed Deep Deterministic Uncertainty (DDU) benchmark. We show that this change also induces early Neural Collapse (NC), an effect linked to better OoD performance. Our method achieves comparable or superior OoD detection scores and classification accuracy in a small fraction of the training time of the benchmark. Additionally, it substantially improves worst case OoD performance over multiple, randomly initialized models. Though we do not suggest that NC is the sole mechanism or a comprehensive explanation for OoD behaviour in deep neural networks (DNN), we believe NC's simple mathematical and geometric structure can provide a framework for analysis of this complex phenomenon in future work.
