论文标题
协变速器的最佳表示
Optimal Representations for Covariate Shift
论文作者
论文摘要
机器学习系统通常会经历培训和测试之间的分配变化。在本文中，我们介绍了一个简单的变分目标，其最佳恰好是所有代表的集合，以确保将风险最小化的人保证对保留贝叶斯预测变量的任何分配变化，例如协变量转移。我们的目标有两个组成部分。首先，表示任务必须保持歧视性，即，某些预测指标必须能够同时最大程度地降低源风险和目标风险。其次，在源和目标之间，表示形式的边际支持必须相同。我们通过设计仅使用未标记的数据和增强来训练强大表示形式的自我监督目标来实现这一实用。我们的目标可以深入了解剪辑的鲁棒性，并进一步改善了剪辑的表示，以在域上实现SOTA结果。
Machine learning systems often experience a distribution shift between training and testing. In this paper, we introduce a simple variational objective whose optima are exactly the set of all representations on which risk minimizers are guaranteed to be robust to any distribution shift that preserves the Bayes predictor, e.g., covariate shifts. Our objective has two components. First, a representation must remain discriminative for the task, i.e., some predictor must be able to simultaneously minimize the source and target risk. Second, the representation's marginal support needs to be the same across source and target. We make this practical by designing self-supervised objectives that only use unlabelled data and augmentations to train robust representations. Our objectives give insights into the robustness of CLIP, and further improve CLIP's representations to achieve SOTA results on DomainBed.
