论文标题
NICO ++：为域概括提供更好的基准测试
NICO++: Towards Better Benchmarking for Domain Generalization
论文作者
论文摘要
尽管现代深度神经网络在独立和分布式（I.I.D.）数据上取得了显着的表现，但它们仍可能在分发变化下崩溃。最新的域泛化评估方法（DG）采用了对域数量有限的妥协策略。我们提出了一个大规模的基准测试，并具有广泛的标记域名NICO ++，以及更多的理性评估方法，用于全面评估DG算法。为了评估DG数据集，我们提出了两个指标，分别量化协方差转移和概念转移。从数据构建的角度提出了两个新颖的概括界限，以证明有限的概念转移和重大的协变量转移有利于评估能力的概括。通过广泛的实验，NICO ++与当前的DG数据集相比显示了其出色的评估能力及其在减轻模型选择中甲骨文知识泄漏引起的不公平性方面的贡献。
Despite the remarkable performance that modern deep neural networks have achieved on independent and identically distributed (I.I.D.) data, they can crash under distribution shifts. Most current evaluation methods for domain generalization (DG) adopt the leave-one-out strategy as a compromise on the limited number of domains. We propose a large-scale benchmark with extensive labeled domains named NICO++ along with more rational evaluation methods for comprehensively evaluating DG algorithms. To evaluate DG datasets, we propose two metrics to quantify covariate shift and concept shift, respectively. Two novel generalization bounds from the perspective of data construction are proposed to prove that limited concept shift and significant covariate shift favor the evaluation capability for generalization. Through extensive experiments, NICO++ shows its superior evaluation capability compared with current DG datasets and its contribution in alleviating unfairness caused by the leak of oracle knowledge in model selection.
