论文标题
我们已经完成了Imagenet吗？
Are we done with ImageNet?
论文作者
论文摘要
是的，否。我们询问Imagenet分类基准的最新进展是否继续代表有意义的概括，或者社区是否已经开始过分地适合其标签程序的特质。因此，我们开发了一个更大的健壮程序，用于收集ImageNet验证集的人类注释。使用这些新标签，我们重新评估了最近提出的Imagenet分类器的准确性，并发现其收益大大比原始标签上报道的要小得多。此外，我们发现原始的成像网标签不再是该独立收集的集合的最佳预测指标，这表明它们在评估视觉模型时的有用性可能即将结束。然而，我们发现我们的注释程序在很大程度上纠正了原始标签中的错误，从而增强了Imagenet作为视觉识别未来研究的有力基准。
Yes, and no. We ask whether recent progress on the ImageNet classification benchmark continues to represent meaningful generalization, or whether the community has started to overfit to the idiosyncrasies of its labeling procedure. We therefore develop a significantly more robust procedure for collecting human annotations of the ImageNet validation set. Using these new labels, we reassess the accuracy of recently proposed ImageNet classifiers, and find their gains to be substantially smaller than those reported on the original labels. Furthermore, we find the original ImageNet labels to no longer be the best predictors of this independently-collected set, indicating that their usefulness in evaluating vision models may be nearing an end. Nevertheless, we find our annotation procedure to have largely remedied the errors in the original labels, reinforcing ImageNet as a powerful benchmark for future research in visual recognition.
