论文标题
NAS Bench-2011：扩展可再现神经体系结构搜索的范围
NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search
论文作者
论文摘要
在过去的几年中，神经建筑搜索（NAS）在许多应用程序中取得了突破性的成功。可能是时候退后一步，分析NAS领域的好坏方面。不同搜索空间下的各种算法搜索体系结构。这些搜索的体系结构经过不同的设置，例如超参数，数据增强，正则化训练。在比较各种NAS算法的性能时，这会引起可比性问题。 NAS-Bench-101已经成功地减轻了这一问题。在这项工作中，我们建议向NAS-Bench-101：NAS-Bench-2011扩展，其中具有不同的搜索空间，在多个数据集上的结果以及更多的诊断信息。 NAS-Bench-201具有固定的搜索空间，并为几乎所有最新的NAS算法提供了统一的基准测试。我们的搜索空间的设计灵感来自最流行的基于单元格的搜索算法的设计，该算法将单元格表示为DAG。这里的每个边缘都与从预定义的操作集中选择的操作相关联。为了使其适用于所有NAS算法，NAS-Bench-201中定义的搜索空间包括由4个节点和5个相关操作选项生成的所有可能的架构，这总共有15,625个候选人。为三个数据集提供了每个架构候选者的培训日志和性能。这使研究人员可以避免对选定候选人进行不必要的重复培训，并仅专注于搜索算法本身。为每个候选人节省的训练时间也很大程度上提高了许多方法的效率。我们提供其他诊断信息，例如细粒度损失和准确性，这可以为NAS算法的新设计提供灵感。为了进一步的支持，我们已经从许多方面进行了分析，并根据10种NAS算法进行了基准测试。
Neural architecture search (NAS) has achieved breakthrough success in a great number of applications in the past few years. It could be time to take a step back and analyze the good and bad aspects in the field of NAS. A variety of algorithms search architectures under different search space. These searched architectures are trained using different setups, e.g., hyper-parameters, data augmentation, regularization. This raises a comparability problem when comparing the performance of various NAS algorithms. NAS-Bench-101 has shown success to alleviate this problem. In this work, we propose an extension to NAS-Bench-101: NAS-Bench-201 with a different search space, results on multiple datasets, and more diagnostic information. NAS-Bench-201 has a fixed search space and provides a unified benchmark for almost any up-to-date NAS algorithms. The design of our search space is inspired from the one used in the most popular cell-based searching algorithms, where a cell is represented as a DAG. Each edge here is associated with an operation selected from a predefined operation set. For it to be applicable for all NAS algorithms, the search space defined in NAS-Bench-201 includes all possible architectures generated by 4 nodes and 5 associated operation options, which results in 15,625 candidates in total. The training log and the performance for each architecture candidate are provided for three datasets. This allows researchers to avoid unnecessary repetitive training for selected candidate and focus solely on the search algorithm itself. The training time saved for every candidate also largely improves the efficiency of many methods. We provide additional diagnostic information such as fine-grained loss and accuracy, which can give inspirations to new designs of NAS algorithms. In further support, we have analyzed it from many aspects and benchmarked 10 recent NAS algorithms.
