论文标题
打破空间爆炸的诅咒：通过搜索有效的NAS
Breaking the Curse of Space Explosion: Towards Efficient NAS with Curriculum Search
论文作者
论文摘要
神经体系结构搜索（NAS）已成为自动找到有效体系结构的重要方法。为了涵盖所有可能的良好体系结构，我们需要在一个具有数十亿个候选架构的非常大的搜索空间中进行搜索。更重要的是，鉴于较大的搜索空间，我们可能会面临一个非常具有挑战性的太空爆炸问题。但是，由于计算资源的局限性，我们只能品尝一小部分体系结构，这为培训提供了不足的信息。结果，现有方法通常可能产生次优体系结构。为了减轻此问题，我们提出了一种课程搜索方法，该方法从一个较小的搜索空间开始，并逐渐结合了学习的知识，以指导搜索在大空间中。通过提出的搜索策略，我们的课程神经体系结构搜索（CNAS）方法可显着提高搜索效率，并发现比现有NAS方法更好的体系结构。对CIFAR-10和Imagenet的广泛实验证明了该方法的有效性。
Neural architecture search (NAS) has become an important approach to automatically find effective architectures. To cover all possible good architectures, we need to search in an extremely large search space with billions of candidate architectures. More critically, given a large search space, we may face a very challenging issue of space explosion. However, due to the limitation of computational resources, we can only sample a very small proportion of the architectures, which provides insufficient information for the training. As a result, existing methods may often produce suboptimal architectures. To alleviate this issue, we propose a curriculum search method that starts from a small search space and gradually incorporates the learned knowledge to guide the search in a large space. With the proposed search strategy, our Curriculum Neural Architecture Search (CNAS) method significantly improves the search efficiency and finds better architectures than existing NAS methods. Extensive experiments on CIFAR-10 and ImageNet demonstrate the effectiveness of the proposed method.
