论文标题
神经架构搜索轻型非本地网络
Neural Architecture Search for Lightweight Non-Local Networks
论文作者
论文摘要
非本地（NL）块已在各种视觉任务中进行了广泛研究。但是，很少探索将NL块嵌入移动神经网络中的NL块，这主要是由于以下挑战：1）NL块通常具有沉重的计算成本，这使得很难在计算资源受到限制的应用程序中应用，并且2）发现最佳的NL配置是一个开放的问题，可以将NL的NL嵌入到移动神经网络中。我们建议Autonl克服以上两个障碍。首先，我们通过挤压转换操作并结合紧凑的特征来提出一个轻质非本地（Lightnl）块。有了新颖的设计选择，所提出的Lightnl块的计算在400倍上比其常规对应物的不牺牲性能。其次，通过放松训练期间的LightNL块的结构，我们提出了一种有效的神经体系结构搜索算法，以端到端的方式学习Lightnl块的最佳配置。值得注意的是，在典型的移动设置（350m FLOPS）下，使用32 GPU小时，在Imagenet上获得了77.7％的TOP-1精度，显着优于先前的移动模型，包括MobileNetV2（+5.7％），FBNET（+2.8％）和MNASNET（+2.1％）。代码和型号可在https://github.com/liyingwei/autonl上找到。
Non-Local (NL) blocks have been widely studied in various vision tasks. However, it has been rarely explored to embed the NL blocks in mobile neural networks, mainly due to the following challenges: 1) NL blocks generally have heavy computation cost which makes it difficult to be applied in applications where computational resources are limited, and 2) it is an open problem to discover an optimal configuration to embed NL blocks into mobile neural networks. We propose AutoNL to overcome the above two obstacles. Firstly, we propose a Lightweight Non-Local (LightNL) block by squeezing the transformation operations and incorporating compact features. With the novel design choices, the proposed LightNL block is 400x computationally cheaper} than its conventional counterpart without sacrificing the performance. Secondly, by relaxing the structure of the LightNL block to be differentiable during training, we propose an efficient neural architecture search algorithm to learn an optimal configuration of LightNL blocks in an end-to-end manner. Notably, using only 32 GPU hours, the searched AutoNL model achieves 77.7% top-1 accuracy on ImageNet under a typical mobile setting (350M FLOPs), significantly outperforming previous mobile models including MobileNetV2 (+5.7%), FBNet (+2.8%) and MnasNet (+2.1%). Code and models are available at https://github.com/LiYingwei/AutoNL.
