论文标题
重新考虑瓶颈结构，以进行有效的移动网络设计
Rethinking Bottleneck Structure for Efficient Mobile Network Design
论文作者
论文摘要
倒残差块最近主要是移动网络的建筑设计。它通过引入两个设计规则来改变经典的残留瓶颈：学习倒残差和使用线性瓶颈。在本文中，我们重新考虑了这种设计更改的必要性，并发现它可能带来信息丢失和梯度混乱的风险。因此，我们建议对结构进行翻转，并提出一种新型的瓶颈设计，称为“沙胶块”，该设计在较高的维度下执行身份映射和空间转换，从而有效地减轻了信息损失和梯度混淆。广泛的实验表明，与普遍的信念不同，这种瓶颈结构比移动网络倒置的结构更有益。在ImageNet分类中，通过简单地用我们的沙胶块替换倒的残留块而不增加参数和计算，分类精度可以提高比MobilenEtV2的1.7％以上。在Pascal VOC 2007测试集上，我们观察到对象检测的地图提高了0.9％。我们通过将其添加到神经体系结构搜索方法飞镖的搜索空间中，进一步验证沙胶块的有效性。降低了25％的参数，分类精度比以前的飞镖模型提高了0.13％。代码可以在以下位置找到：https：//github.com/zhoudaquan/rethinking_bottleneck_design。
The inverted residual block is dominating architecture design for mobile networks recently. It changes the classic residual bottleneck by introducing two design rules: learning inverted residuals and using linear bottlenecks. In this paper, we rethink the necessity of such design changes and find it may bring risks of information loss and gradient confusion. We thus propose to flip the structure and present a novel bottleneck design, called the sandglass block, that performs identity mapping and spatial transformation at higher dimensions and thus alleviates information loss and gradient confusion effectively. Extensive experiments demonstrate that, different from the common belief, such bottleneck structure is more beneficial than the inverted ones for mobile networks. In ImageNet classification, by simply replacing the inverted residual block with our sandglass block without increasing parameters and computation, the classification accuracy can be improved by more than 1.7% over MobileNetV2. On Pascal VOC 2007 test set, we observe that there is also 0.9% mAP improvement in object detection. We further verify the effectiveness of the sandglass block by adding it into the search space of neural architecture search method DARTS. With 25% parameter reduction, the classification accuracy is improved by 0.13% over previous DARTS models. Code can be found at: https://github.com/zhoudaquan/rethinking_bottleneck_design.
