论文标题
Eagleeye：有效的神经网络修剪的快速子网络评估
EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruning
论文作者
论文摘要
找出经过训练的深神经网络（DNN）的计算冗余部分是修剪算法的关键问题。许多算法尝试通过引入各种评估方法来预测修剪子网络的模型性能。但是它们要么不准确，要么非常复杂。在这项工作中，我们提出了一种称为Eagleeye的修剪方法，其中采用基于自适应批准归一化的简单而有效的评估组件，以揭示不同修剪的DNN结构与最终结算准确性之间的牢固相关性。这种强烈的相关性使我们能够在不实际进行细微调整的情况下快速发现具有最高潜在准确性的修剪候选人。该模块也可以插入和改进一些现有的修剪算法。与我们实验中所有研究的修剪算法相比，Eagleeye可以实现更好的修剪性能。具体来说，为了修剪Mobilenet V1和Resnet-50，Eagleeye的表现都优于将方法比较3.8％。即使在修剪Mobilenet V1紧凑型模型的更具挑战性的实验中，Eagleeye的精度达到了70.9％的最高精度，而总体50％的操作（FLOPS）均已修剪。所有精度结果都是Top-1 Imagenet分类精度。源代码和型号可用于开源社区https://github.com/anonymous47823493/eagleeye。
Finding out the computational redundant part of a trained Deep Neural Network (DNN) is the key question that pruning algorithms target on. Many algorithms try to predict model performance of the pruned sub-nets by introducing various evaluation methods. But they are either inaccurate or very complicated for general application. In this work, we present a pruning method called EagleEye, in which a simple yet efficient evaluation component based on adaptive batch normalization is applied to unveil a strong correlation between different pruned DNN structures and their final settled accuracy. This strong correlation allows us to fast spot the pruned candidates with highest potential accuracy without actually fine-tuning them. This module is also general to plug-in and improve some existing pruning algorithms. EagleEye achieves better pruning performance than all of the studied pruning algorithms in our experiments. Concretely, to prune MobileNet V1 and ResNet-50, EagleEye outperforms all compared methods by up to 3.8%. Even in the more challenging experiments of pruning the compact model of MobileNet V1, EagleEye achieves the highest accuracy of 70.9% with an overall 50% operations (FLOPs) pruned. All accuracy results are Top-1 ImageNet classification accuracy. Source code and models are accessible to open-source community https://github.com/anonymous47823493/EagleEye .
