论文标题
胶囊带有倒点产品注意路由的胶囊
Capsules with Inverted Dot-Product Attention Routing
论文作者
论文摘要
我们为胶囊网络引入了一种新的路由算法，其中仅根据父母的状态与子女投票之间的同意，将儿童胶囊被路由到父母。新机制1）通过倒点 - 产物的关注设计路由； 2）将层归一化作为归一化； 3）用并发的迭代路由替换顺序迭代路由。与先前提出的路由算法相比，我们的方法可以改善基准数据集（例如CIFAR-10和CIFAR-100）的性能，并且它使用强大的CNN（RESNET-18）执行AT-PAR，参数少4倍。在识别叠加数字图像的数字的不同任务中，提出的胶囊模型对CNN的表现相当有利，给定每层相同数量的层和神经元。我们认为，我们的工作增加了将胶囊网络应用于复杂的现实世界任务的可能性。 Our code is publicly available at: https://github.com/apple/ml-capsules-inverted-attention-routing An alternative implementation is available at: https://github.com/yaohungt/Capsules-Inverted-Attention-Routing/blob/master/README.md
We introduce a new routing algorithm for capsule networks, in which a child capsule is routed to a parent based only on agreement between the parent's state and the child's vote. The new mechanism 1) designs routing via inverted dot-product attention; 2) imposes Layer Normalization as normalization; and 3) replaces sequential iterative routing with concurrent iterative routing. When compared to previously proposed routing algorithms, our method improves performance on benchmark datasets such as CIFAR-10 and CIFAR-100, and it performs at-par with a powerful CNN (ResNet-18) with 4x fewer parameters. On a different task of recognizing digits from overlayed digit images, the proposed capsule model performs favorably against CNNs given the same number of layers and neurons per layer. We believe that our work raises the possibility of applying capsule networks to complex real-world tasks. Our code is publicly available at: https://github.com/apple/ml-capsules-inverted-attention-routing An alternative implementation is available at: https://github.com/yaohungt/Capsules-Inverted-Attention-Routing/blob/master/README.md
