论文标题
神经网络的异步分散学习
Asynchronous Decentralized Learning of a Neural Network
论文作者
论文摘要
在这项工作中，我们利用异步计算框架（即Arock）在分散的场景中学习一个名为“自我大小估计前馈神经网络（SSFN）”的深神经网络。使用该算法，即异步分散的SSFN（DSSFN），我们在某些技术假设下提供了集中的等效解决方案。异步DSSFN通过允许一个节点激活和一个侧交流来放松通信瓶颈，从而大大降低了通信开销，从而提高了学习速度。我们将异步DSSFN与实验结果中的传统同步DSSFN进行了比较，这显示了异步DSSFN的竞争性能，尤其是当通信网络稀疏时。
In this work, we exploit an asynchronous computing framework namely ARock to learn a deep neural network called self-size estimating feedforward neural network (SSFN) in a decentralized scenario. Using this algorithm namely asynchronous decentralized SSFN (dSSFN), we provide the centralized equivalent solution under certain technical assumptions. Asynchronous dSSFN relaxes the communication bottleneck by allowing one node activation and one side communication, which reduces the communication overhead significantly, consequently increasing the learning speed. We compare asynchronous dSSFN with traditional synchronous dSSFN in the experimental results, which shows the competitive performance of asynchronous dSSFN, especially when the communication network is sparse.
