论文标题
关于无线分散学习的沟通延迟
On the Communication Latency of Wireless Decentralized Learning
论文作者
论文摘要
我们考虑一个无线网络，其中包括位于Radius $ r $的圆形区域内的$ N $节点，该网络正在使用其本地数据集来优化分散的学习算法，以优化全球目标函数。为了启用整个网络的梯度交换，我们假设每个节点仅与一组相邻的节点进行通信，这些节点在本身的$ r n^{ - β} $之内，其中$β\ in（0，\ frac {1}} {2}）$。 We use tools from network information theory and random geometric graph theory to show that the communication delay for a single round of exchanging gradients on all the links throughout the network scales as $\mathcal{O}\left(\frac{n^{2-3β}}{β\log n}\right)$, increasing (at different rates) with both the number of nodes and the gradient exchange threshold distance.
We consider a wireless network comprising $n$ nodes located within a circular area of radius $R$, which are participating in a decentralized learning algorithm to optimize a global objective function using their local datasets. To enable gradient exchanges across the network, we assume each node communicates only with a set of neighboring nodes, which are within a distance $R n^{-β}$ of itself, where $β\in(0,\frac{1}{2})$. We use tools from network information theory and random geometric graph theory to show that the communication delay for a single round of exchanging gradients on all the links throughout the network scales as $\mathcal{O}\left(\frac{n^{2-3β}}{β\log n}\right)$, increasing (at different rates) with both the number of nodes and the gradient exchange threshold distance.
