论文标题
随机神经网络中活动的维度
Dimension of activity in random neural networks
论文作者
论文摘要
神经网络是高维非线性动力学系统，通过许多连接单元的协调活动来处理信息。了解生物学和机器学习网络的功能和学习如何需要了解这种协调活动的结构，例如，在单位之间的交叉协方差中包含的信息。自洽动力平均场理论（DMFT）阐明了随机神经网络的几个特征 - 特别是它们可以产生混乱的活动 - 但是，尚未提供使用这种方法对跨协方差的计算。在这里，我们通过两个站点的腔DMFT自兼协方差。我们使用该理论来探测具有独立且相同分布（I.I.D.）耦合的经典随机网络模型中活动协调的时空特征，显示了活动的广泛但分数较低的活动维度和长期的人口级时尺度。我们的公式适用于广泛的单单元动力学，并推广到非i.i.d。耦合。作为后者的一个例子，我们分析了部分对称耦合的情况。
Neural networks are high-dimensional nonlinear dynamical systems that process information through the coordinated activity of many connected units. Understanding how biological and machine-learning networks function and learn requires knowledge of the structure of this coordinated activity, information contained, for example, in cross covariances between units. Self-consistent dynamical mean field theory (DMFT) has elucidated several features of random neural networks -- in particular, that they can generate chaotic activity -- however, a calculation of cross covariances using this approach has not been provided. Here, we calculate cross covariances self-consistently via a two-site cavity DMFT. We use this theory to probe spatiotemporal features of activity coordination in a classic random-network model with independent and identically distributed (i.i.d.) couplings, showing an extensive but fractionally low effective dimension of activity and a long population-level timescale. Our formulae apply to a wide range of single-unit dynamics and generalize to non-i.i.d. couplings. As an example of the latter, we analyze the case of partially symmetric couplings.
