论文标题
使用非平衡响应理论了解复发性神经网络
Understanding Recurrent Neural Networks Using Nonequilibrium Response Theory
论文作者
论文摘要
复发性神经网络（RNN）是大脑启发的模型，广泛用于机器学习，用于分析顺序数据。目前的工作是对使用非平衡统计力学的响应理论进行更深入了解RNN如何处理信号的贡献。对于由输入信号驱动的一类连续时间随机RNN（SRNN），我们为其输出提供了Volterra类型串联表示。该表示形式可解释，并从SRNN体系结构中解散输入信号。该系列的内核是针对完全确定输出的不受干扰的动力学递归定义的相关函数。利用这种表示形式的连接及其对粗糙路径理论的影响，我们确定了一个通用特征 - 响应特征，事实证明，该特征是输入信号的张量产物的签名和自然支持的基础。特别是，我们表明SRNNS只有优化的读数层中的权重，并且隐藏层中的权重保持固定且未优化，可以看作是在与响应功能相关的再现内核Hilbert空间上运行的内核机。
Recurrent neural networks (RNNs) are brain-inspired models widely used in machine learning for analyzing sequential data. The present work is a contribution towards a deeper understanding of how RNNs process input signals using the response theory from nonequilibrium statistical mechanics. For a class of continuous-time stochastic RNNs (SRNNs) driven by an input signal, we derive a Volterra type series representation for their output. This representation is interpretable and disentangles the input signal from the SRNN architecture. The kernels of the series are certain recursively defined correlation functions with respect to the unperturbed dynamics that completely determine the output. Exploiting connections of this representation and its implications to rough paths theory, we identify a universal feature -- the response feature, which turns out to be the signature of tensor product of the input signal and a natural support basis. In particular, we show that SRNNs, with only the weights in the readout layer optimized and the weights in the hidden layer kept fixed and not optimized, can be viewed as kernel machines operating on a reproducing kernel Hilbert space associated with the response feature.
