论文标题
培训响应法解释了深度神经网络如何学习
The training response law explains how deep neural networks learn
论文作者
论文摘要
深度神经网络是这十年中广泛应用的技术。尽管采用了富有成果的应用，但其背后的机制仍有待阐明。我们使用非常简单的监督学习编码问题来研究学习过程。结果，我们在培训响应中找到了一个简单的定律，该定律描述了神经切线内核。响应由诸如衰减之类的权力定律组成，乘以简单的响应内核。我们可以使用法律构建一个简单的均值动态模型，该模型解释了网络的学习方式。在学习中，沿着内核之间的竞争，输入空间分为子空间。随着迭代的分裂和衰老，网络变得更加复杂，但最终失去了其可塑性。
Deep neural network is the widely applied technology in this decade. In spite of the fruitful applications, the mechanism behind that is still to be elucidated. We study the learning process with a very simple supervised learning encoding problem. As a result, we found a simple law, in the training response, which describes neural tangent kernel. The response consists of a power law like decay multiplied by a simple response kernel. We can construct a simple mean-field dynamical model with the law, which explains how the network learns. In the learning, the input space is split into sub-spaces along competition between the kernels. With the iterated splits and the aging, the network gets more complexity, but finally loses its plasticity.
