论文标题
错误驱动的输入调制：解决信用分配问题而没有向后通行证
Error-driven Input Modulation: Solving the Credit Assignment Problem without a Backward Pass
论文作者
论文摘要
人工神经网络中的监督学习通常依赖于反向传播，在该反向传播中，权重根据误差函数梯度进行更新，并从输出层到输入层依次传播。尽管这种方法已被证明在广泛的应用领域中有效，但在许多方面缺乏生物学上的合理性，包括体重对称问题，学习对非本地信号的依赖性，错误传播期间神经活动的冻结以及更新锁定问题。已经引入了替代性培训计划，包括标志对称性，反馈对准和直接反馈对齐，但它们总是依靠向后通行证，阻碍了同时解决所有问题的可能性。在这里，我们建议用第二个正向通行证替换向后通行证，其中根据网络的误差调制输入信号。我们表明，这项新颖的学习规则全面解决了上述所有问题，并且可以应用于完全连接和卷积模型。我们测试了有关MNIST，CIFAR-10和CIFAR-100的学习规则。这些结果有助于将生物学原理纳入机器学习。
Supervised learning in artificial neural networks typically relies on backpropagation, where the weights are updated based on the error-function gradients and sequentially propagated from the output layer to the input layer. Although this approach has proven effective in a wide domain of applications, it lacks biological plausibility in many regards, including the weight symmetry problem, the dependence of learning on non-local signals, the freezing of neural activity during error propagation, and the update locking problem. Alternative training schemes have been introduced, including sign symmetry, feedback alignment, and direct feedback alignment, but they invariably rely on a backward pass that hinders the possibility of solving all the issues simultaneously. Here, we propose to replace the backward pass with a second forward pass in which the input signal is modulated based on the error of the network. We show that this novel learning rule comprehensively addresses all the above-mentioned issues and can be applied to both fully connected and convolutional models. We test this learning rule on MNIST, CIFAR-10, and CIFAR-100. These results help incorporate biological principles into machine learning.
