论文标题
梯度下降神经网络训练的通用性
Universality of Gradient Descent Neural Network Training
论文作者
论文摘要
已经观察到，神经网络的设计选择通常对于成功优化至关重要。因此，在本文中，我们讨论了一个问题，即是否始终可以重新设计神经网络，以便它可以很好地训练梯度下降。这将产生以下普遍性结果：如果对于给定的网络，有任何算法可以找到分类任务的良好网络权重，那么该网络的扩展就会扩展，可以通过单纯的梯度下降训练来重现这些权重，并且通过单纯的梯度下降训练。该构建不是用于实用计算的，而是关于元学习和相关方法的可能性的一些方向。
It has been observed that design choices of neural networks are often crucial for their successful optimization. In this article, we therefore discuss the question if it is always possible to redesign a neural network so that it trains well with gradient descent. This yields the following universality result: If, for a given network, there is any algorithm that can find good network weights for a classification task, then there exists an extension of this network that reproduces these weights and the corresponding forward output by mere gradient descent training. The construction is not intended for practical computations, but it provides some orientation on the possibilities of meta-learning and related approaches.
