论文标题
部分可观测时空混沌系统的无模型预测
Stochastic Neural Networks with Infinite Width are Deterministic
论文作者
论文摘要
理论上，这项工作研究了随机神经网络，这是一种主要的神经网络。我们证明，随着优化的随机神经网络的宽度倾向于无穷大，其对训练集的预测差异降低到零。我们的理论证明，在模型中添加随机性可以通过引入平均效果来帮助使模型正规化的共同直觉合理。我们的理论可能与具有辍学和贝叶斯潜在变量模型的神经网络相关的两个常见示例。因此，我们的结果有助于更好地理解随机性如何影响神经网络的学习，并可能为实践问题设计更好的体系结构。
This work theoretically studies stochastic neural networks, a main type of neural network in use. We prove that as the width of an optimized stochastic neural network tends to infinity, its predictive variance on the training set decreases to zero. Our theory justifies the common intuition that adding stochasticity to the model can help regularize the model by introducing an averaging effect. Two common examples that our theory can be relevant to are neural networks with dropout and Bayesian latent variable models in a special limit. Our result thus helps better understand how stochasticity affects the learning of neural networks and potentially design better architectures for practical problems.
