论文标题
复发性神经网络如何混乱？
How Chaotic Are Recurrent Neural Networks?
论文作者
论文摘要
复发性神经网络（RNN）是非线性动态系统。先前的工作认为，RNN可能会遭受混乱现象的困扰，在这种情况下，该系统对初始状态很敏感，并且从长远来看是不可预测的。但是，在本文中，我们进行了系统的经验分析，表明香草或长期记忆（LSTM）RNN在诸如文本生成之类的实际应用中不会在训练过程中表现出混乱的行为。我们的发现表明，未来在这个方向上的工作应解决RNN非线性动力学的另一面。
Recurrent neural networks (RNNs) are non-linear dynamic systems. Previous work believes that RNN may suffer from the phenomenon of chaos, where the system is sensitive to initial states and unpredictable in the long run. In this paper, however, we perform a systematic empirical analysis, showing that a vanilla or long short term memory (LSTM) RNN does not exhibit chaotic behavior along the training process in real applications such as text generation. Our findings suggest that future work in this direction should address the other side of non-linear dynamics for RNN.
