论文标题
通过发展自我修改神经网络的自适应加强学习
Adaptive Reinforcement Learning through Evolving Self-Modifying Neural Networks
论文作者
论文摘要
生物神经网络中看到的自适应学习能力在很大程度上是突触连通性塑料变化出现的自我修改行为的产物。当前的加固学习方法（RL）仅在指定时间间隔反射后适应新的互动，从而阻止了在线适应性的出现。最近证明，通过赋予人工神经编码可塑性的人工神经网络来解决这一问题的最新工作已被证明可以改善使用反向传播训练的简单RL任务的性能，但尚未扩展到更大的问题。在这里，我们研究了一个充满挑战的四足动物领域中的元学习问题，其中四足动物的每条腿都有可能变得不可用，要求代理通过继续与其余四肢进行运动来适应。结果表明，使用自修改的塑料网络进化的代理更能够适应复杂的元学习学习任务，甚至超过了使用基于梯度的算法更新的同一网络，同时花费更少的时间来训练。
The adaptive learning capabilities seen in biological neural networks are largely a product of the self-modifying behavior emerging from online plastic changes in synaptic connectivity. Current methods in Reinforcement Learning (RL) only adjust to new interactions after reflection over a specified time interval, preventing the emergence of online adaptivity. Recent work addressing this by endowing artificial neural networks with neuromodulated plasticity have been shown to improve performance on simple RL tasks trained using backpropagation, but have yet to scale up to larger problems. Here we study the problem of meta-learning in a challenging quadruped domain, where each leg of the quadruped has a chance of becoming unusable, requiring the agent to adapt by continuing locomotion with the remaining limbs. Results demonstrate that agents evolved using self-modifying plastic networks are more capable of adapting to complex meta-learning learning tasks, even outperforming the same network updated using gradient-based algorithms while taking less time to train.
