论文标题
粉末世界：通过丰富的任务分布理解概括的平台
Powderworld: A Platform for Understanding Generalization via Rich Task Distributions
论文作者
论文摘要
强化学习的巨大挑战之一是能够推广到新任务。但是，一般代理需要一组富裕，多样化的任务才能进行训练。为此类任务设计一个“基础环境”很棘手 - 理想的环境将支持一系列新兴现象，表现力的任务空间和快速运行时。为了朝着解决这项研究瓶颈迈出一步，这项工作提出了Powderworld，这是直接在GPU上运行的轻巧但表现力的模拟环境。在粉末世界中，提出了两个激励挑战的分布，一个用于世界建模，另一项用于增强学习。每个都包含手工设计的测试任务以检查概括。实验表明，提高环境的复杂性可以改善世界模型和某些强化学习者的概括，但可能会抑制在高空环境中学习。 Powderworld旨在通过提供相同核心规则引起的各种任务来源来支持概括研究。
One of the grand challenges of reinforcement learning is the ability to generalize to new tasks. However, general agents require a set of rich, diverse tasks to train on. Designing a `foundation environment' for such tasks is tricky -- the ideal environment would support a range of emergent phenomena, an expressive task space, and fast runtime. To take a step towards addressing this research bottleneck, this work presents Powderworld, a lightweight yet expressive simulation environment running directly on the GPU. Within Powderworld, two motivating challenges distributions are presented, one for world-modelling and one for reinforcement learning. Each contains hand-designed test tasks to examine generalization. Experiments indicate that increasing the environment's complexity improves generalization for world models and certain reinforcement learning agents, yet may inhibit learning in high-variance environments. Powderworld aims to support the study of generalization by providing a source of diverse tasks arising from the same core rules.
