论文标题
通过反向加强学习学习回顾性知识
Learning Retrospective Knowledge with Reverse Reinforcement Learning
论文作者
论文摘要
我们提出了一种反向增强学习（反向RL）来表示回顾性知识。一般价值功能（GVF）在表示预测知识方面取得了巨大的成功，即回答有关未来结果的问题，例如“如果我们从A到B开车到B中，预期会消耗多少燃料？”。但是，GVFS无法回答诸如“我们希望汽车在$ t $的B上给出多少燃料？”。要回答这个问题，我们需要知道何时有一辆坦克的坦克，以及那辆车如何到达B。由于这些问题强调了过去事件对当前的影响，因此我们将其答案称为回顾性知识。在本文中，我们展示了如何用反向GVF表示回顾性知识，这些知识是通过反向RL训练的。我们从经验上证明了反向GVF在表示学习和异常检测中的实用性。
We present a Reverse Reinforcement Learning (Reverse RL) approach for representing retrospective knowledge. General Value Functions (GVFs) have enjoyed great success in representing predictive knowledge, i.e., answering questions about possible future outcomes such as "how much fuel will be consumed in expectation if we drive from A to B?". GVFs, however, cannot answer questions like "how much fuel do we expect a car to have given it is at B at time $t$?". To answer this question, we need to know when that car had a full tank and how that car came to B. Since such questions emphasize the influence of possible past events on the present, we refer to their answers as retrospective knowledge. In this paper, we show how to represent retrospective knowledge with Reverse GVFs, which are trained via Reverse RL. We demonstrate empirically the utility of Reverse GVFs in both representation learning and anomaly detection.
