论文标题
部分可观测时空混沌系统的无模型预测
KeyCLD: Learning Constrained Lagrangian Dynamics in Keypoint Coordinates from Images
论文作者
论文摘要
我们提出KeyCld，这是一个从图像中学习拉格朗日动态的框架。学到的关键点代表图像中的语义标志性，可以直接代表状态动态。我们表明，将这种状态解释为笛卡尔坐标，再加上明确的自动限制，允许用约束的拉格朗日表达动力学。在图像序列上，KeyCLD经过训练的无监督端到端。我们的方法明确对质量基质，势能和输入矩阵进行建模，从而允许基于能量的控制。我们展示了从DM_Control Pendulum，Cartpole和Acrobot环境中的图像中学习Lagrangian动力学的学习。 KEYCLD可以在这些系统上学习，无论它们是未发动的，不足的还是完全驱动的。训练有素的模型能够产生长期的视频预测，表明动态是准确学习的。我们与lag-vae，lag-cavae和hgn进行了比较，并研究了拉格朗日先验和约束函数的好处。 KeyCLD在所有基准上实现了最高的有效预测时间。此外，非常简单的能量塑形控制器成功地应用于完全致动的系统上。请参阅我们的项目页面以获取代码和其他结果：https：//rdaems.github.io/keycld/
We present KeyCLD, a framework to learn Lagrangian dynamics from images. Learned keypoints represent semantic landmarks in images and can directly represent state dynamics. We show that interpreting this state as Cartesian coordinates, coupled with explicit holonomic constraints, allows expressing the dynamics with a constrained Lagrangian. KeyCLD is trained unsupervised end-to-end on sequences of images. Our method explicitly models the mass matrix, potential energy and the input matrix, thus allowing energy based control. We demonstrate learning of Lagrangian dynamics from images on the dm_control pendulum, cartpole and acrobot environments. KeyCLD can be learned on these systems, whether they are unactuated, underactuated or fully actuated. Trained models are able to produce long-term video predictions, showing that the dynamics are accurately learned. We compare with Lag-VAE, Lag-caVAE and HGN, and investigate the benefit of the Lagrangian prior and the constraint function. KeyCLD achieves the highest valid prediction time on all benchmarks. Additionally, a very straightforward energy shaping controller is successfully applied on the fully actuated systems. Please refer to our project page for code and additional results: https://rdaems.github.io/keycld/
