论文标题
分布式Q学习，用于随机LQ控制的不明性不确定性
Distributed Q-Learning for Stochastic LQ Control with Unknown Uncertainty
论文作者
论文摘要
本文研究了无限时间范围内有线性二次标准的离散时间随机控制问题。我们专注于一类控制系统的系统矩阵与涉及未知统计属性的随机参数相关联。特别是，我们设计了一个分布式的Q学习算法来应对Riccati方程并得出最佳控制器稳定系统。关键技术是，我们将求解riccati方程的问题转换为推导矩阵方程的零点，并设计分布式随机近似方法以计算零点的估计值。收敛分析证明，分布式Q学习算法最终会收敛到正确的值。一个数值示例阐明了分布式Q学习算法渐近收敛。
This paper studies a discrete-time stochastic control problem with linear quadratic criteria over an infinite-time horizon. We focus on a class of control systems whose system matrices are associated with random parameters involving unknown statistical properties. In particular, we design a distributed Q-learning algorithm to tackle the Riccati equation and derive the optimal controller stabilizing the system. The key technique is that we convert the problem of solving the Riccati equation into deriving the zero point of a matrix equation and devise a distributed stochastic approximation method to compute the estimates of the zero point. The convergence analysis proves that the distributed Q-learning algorithm converges to the correct value eventually. A numerical example sheds light on that the distributed Q-learning algorithm converges asymptotically.
