论文标题
在连续的时间限制中学习的元学习
Meta Learning in the Continuous Time Limit
论文作者
论文摘要
在本文中，我们建立了普通的微分方程（ODE），该方程（ODE）是模型 - 静态元学习（MAML）的训练动力学的基础。我们对过程的连续时间限制视图消除了梯度下降的手动步长的影响，并将现有的梯度下降训练算法作为特定离散化导致的特殊情况。我们表明，即使相应的MAML损耗是非convex，MAML ODE的线性收敛速率也具有MAML损耗函数的近似固定点。此外，通过对MAML ODE的分析，我们提出了一种新的BI-MAML培训算法，该算法大大减轻了与现有MAML培训方法相关的计算负担。为了补充我们的理论发现，我们进行了经验实验，以展示我们提出的方法在现有工作方面的优越性。
In this paper, we establish the ordinary differential equation (ODE) that underlies the training dynamics of Model-Agnostic Meta-Learning (MAML). Our continuous-time limit view of the process eliminates the influence of the manually chosen step size of gradient descent and includes the existing gradient descent training algorithm as a special case that results from a specific discretization. We show that the MAML ODE enjoys a linear convergence rate to an approximate stationary point of the MAML loss function for strongly convex task losses, even when the corresponding MAML loss is non-convex. Moreover, through the analysis of the MAML ODE, we propose a new BI-MAML training algorithm that significantly reduces the computational burden associated with existing MAML training methods. To complement our theoretical findings, we perform empirical experiments to showcase the superiority of our proposed methods with respect to the existing work.
