论文标题
更好的无参数随机优化，并具有用于硬币出生的ode更新
Better Parameter-free Stochastic Optimization with ODE Updates for Coin-Betting
论文作者
论文摘要
无参数的随机梯度下降（PFSGD）算法不需要设置学习率，同时达到最佳理论性能。然而，在实际应用中，调谐随机梯度下降（SGD）和PFSGD之间仍然存在经验差距。在本文中，我们基于截短模型上的连续时间出生的新算法来弥合经验差距。新更新是通过普通微分方程（ODE）的解决方案得出的，并以封闭形式求解。我们从经验上表明，这种新的无参数算法以“最佳默认”学习率优于算法，并且几乎与精心调整的基线的性能相匹配而无需调音。
Parameter-free stochastic gradient descent (PFSGD) algorithms do not require setting learning rates while achieving optimal theoretical performance. In practical applications, however, there remains an empirical gap between tuned stochastic gradient descent (SGD) and PFSGD. In this paper, we close the empirical gap with a new parameter-free algorithm based on continuous-time Coin-Betting on truncated models. The new update is derived through the solution of an Ordinary Differential Equation (ODE) and solved in a closed form. We show empirically that this new parameter-free algorithm outperforms algorithms with the "best default" learning rates and almost matches the performance of finely tuned baselines without anything to tune.
