论文标题
多级训练方法
A Multilevel Approach to Training
论文作者
论文摘要
我们提出了一种基于非线性多级最小化技术的新型训练方法，该方法通常用于解决离散的大规模部分微分方程。我们的多级训练方法通过减少样本数来构建多级层次结构。然后，通过使用较少样品构建的内部训练替代模型来增强原始模型的训练。我们使用一阶一致性方法构建替代模型。这引起了替代模型，其梯度是完整梯度的随机估计器，但与标准随机梯度估计器相比，方差降低。我们说明了基于逻辑回归的机器学习应用程序对机器学习应用程序的收敛行为。与牛顿的亚采样和降低方法的比较证明了我们多级方法的效率。
We propose a novel training method based on nonlinear multilevel minimization techniques, commonly used for solving discretized large scale partial differential equations. Our multilevel training method constructs a multilevel hierarchy by reducing the number of samples. The training of the original model is then enhanced by internally training surrogate models constructed with fewer samples. We construct the surrogate models using first-order consistency approach. This gives rise to surrogate models, whose gradients are stochastic estimators of the full gradient, but with reduced variance compared to standard stochastic gradient estimators. We illustrate the convergence behavior of the proposed multilevel method to machine learning applications based on logistic regression. A comparison with subsampled Newton's and variance reduction methods demonstrate the efficiency of our multilevel method.
