论文标题
带有一般凸正则化的随机特征模型：具有精确渐近学习曲线的细粒分析
Random Features Model with General Convex Regularization: A Fine Grained Analysis with Precise Asymptotic Learning Curves
论文作者
论文摘要
我们计算具有可分离的强凸正则化或$ \ ell_1 $正则化的最小二乘特征（RF）模型的学习曲线的精确渐近表达式。我们提出了一种新型的多层次应用，对凸高斯最小定理（CGMT）的新型多级应用来克服传统的困难，以找到具有相关数据的随机特征模型的可计算表达式。我们的结果采用可计算的4维标量优化的形式。与以前的结果相反，我们的方法不需要求解通常棘手的近端算子，该操作员随模型参数的数量而定。此外，我们将RF模型的培训和概括错误的普遍性结果扩展到$ \ ell_1 $正则化。特别是，我们证明在温和条件下，具有弹性网或$ \ ell_1 $正则化的随机特征模型在渐近上等同于具有相同第一和第二矩的代孕高斯模型。我们从数值上证明了结果的预测能力，并在实验上表明，即使在非轴注状态下，预测的测试误差也是准确的。
We compute precise asymptotic expressions for the learning curves of least squares random feature (RF) models with either a separable strongly convex regularization or the $\ell_1$ regularization. We propose a novel multi-level application of the convex Gaussian min max theorem (CGMT) to overcome the traditional difficulty of finding computable expressions for random features models with correlated data. Our result takes the form of a computable 4-dimensional scalar optimization. In contrast to previous results, our approach does not require solving an often intractable proximal operator, which scales with the number of model parameters. Furthermore, we extend the universality results for the training and generalization errors for RF models to $\ell_1$ regularization. In particular, we demonstrate that under mild conditions, random feature models with elastic net or $\ell_1$ regularization are asymptotically equivalent to a surrogate Gaussian model with the same first and second moments. We numerically demonstrate the predictive capacity of our results, and show experimentally that the predicted test error is accurate even in the non-asymptotic regime.
