论文标题
半空间近端随机梯度方法，用于群体正规化问题
Half-Space Proximal Stochastic Gradient Method for Group-Sparsity Regularized Problem
论文作者
论文摘要
在加工学习应用程序中增强模型的解释性，例如特征选择，压缩感应和模型压缩方面的模型可解释性非常重要。但是，对于大规模的随机训练问题，通常难以实现有效的组稀疏性探索。特别是，最新的随机优化算法通常仅生成密集的解决方案。为了克服这一短缺，我们提出了一种随机方法 - 半空间随机投影梯度（HSPG）方法来搜索高组稀疏性的解决方案，同时保持收敛性。 HSPG方法由简单的Prox-SG步骤初始化，依赖于新颖的半空间步骤来大大提高稀疏度。从数值上讲，HSPG通过计算较高组稀疏性，竞争目标值和泛化精度来证明其在深神网络中的优势，例如VGG16，RESNET18和MOBILENETV1。
Optimizing with group sparsity is significant in enhancing model interpretability in machining learning applications, e.g., feature selection, compressed sensing and model compression. However, for large-scale stochastic training problems, effective group sparsity exploration are typically hard to achieve. Particularly, the state-of-the-art stochastic optimization algorithms usually generate merely dense solutions. To overcome this shortage, we propose a stochastic method -- Half-space Stochastic Projected Gradient (HSPG) method to search solutions of high group sparsity while maintain the convergence. Initialized by a simple Prox-SG Step, the HSPG method relies on a novel Half-Space Step to substantially boost the sparsity level. Numerically, HSPG demonstrates its superiority in deep neural networks, e.g., VGG16, ResNet18 and MobileNetV1, by computing solutions of higher group sparsity, competitive objective values and generalization accuracy.
