论文标题
过度实现的词典学习中的恢复和概括
Recovery and Generalization in Over-Realized Dictionary Learning
论文作者
论文摘要
在过去的二十年的研究中，字典学习领域收集了大量成功的应用程序，并且只有在与基本词典的模型类中进行优化时，才知道模型恢复的理论保证。这项工作表征了令人惊讶的现象，即通过在较大的超实现模型的空间中搜索字典恢复可以促进。该观察结果是一般的，并且与所使用的特定词典学习算法无关。我们在实践中彻底证明了这一观察结果，并通过将恢复措施与概括界定限制，对这种现象进行了分析。特别是，我们表明，模型恢复可以通过经验风险，模型依赖性数量和概括差距来限制，这反映了我们的经验发现。我们进一步表明，可以采用有效且可证明的正确的蒸馏方法来从过度实现的模型中恢复正确的原子。结果，我们的元算法提供了词典估计值，始终可以更好地恢复地面真相模型。
In over two decades of research, the field of dictionary learning has gathered a large collection of successful applications, and theoretical guarantees for model recovery are known only whenever optimization is carried out in the same model class as that of the underlying dictionary. This work characterizes the surprising phenomenon that dictionary recovery can be facilitated by searching over the space of larger over-realized models. This observation is general and independent of the specific dictionary learning algorithm used. We thoroughly demonstrate this observation in practice and provide an analysis of this phenomenon by tying recovery measures to generalization bounds. In particular, we show that model recovery can be upper-bounded by the empirical risk, a model-dependent quantity and the generalization gap, reflecting our empirical findings. We further show that an efficient and provably correct distillation approach can be employed to recover the correct atoms from the over-realized model. As a result, our meta-algorithm provides dictionary estimates with consistently better recovery of the ground-truth model.
