论文标题
简约的无监督学习，稀疏的歧管变换
Minimalistic Unsupervised Learning with the Sparse Manifold Transform
论文作者
论文摘要
我们描述了一种简约和可解释的方法，用于无监督学习，而无需诉诸数据增强，超参数调整或其他工程设计，该方法可实现与SOTA SSL方法接近的性能。我们的方法利用了稀疏的歧管变换，该变换统一了稀疏的编码，多种多样的学习和慢速特征分析。通过单层确定性稀疏歧管变换，可以在MNIST上获得99.3％的KNN TOP-1精度，CIFAR-10上的knn Top-1精度为81.1％，CIFAR-100上的knn Top-1精度为53.2％。借助简单的灰度增强，该型号在CIFAR-10上获得了83.2％的KNN TOP-1精度，而CIFAR-100获得了57％。这些结果显着缩小了简单的“白盒”方法与SOTA方法之间的差距。此外，我们提供可视化以解释如何形成无监督的表示转换。所提出的方法紧密连接到潜在的自我监督方法，可以作为最简单的Vicreg形式。尽管我们简单的建设性模型和SOTA方法之间仍然存在较小的性能差距，但证据表明，这是实现无监督学习的原则性和白色框方法的有希望的方向。
We describe a minimalistic and interpretable method for unsupervised learning, without resorting to data augmentation, hyperparameter tuning, or other engineering designs, that achieves performance close to the SOTA SSL methods. Our approach leverages the sparse manifold transform, which unifies sparse coding, manifold learning, and slow feature analysis. With a one-layer deterministic sparse manifold transform, one can achieve 99.3% KNN top-1 accuracy on MNIST, 81.1% KNN top-1 accuracy on CIFAR-10 and 53.2% on CIFAR-100. With a simple gray-scale augmentation, the model gets 83.2% KNN top-1 accuracy on CIFAR-10 and 57% on CIFAR-100. These results significantly close the gap between simplistic "white-box" methods and the SOTA methods. Additionally, we provide visualization to explain how an unsupervised representation transform is formed. The proposed method is closely connected to latent-embedding self-supervised methods and can be treated as the simplest form of VICReg. Though there remains a small performance gap between our simple constructive model and SOTA methods, the evidence points to this as a promising direction for achieving a principled and white-box approach to unsupervised learning.
