论文标题
模式辅助无监督的学习限制性玻尔兹曼机器
Mode-Assisted Unsupervised Learning of Restricted Boltzmann Machines
论文作者
论文摘要
受限的玻尔兹曼机器（RBMS）是一类强大的生成模型，但是他们的训练需要计算梯度，该梯度与典型损失功能的监督反向传播不同，甚至很难近似。在这里，我们表明，将标准梯度更新与源自RBM基态样本（模式）构建的偏置方向的正确相结合，可显着改善其对传统梯度方法的训练。除了较低的融合相对熵（KL Divergence）外，我们称这种方法称为模式训练，可促进更快的训练和稳定性。除了该方法的稳定性和收敛性的证明外，我们还证明了它在合成数据集上的功效，在该数据集中我们可以准确地计算KL差异以及更大的机器学习标准MNIST。我们建议的模式训练非常广泛，因为它可以与任何给定的梯度方法结合使用，并且很容易扩展到更通用的基于能量的神经网络结构，例如深，卷积和无限制的玻尔兹曼机器。
Restricted Boltzmann machines (RBMs) are a powerful class of generative models, but their training requires computing a gradient that, unlike supervised backpropagation on typical loss functions, is notoriously difficult even to approximate. Here, we show that properly combining standard gradient updates with an off-gradient direction, constructed from samples of the RBM ground state (mode), improves their training dramatically over traditional gradient methods. This approach, which we call mode training, promotes faster training and stability, in addition to lower converged relative entropy (KL divergence). Along with the proofs of stability and convergence of this method, we also demonstrate its efficacy on synthetic datasets where we can compute KL divergences exactly, as well as on a larger machine learning standard, MNIST. The mode training we suggest is quite versatile, as it can be applied in conjunction with any given gradient method, and is easily extended to more general energy-based neural network structures such as deep, convolutional and unrestricted Boltzmann machines.
