论文标题
使用贝叶斯学习规则培训限制了二进制突触的限制玻尔兹曼机器
Training Restricted Boltzmann Machines with Binary Synapses using the Bayesian Learning Rule
论文作者
论文摘要
具有低精度突触的受限玻尔兹曼机器（RBMS）具有很高的能源效率。但是，由于突触的离散性质，使用二进制突触的培训RBM具有挑战性。最近，黄提出了一种有效的方法，通过在变化推理框架下使用梯度上升和传递算法的消息的组合来训练用二进制突触训练RBM。但是，需要其他启发式剪裁操作。在此技术说明中，受黄的作品的启发，我们建议使用贝叶斯学习规则的一种替代优化方法，这是一种自然的梯度变异推理方法。与黄的方法相反，我们更新了变异对称Bernoulli分布的自然参数，而不是期望参数。由于自然参数在整个真实域中都采用值，因此不需要额外的剪辑。有趣的是，\ cite {huang2019data}中的算法可以看作是所提出的算法的一阶近似，这证明了其启发式剪辑的疗效。
Restricted Boltzmann machines (RBMs) with low-precision synapses are much appealing with high energy efficiency. However, training RBMs with binary synapses is challenging due to the discrete nature of synapses. Recently Huang proposed one efficient method to train RBMs with binary synapses by using a combination of gradient ascent and the message passing algorithm under the variational inference framework. However, additional heuristic clipping operation is needed. In this technical note, inspired from Huang's work , we propose one alternative optimization method using the Bayesian learning rule, which is one natural gradient variational inference method. As opposed to Huang's method, we update the natural parameters of the variational symmetric Bernoulli distribution rather than the expectation parameters. Since the natural parameters take values in the entire real domain, no additional clipping is needed. Interestingly, the algorithm in \cite{huang2019data} could be viewed as one first-order approximation of the proposed algorithm, which justifies its efficacy with heuristic clipping.
