论文标题
可学习的Bernoulli辍学用于贝叶斯深度学习
Learnable Bernoulli Dropout for Bayesian Deep Learning
论文作者
论文摘要
在这项工作中，我们提出了可学习的bernoulli辍学（LBD），这是一种新的模型 - 不合时宜的辍学方案，将辍学率视为与其他模型参数共同优化的参数。通过对Bernoulli辍学的概率建模，我们的方法可以在深层模型中进行更健壮的预测和不确定性定量。特别是，当与变异自动编码器（VAE）结合使用时，LBD可以灵活地半图像后表示，从而导致新的半密度VAE〜（SIVAE）模型。我们使用增强 - 格雷福斯 - 摩尔（ARM）（ARM）（一种无偏见且低相差梯度估计器的辍学参数）解决了对辍学参数的优化。与其他常用的辍学方案相比，我们对一系列任务的实验表明，我们的方法的表现出色。总体而言，LBD可提高图像分类和语义分割的准确性和不确定性估计。此外，使用Sivae，我们可以在协作过滤方面实现最新的性能，以在几个公共数据集上进行隐式反馈。
In this work, we propose learnable Bernoulli dropout (LBD), a new model-agnostic dropout scheme that considers the dropout rates as parameters jointly optimized with other model parameters. By probabilistic modeling of Bernoulli dropout, our method enables more robust prediction and uncertainty quantification in deep models. Especially, when combined with variational auto-encoders (VAEs), LBD enables flexible semi-implicit posterior representations, leading to new semi-implicit VAE~(SIVAE) models. We solve the optimization for training with respect to the dropout parameters using Augment-REINFORCE-Merge (ARM), an unbiased and low-variance gradient estimator. Our experiments on a range of tasks show the superior performance of our approach compared with other commonly used dropout schemes. Overall, LBD leads to improved accuracy and uncertainty estimates in image classification and semantic segmentation. Moreover, using SIVAE, we can achieve state-of-the-art performance on collaborative filtering for implicit feedback on several public datasets.
