论文标题
Sumo：潜在变量模型的对数边缘概率的无偏估计
SUMO: Unbiased Estimation of Log Marginal Probability for Latent Variable Models
论文作者
论文摘要
用于训练潜在变量模型的标准变分下限产生大多数感兴趣的偏差估计。我们基于无限序列的随机截断，引入了对数边缘可能性及其对潜在变量模型的梯度的无偏估计器。如果通过编码器架构进行参数化，则可以优化编码器的参数以最大程度地减少该估计器的方差。我们表明，使用我们的估算器训练的模型可提供更好的测试可能性，而基于同一平均计算成本的基于标准的重要性采样方法。该估计量还允许使用潜在变量模型来进行任务，而无偏估计器而不是边缘可能性下限是首选的，例如最大程度地减少反向KL差异并估算得分函数。
Standard variational lower bounds used to train latent variable models produce biased estimates of most quantities of interest. We introduce an unbiased estimator of the log marginal likelihood and its gradients for latent variable models based on randomized truncation of infinite series. If parameterized by an encoder-decoder architecture, the parameters of the encoder can be optimized to minimize its variance of this estimator. We show that models trained using our estimator give better test-set likelihoods than a standard importance-sampling based approach for the same average computational cost. This estimator also allows use of latent variable models for tasks where unbiased estimators, rather than marginal likelihood lower bounds, are preferred, such as minimizing reverse KL divergences and estimating score functions.
