论文标题
f-Divergence变异推断
f-Divergence Variational Inference
论文作者
论文摘要
本文介绍了$ f $ -Divergence变量推断（$ f $ -VI），该推断将变量推断概括为所有$ f $ -Diverences。通过最小化狡猾的替代$ f $ -Divergence的启动，该代理与$ f $ divergence共享统计一致性，$ f $ -VI框架不仅统一了许多现有的VI方法，例如。 Kullback-Leibler VI，Rényi的$α$ -VI和$χ$ -VI，但为VI提供了标准化工具包，但要与$ f $ divergence family的任意分歧。得出了一般的$ f $ variational Bound，并提供了边缘可能性（或证据）的三明治估计。 $ f $ -vi的开发通过随机优化方案展开，该方案利用了重新聚集技巧，重要性加权和蒙特卡洛的近似；还提出了一种概括众所周知的坐标上升变量推理（CAVI）的平均场近似方案，以$ f $ -VI提出。提供了经验示例，包括各种自动编码器和贝叶斯神经网络，以证明$ f $ -VI的有效性和广泛的适用性。
This paper introduces the $f$-divergence variational inference ($f$-VI) that generalizes variational inference to all $f$-divergences. Initiated from minimizing a crafty surrogate $f$-divergence that shares the statistical consistency with the $f$-divergence, the $f$-VI framework not only unifies a number of existing VI methods, e.g. Kullback-Leibler VI, Rényi's $α$-VI, and $χ$-VI, but offers a standardized toolkit for VI subject to arbitrary divergences from $f$-divergence family. A general $f$-variational bound is derived and provides a sandwich estimate of marginal likelihood (or evidence). The development of the $f$-VI unfolds with a stochastic optimization scheme that utilizes the reparameterization trick, importance weighting and Monte Carlo approximation; a mean-field approximation scheme that generalizes the well-known coordinate ascent variational inference (CAVI) is also proposed for $f$-VI. Empirical examples, including variational autoencoders and Bayesian neural networks, are provided to demonstrate the effectiveness and the wide applicability of $f$-VI.
