论文标题
可靠的分类变异推断与离散归一化流的混合物
Reliable Categorical Variational Inference with Mixture of Discrete Normalizing Flows
论文作者
论文摘要
越来越多地基于基于梯度的预期来估计的采样的优化。然后处理离散的潜在变量是具有挑战性的，因为采样过程是无可分析的。连续的放松，例如用于分类分布的Gumbel-Softmax，可以实现基于梯度的优化，但不能定义离散观测的有效概率质量。在实践中，选择放松的量很难，并且需要优化一个与所需的目标不符的目标，这会导致问题，尤其是模型具有强大有意义的先验。我们通过将其作为离散归一化流的混合物组成，为分类分布提供了替代可区分的重聚化。它定义了适当的离散分布，允许直接优化证据下限，并且对控制放松的高参数较不敏感。
Variational approximations are increasingly based on gradient-based optimization of expectations estimated by sampling. Handling discrete latent variables is then challenging because the sampling process is not differentiable. Continuous relaxations, such as the Gumbel-Softmax for categorical distribution, enable gradient-based optimization, but do not define a valid probability mass for discrete observations. In practice, selecting the amount of relaxation is difficult and one needs to optimize an objective that does not align with the desired one, causing problems especially with models having strong meaningful priors. We provide an alternative differentiable reparameterization for categorical distribution by composing it as a mixture of discrete normalizing flows. It defines a proper discrete distribution, allows directly optimizing the evidence lower bound, and is less sensitive to the hyperparameter controlling relaxation.
