论文标题
具有连续和离散先验的VAE潜在代码的各种相互信息最大化框架
Variational Mutual Information Maximization Framework for VAE Latent Codes with Continuous and Discrete Priors
论文作者
论文摘要
学习数据的可解释和分解表示是机器学习研究的关键主题。变分自动编码器（VAE）是一种可扩展的方法，用于学习复杂数据的有向潜可变量模型。它采用了一个清晰可解释的目标，可以轻松优化。但是，该目标不能为潜在变量表示的质量提供明确的衡量标准，这可能导致其质量差。我们为VAE提出了各种相互信息的最大化框架，以解决此问题。与其他方法相比，它提供了一个明确的目标，可最大程度地利用潜在代码和观测之间的相互信息的下限。该目标充当正规化器，迫使VAE不忽略潜在变量，并允许一个人选择其特定组成部分，这对于观测值最有用。最重要的是，提出的框架提供了一种评估固定VAE模型的潜在代码和观察结果之间的相互信息的方法。我们已经在使用高斯和关节高斯和离散的潜在变量的VAE模型上进行了实验。我们的结果表明，所提出的方法加强了潜在的代码与观察之间的关系，并改善了学习的表示形式。
Learning interpretable and disentangled representations of data is a key topic in machine learning research. Variational Autoencoder (VAE) is a scalable method for learning directed latent variable models of complex data. It employs a clear and interpretable objective that can be easily optimized. However, this objective does not provide an explicit measure for the quality of latent variable representations which may result in their poor quality. We propose Variational Mutual Information Maximization Framework for VAE to address this issue. In comparison to other methods, it provides an explicit objective that maximizes lower bound on mutual information between latent codes and observations. The objective acts as a regularizer that forces VAE to not ignore the latent variable and allows one to select particular components of it to be most informative with respect to the observations. On top of that, the proposed framework provides a way to evaluate mutual information between latent codes and observations for a fixed VAE model. We have conducted our experiments on VAE models with Gaussian and joint Gaussian and discrete latent variables. Our results illustrate that the proposed approach strengthens relationships between latent codes and observations and improves learned representations.
