论文标题
GACS-KORNER通用信息变变自动编码器
Gacs-Korner Common Information Variational Autoencoder
论文作者
论文摘要
我们提出了一个通用信息的概念，该概念允许人们量化和分开两个随机变量之间共享的信息与每个信息所独有的信息。我们的共同信息概念是由对功能家族的优化问题定义的，并将Gács-Körner共同信息恢复为特殊情况。重要的是，我们的概念可以使用来自基础数据分布的样品进行经验近似。然后，我们使用传统的变异自动编码器的简单修改来提供一种分区和量化常见和独特信息的方法。从经验上讲，我们证明我们的配方使我们能够学习语义上有意义的常见和独特的变化因素，即使在图像和视频等高维数据上也是如此。此外，在知道地面潜在因素的数据集上，我们表明我们可以准确量化随机变量之间的共同信息。
We propose a notion of common information that allows one to quantify and separate the information that is shared between two random variables from the information that is unique to each. Our notion of common information is defined by an optimization problem over a family of functions and recovers the Gács-Körner common information as a special case. Importantly, our notion can be approximated empirically using samples from the underlying data distribution. We then provide a method to partition and quantify the common and unique information using a simple modification of a traditional variational auto-encoder. Empirically, we demonstrate that our formulation allows us to learn semantically meaningful common and unique factors of variation even on high-dimensional data such as images and videos. Moreover, on datasets where ground-truth latent factors are known, we show that we can accurately quantify the common information between the random variables.
