论文标题
镜下下降，在测量空间中具有相对平滑度
Mirror Descent with Relative Smoothness in Measure Spaces, with application to Sinkhorn and EM
论文作者
论文摘要
机器学习中的许多问题可以被提出为在量度的矢量空间上优化凸功能。本文研究了这种无限维度的镜子下降算法的收敛性。通过定向衍生物来定义布雷格曼的差异，我们得出了该方案的收敛，以进行相对平滑和凸的功能对。这样的假设允许处理非平滑功能，例如kullback-leibler（kl）差异。将我们的结果应用于关节分布和KL，我们表明，在连续设置中，Sinkhorn的原始最佳传输的原始迭代对应于镜下下降，并且我们获得了其（子）线性收敛的新证明。我们还表明，期望最大化（EM）始终可以正式写作作为镜下下降。在固定混合物参数时仅对潜在分布进行优化时（对应于Richardson - 肺部卷积方案，在信号处理中 - 我们得出了均方根收敛速率。
Many problems in machine learning can be formulated as optimizing a convex functional over a vector space of measures. This paper studies the convergence of the mirror descent algorithm in this infinite-dimensional setting. Defining Bregman divergences through directional derivatives, we derive the convergence of the scheme for relatively smooth and convex pairs of functionals. Such assumptions allow to handle non-smooth functionals such as the Kullback--Leibler (KL) divergence. Applying our result to joint distributions and KL, we show that Sinkhorn's primal iterations for entropic optimal transport in the continuous setting correspond to a mirror descent, and we obtain a new proof of its (sub)linear convergence. We also show that Expectation Maximization (EM) can always formally be written as a mirror descent. When optimizing only on the latent distribution while fixing the mixtures parameters -- which corresponds to the Richardson--Lucy deconvolution scheme in signal processing -- we derive sublinear rates of convergence.
