论文标题
logit模型中切线变换算法的统计最佳和稳定性
Statistical optimality and stability of tangent transform algorithms in logit models
论文作者
论文摘要
在原本棘手的非缀合物模型中找到变分近似的系统方法是通过较小的边际可能性来利用凸双重性的一般原理，从而使问题可探讨。尽管这种方法在非缀合物贝叶斯模型的变异推断的背景下很受欢迎，但缺乏统计最佳和算法融合的理论保证。为了关注逻辑回归模型，我们为数据生成过程提供了温和的条件，以推导出非反应上限到变化最佳的风险。我们证明，如果人们认为通过将算法的可能性略有变化，可以使这些假设完全放松，从而提高了分数能力的可能性。接下来，我们利用动力学系统的理论为物流和多项式逻辑回归中的这种算法提供收敛保证。特别是，我们建立了算法的局部渐近稳定性，而没有任何关于数据生成过程的假设。我们探讨了一个涉及半正交设计的特殊情况，在该案例下，获得了全球融合。该理论使用几项数值研究进一步说明。
A systematic approach to finding variational approximation in an otherwise intractable non-conjugate model is to exploit the general principle of convex duality by minorizing the marginal likelihood that renders the problem tractable. While such approaches are popular in the context of variational inference in non-conjugate Bayesian models, theoretical guarantees on statistical optimality and algorithmic convergence are lacking. Focusing on logistic regression models, we provide mild conditions on the data generating process to derive non-asymptotic upper bounds to the risk incurred by the variational optima. We demonstrate that these assumptions can be completely relaxed if one considers a slight variation of the algorithm by raising the likelihood to a fractional power. Next, we utilize the theory of dynamical systems to provide convergence guarantees for such algorithms in logistic and multinomial logit regression. In particular, we establish local asymptotic stability of the algorithm without any assumptions on the data-generating process. We explore a special case involving a semi-orthogonal design under which a global convergence is obtained. The theory is further illustrated using several numerical studies.
