论文标题
一致性和模型选择最优性的证明
A proof of consistency and model-selection optimality on the empirical Bayes method
论文作者
论文摘要
我们研究了大型自由度模型的高参数推断中最大边际可能性估计值（MMLE）的一致性和最佳性。我们在指数族中进行主要分析，其中自然参数是超参数。首先，当估计可能性和先验中的方差尺度时，我们证明了MMLE对于一般线性模型的一致性。该证明独立于数据的数量比与模型参数的数字比，除了相关的正规化最小二乘模型参数估计值的不足之处，该估计值均非无偏见。其次，我们将证明概括为具有有限数量的超参数的其他模型。我们发现，指数家族中成本函数的广泛特性通常会产生MMLE对于可能性超参数的一致性。此外，我们在渐近上几乎可以肯定地显示MMLE的MMLE，即使在假设的预测分布下，适用于非贵重模型家族的预测分布下，真实的数据分布在模型空间之外。我们的证明使用多参数MMLE在许多模型参数的渐近学中验证了经验贝叶斯法，从而确保了经验 - 杂交 - 内向交叉验证的相同资格。
We study the consistency and optimality of the maximum marginal likelihood estimate (MMLE) in the hyperparameter inference for large-degree-of-freedom models. We perform main analyses within the exponential family, where the natural parameters are hyperparameters. First, we prove the consistency of the MMLE for the general linear models when estimating the scales of variance in the likelihood and prior. The proof is independent of the number ratio of data to model parameters and excepts the ill-posedness of the associated regularized least-square model-parameter estimate that is shown asymptotically unbiased. Second, we generalize the proof to other models with a finite number of hyperparameters. We find that the extensive properties of cost functions in the exponential family generally yield the consistency of the MMLE for the likelihood hyperparameters. Besides, we show the MMLE asymptotically almost surely minimizes the Kullback-Leibler divergence between the prior and true predictive distributions even if the true data distribution is outside the model space under the hypothetical asymptotic normality of the predictive distributions applicable to non-exponential model families. Our proof validates the empirical Bayes method using the hyperparameter MMLE in the asymptotics of many model parameters, ensuring the same qualification for the empirical-cross-entropy cross-validation.
