论文标题
构建具有有限差异的无偏梯度估计器，以进行条件随机优化
Constructing unbiased gradient estimators with finite variance for conditional stochastic optimization
论文作者
论文摘要
我们研究了用于解决条件随机优化问题的随机梯度下降，其中要最小化的目标是通过参数嵌套期望给出的，相对于一个随机变量，与其他随机变量相对于一个随机变量和内部条件期望，给出了外部期望。这种参数嵌套期望的梯度再次表示为嵌套期望，这使得标准嵌套的蒙特卡洛估计量很难公正。在本文中，我们在某些条件下表明，多级蒙特卡洛梯度估计器是公正的，具有有限的方差和有限的预期计算成本，因此从随机优化的标准理论中，对参数（非嵌套）期望的预期直接适用。我们还讨论了一个特殊情况，可以为此构建一个具有有限差异和成本的无偏梯度估计器。
We study stochastic gradient descent for solving conditional stochastic optimization problems, in which an objective to be minimized is given by a parametric nested expectation with an outer expectation taken with respect to one random variable and an inner conditional expectation with respect to the other random variable. The gradient of such a parametric nested expectation is again expressed as a nested expectation, which makes it hard for the standard nested Monte Carlo estimator to be unbiased. In this paper, we show under some conditions that a multilevel Monte Carlo gradient estimator is unbiased and has finite variance and finite expected computational cost, so that the standard theory from stochastic optimization for a parametric (non-nested) expectation directly applies. We also discuss a special case for which yet another unbiased gradient estimator with finite variance and cost can be constructed.
