论文标题
通过追踪梯度下降来估计培训数据影响
Estimating Training Data Influence by Tracing Gradient Descent
论文作者
论文摘要
我们介绍了一种称为Tracin的方法，该方法计算训练示例对模型的预测的影响。这个想法是在使用感兴趣的训练示例时，追踪训练过程中测试点的损失如何变化。我们提供了Tracin的可扩展实现：（a）确切计算的一阶梯度近似，（b）保存的标准训练程序的检查点以及（c）深神经网络的樱桃挑选层。与先前提出的方法相反，特拉辛易于实现。它所需的就是能够使用梯度，检查点和损失功能。该方法是一般的。它适用于使用随机梯度下降训练的任何机器学习模型或IT的变体，建筑，域和任务的不可知论。我们希望该方法在研究和改善培训数据的过程中会广泛使用。
We introduce a method called TracIn that computes the influence of a training example on a prediction made by the model. The idea is to trace how the loss on the test point changes during the training process whenever the training example of interest was utilized. We provide a scalable implementation of TracIn via: (a) a first-order gradient approximation to the exact computation, (b) saved checkpoints of standard training procedures, and (c) cherry-picking layers of a deep neural network. In contrast with previously proposed methods, TracIn is simple to implement; all it needs is the ability to work with gradients, checkpoints, and loss functions. The method is general. It applies to any machine learning model trained using stochastic gradient descent or a variant of it, agnostic of architecture, domain and task. We expect the method to be widely useful within processes that study and improve training data.
