论文标题
fastif：可扩展的影响功能，用于有效的模型解释和调试
FastIF: Scalable Influence Functions for Efficient Model Interpretation and Debugging
论文作者
论文摘要
影响功能近似于测试预测的训练数据点的“影响”，并具有多种应用。尽管很受欢迎，但它们的计算成本在模型和培训数据大小上的扩展效果不佳。我们提出Fastif，这是一组简单的修改，以影响功能，从而大大改善其运行时间。我们使用K-Nearest邻居（KNN）将搜索空间缩小到良好候选数据点的子集，确定在估计逆Hessian-vector产品方面最好平衡速度质量权衡的配置，并引入快速并行变体。我们提出的方法达到了约80倍的速度，而与原始影响值高度相关。随着快速影响功能的可用性，我们证明了它们在四个应用中的有用性。首先，我们检查有影响力的数据点是否可以使用模拟性框架“解释”测试时间行为。其次，我们可视化训练和测试数据点之间的影响力相互作用。第三，我们表明我们可以通过对某些有影响力的数据点进行其他微调来纠正模型错误，从而在HANS数据集中提高了训练有素的Multinli模型的准确性2.5％。最后，我们在训练过程中未见的数据点上进行了类似的设置但微调，将模型准确性分别提高了2.8％和1.7％，分别在HANS和ANLI数据集上提高了1.7％。总体而言，我们的快速影响功能可以有效地应用于大型模型和数据集，我们的实验证明了模型解释和纠正模型误差的影响功能的潜力。代码可从https://github.com/salesforce/fast-influence-functions获得
Influence functions approximate the "influences" of training data-points for test predictions and have a wide variety of applications. Despite the popularity, their computational cost does not scale well with model and training data size. We present FastIF, a set of simple modifications to influence functions that significantly improves their run-time. We use k-Nearest Neighbors (kNN) to narrow the search space down to a subset of good candidate data points, identify the configurations that best balance the speed-quality trade-off in estimating the inverse Hessian-vector product, and introduce a fast parallel variant. Our proposed method achieves about 80X speedup while being highly correlated with the original influence values. With the availability of the fast influence functions, we demonstrate their usefulness in four applications. First, we examine whether influential data-points can "explain" test time behavior using the framework of simulatability. Second, we visualize the influence interactions between training and test data-points. Third, we show that we can correct model errors by additional fine-tuning on certain influential data-points, improving the accuracy of a trained MultiNLI model by 2.5% on the HANS dataset. Finally, we experiment with a similar setup but fine-tuning on datapoints not seen during training, improving the model accuracy by 2.8% and 1.7% on HANS and ANLI datasets respectively. Overall, our fast influence functions can be efficiently applied to large models and datasets, and our experiments demonstrate the potential of influence functions in model interpretation and correcting model errors. Code is available at https://github.com/salesforce/fast-influence-functions
