论文标题
选择对撞机的偏见以大语言模型
Selection Collider Bias in Large Language Models
论文作者
论文摘要
在本文中，我们激励样本选择背后的因果机制引起的对撞机偏见（选择对撞机偏见），这些偏见可能导致大型语言模型（LLMS）学习在现实世界中无条件独立的实体之间的无条件依赖性。我们表明，选择对撞机偏差可以在未指定的学习任务中放大，尽管难以克服，但我们描述了一种方法来利用所产生的虚假相关性，以确定何时可能不确定其预测模型。我们展示了一个不确定性指标，该指标与Winogender Schemas评估集的扩展版本的性别代词中的任务中的人类不确定性相匹配，并且我们提供了一个在线演示，用户可以将我们的不确定性度量应用于他们自己的文本和模型。
In this paper we motivate the causal mechanisms behind sample selection induced collider bias (selection collider bias) that can cause Large Language Models (LLMs) to learn unconditional dependence between entities that are unconditionally independent in the real world. We show that selection collider bias can become amplified in underspecified learning tasks, and although difficult to overcome, we describe a method to exploit the resulting spurious correlations for determination of when a model may be uncertain about its prediction. We demonstrate an uncertainty metric that matches human uncertainty in tasks with gender pronoun underspecification on an extended version of the Winogender Schemas evaluation set, and we provide an online demo where users can apply our uncertainty metric to their own texts and models.
