论文标题
牙刷在厨房里做什么？变形金刚如何认为我们的世界是结构化的
What do Toothbrushes do in the Kitchen? How Transformers Think our World is Structured
论文作者
论文摘要
现在，基于变压器的模型在NLP中占主导地位。在许多方面，他们的表现都超过了基于静态模型的方法。这一成功反过来促使研究揭示了变形金刚产生的语言模型中的许多偏见。在本文中，我们利用这项对偏见的研究来研究基于变压器的语言模型允许在多大程度上提取有关对象关系的知识（x发生在y; x中。为此，我们将上下文化模型与它们的静态对应物进行了比较。我们将此比较取决于许多相似性测量和分类器的应用。我们的结果是三倍：首先，我们表明模型与不同的相似性度量相结合在允许提取的知识量方面有很大差异。其次，我们的结果表明，相似性度量的表现要比基于分类器的方法差得多。第三，我们表明，令人惊讶的是，静态模型的性能几乎和上下文化模型一样 - 在某些情况下甚至更好。
Transformer-based models are now predominant in NLP. They outperform approaches based on static models in many respects. This success has in turn prompted research that reveals a number of biases in the language models generated by transformers. In this paper we utilize this research on biases to investigate to what extent transformer-based language models allow for extracting knowledge about object relations (X occurs in Y; X consists of Z; action A involves using X). To this end, we compare contextualized models with their static counterparts. We make this comparison dependent on the application of a number of similarity measures and classifiers. Our results are threefold: Firstly, we show that the models combined with the different similarity measures differ greatly in terms of the amount of knowledge they allow for extracting. Secondly, our results suggest that similarity measures perform much worse than classifier-based approaches. Thirdly, we show that, surprisingly, static models perform almost as well as contextualized models -- in some cases even better.
