论文标题
探索上下文单词嵌入和知识图嵌入的组合
Exploring the Combination of Contextual Word Embeddings and Knowledge Graph Embeddings
论文作者
论文摘要
``经典''单词嵌入（例如Word2Vec）已被证明可以根据其分布属性捕获单词的语义。但是，它们代表一个单词可能拥有的不同含义的能力是有限的。这种方法也不能明确编码实体之间的关系，如单词所示。知识库的嵌入（KB）捕获了用单词表示的实体之间的明确关系，但无法直接捕获这些单词的语法属性。据我们所知，最近的研究集中在表示方面，从而增强了一个人的优势。在这项工作中，我们开始使用上下文和KB嵌入在同一级别上探索另一种方法，并提出了两个任务 - 实体键入和一个关系打字任务 - 评估上下文和KB嵌入的性能。我们还评估了上下文和KB嵌入与这两个任务的串联模型，并在第一个任务上获得了最终结果。我们希望我们的工作可以作为在这种方法方向发展的模型和数据集的基础。
``Classical'' word embeddings, such as Word2Vec, have been shown to capture the semantics of words based on their distributional properties. However, their ability to represent the different meanings that a word may have is limited. Such approaches also do not explicitly encode relations between entities, as denoted by words. Embeddings of knowledge bases (KB) capture the explicit relations between entities denoted by words, but are not able to directly capture the syntagmatic properties of these words. To our knowledge, recent research have focused on representation learning that augment the strengths of one with the other. In this work, we begin exploring another approach using contextual and KB embeddings jointly at the same level and propose two tasks -- an entity typing and a relation typing task -- that evaluate the performance of contextual and KB embeddings. We also evaluated a concatenated model of contextual and KB embeddings with these two tasks, and obtain conclusive results on the first task. We hope our work may contribute as a basis for models and datasets that develop in the direction of this approach.
