论文标题
Fat Albert：使用基于Bert的语义相似性注意力层找到大文本中的答案
FAT ALBERT: Finding Answers in Large Texts using Semantic Similarity Attention Layer based on BERT
论文作者
论文摘要
基于机器的文本理解一直是自然语言处理的重要研究领域。一旦实现了文本上下文和语义的充分理解，就可以培训深度学习模型来解决大量任务，例如文本摘要，分类和问答。在本文中，我们专注于回答问题，特别是多项选择类型的问题。我们开发了一个基于最先进的变压器网络Bert的模型。此外，我们通过通过语义相似性模型提取最高影响句子来减轻BERT支持大型文本语料库的能力。对我们提出的模型的评估表明，它的表现优于电影QA挑战赛中的主要模型，目前，测试准确性为87.79％，我们目前在排行榜中排名第一。最后，我们讨论了模型的缺点，并提出了可能的改进以克服这些局限性。
Machine based text comprehension has always been a significant research field in natural language processing. Once a full understanding of the text context and semantics is achieved, a deep learning model can be trained to solve a large subset of tasks, e.g. text summarization, classification and question answering. In this paper we focus on the question answering problem, specifically the multiple choice type of questions. We develop a model based on BERT, a state-of-the-art transformer network. Moreover, we alleviate the ability of BERT to support large text corpus by extracting the highest influence sentences through a semantic similarity model. Evaluations of our proposed model demonstrate that it outperforms the leading models in the MovieQA challenge and we are currently ranked first in the leader board with test accuracy of 87.79%. Finally, we discuss the model shortcomings and suggest possible improvements to overcome these limitations.
