论文标题
法律文件摘要的评估框架
An Evaluation Framework for Legal Document Summarization
论文作者
论文摘要
法律从业人员必须经过众多漫长的法律案件程序，以实现各种类别的实践，例如土地纠纷，腐败等。因此，重要的是要汇总这些文件，并确保摘要包含与此案类别相匹配的短语。据我们所知，没有评估指标可以根据其意图评估摘要。我们提出了一个基于自动意图的摘要指标，与其他自动化指标（如BLEU，Rouge-L等）相比，它与人类评估有了更好的一致性，就人为满意度而言。我们还通过注释法律文档中的意图短语来策划数据集，并展示有关如何自动化该系统的概念证明。此外，所有要生成可重现结果的代码和数据都可以在GitHub上获得。
A law practitioner has to go through numerous lengthy legal case proceedings for their practices of various categories, such as land dispute, corruption, etc. Hence, it is important to summarize these documents, and ensure that summaries contain phrases with intent matching the category of the case. To the best of our knowledge, there is no evaluation metric that evaluates a summary based on its intent. We propose an automated intent-based summarization metric, which shows a better agreement with human evaluation as compared to other automated metrics like BLEU, ROUGE-L etc. in terms of human satisfaction. We also curate a dataset by annotating intent phrases in legal documents, and show a proof of concept as to how this system can be automated. Additionally, all the code and data to generate reproducible results is available on Github.
