Named Entity Recognition in Historical Legal Texts: A Comparative Evaluation of Rule-Based, Neural, and Hybrid Models on 19th-Century British Records

Abstract

This study investigates the challenges and methodological approaches to named entity recognition (NER) in historical legal documents, with a focus on nineteenth-century British court records, land registers, and parliamentary petitions. Situated at the intersection of computational linguistics, digital humanities, and legal corpus linguistics, the research evaluates the effectiveness of contemporary NER systems when applied to diachronic and domain-specific textual data. A manually annotated corpus of 200,000 words of historical legal text was constructed to serve as the evaluation benchmark. Five NER approaches were tested: spaCy’s out-of-the-box model, a BERT-based neural model, a fine-tuned legal-domain NER system, a rule-based system, and a hybrid rule-based/machine learning architecture. Performance was assessed using precision, recall, and F1-score metrics to enable systematic comparison across models. The results demonstrate that standard off-the-shelf NER systems perform substantially below their expected benchmarks when applied to historical legal language, achieving an average F1-score of 0.58 compared to approximately 0.87 on modern standard datasets. This performance degradation is primarily attributed to archaic lexical forms, inconsistent orthographic conventions, and domain-specific legal phrasing. Fine-tuning models on in-domain historical legal training data significantly improves performance, yielding a mean F1-score of 0.79, indicating the importance of domain adaptation. The hybrid rule-based and machine learning system achieves the highest overall performance, with an F1-score of 0.83, demonstrating the value of integrating linguistic rules with statistical learning methods in low-resource historical settings. The study provides a systematic benchmark for NER in historical legal corpora and proposes a methodological framework for improving entity extraction in archival and legal digital humanities research. It contributes to advancing computational approaches for the analysis of non-standardized historical textual data.