Automatic Detection of Stance Markers in Online Political Debate: A Transformer-Based Corpus Study of Reddit and Kialo Discourse

Abstract

This study develops and evaluates an automatic system for detecting stance markers in online political debate forums, addressing both computational and linguistic challenges associated with stance identification in informal and adversarial digital discourse. Grounded in du Bois’s (2007) stance triangle, Somasundaran and Wiebe’s (2010) opinion mining framework, and recent transformer-based natural language processing approaches (Devlin et al., 2019), the research combines corpus annotation, machine learning, and discourse analysis. A 300,000-word corpus of political debate forum posts was compiled from Reddit and Kialo and manually annotated for multiple categories of stance markers, including evidential, affective, contrastive, and evaluative forms. This annotated dataset was used to fine-tune a BERT-based stance detection model, which was subsequently evaluated against a held-out manually annotated test set. The results demonstrate strong system performance, with the model achieving an F1-score of 0.81 for stance marker identification and 0.74 for stance polarity classification, significantly outperforming established baseline approaches. These findings indicate the effectiveness of transformer-based architectures in capturing the nuanced pragmatic and discourse-level features of stance in informal digital environments. Beyond computational performance, the study conducts a qualitative linguistic analysis of the detected stance markers, revealing a distinctive pragmatic ecology in online political debate. This ecology is characterized by a high frequency of evidential and contrastive stance markers, which are used to construct argumentation, challenge opposing views, and establish epistemic authority. In contrast, affective stance markers are comparatively less frequent than in academic or journalistic discourse, suggesting a discourse style oriented more toward rational contestation than emotional expression. The study contributes to computational linguistics, discourse analysis, and opinion mining research by advancing methods for stance detection and by providing a fine-grained linguistic account of stance construction in contemporary online political communication.