ETD

• dataset curation
• explainable ai
• large language models
• online threat detection
• structured information extraction
• synthetic data ugmentation

The rise of social media has amplified online aggression: while hate speech detection is well studied, threat detection remains relatively underexplored. This thesis aims to address this gap by focusing on the identification of explicit and implicit threats in online content. To this end, a new dataset was developed, as existing resources primarily target hate speech and often lack a consistent definition of threat. Moreover, prior datasets frequently include heterogeneous categories, such as legal or sexual threats, that do not align with the specific notion of threat considered in this work. The final dataset comprises 19,692 entries, encompassing both organic social media messages collected from Twitter, Telegram, and Gab, and a subset of synthetically generated instances, was carefully curated and subjected to a multilabel reannotation process to establish a reliable Gold Standard for threat detection.
The study began by validating the dataset against established baselines, like Perspective API. Then it expanded the investigation to LLMs, testing various prompt engineering techniques across a diverse range of models. Building upon these classification experiments, this research advances toward a structured framework designed to extract the constituent elements of a threat. While the baseline models yield promising results,the proposed methodology demonstrates a superior capacity for refinement, significantly improving upon established benchmarks.
Experimental results demonstrate that the proposed method significantly advances beyond the Encoder-Decoder baseline, raising the F1-score from 0.53 to 0.80. Furthermore, the Information Extraction strategy proves superior to purely generative approaches, outperforming both LLMs and LRMs, which achieved F1-scores of 0.70 and 0.74, respectively. Furthermore, since the proposed solution is built upon Large Language Models, its performance remains inherently sensitive to prompt engineering and hyperparameter tuning. While this dependency on prompt configuration introduces a degree of variability, it simultaneously presents a significant opportunity for context aware explanations of the threats themselves. By explicating the underlying rationale for a classification, this approach serves as a robust foundation for future research in explainable threat detection.