• Backdoor Detection
• LLMs
• Open Source
• Security
• Software Supply Chain
• Zero-Shot Classification

The XZ backdoor incident, where malicious code was stealthily introduced into a widely used compression library, exposed the growing risks in the open source software supply chain and highlighted the enormous challenges faced by open source maintainers. These maintainers are often burdened with reviewing massive volumes of code contributions, which makes it difficult to manually catch subtle security threats such as backdoors. This thesis proposes to detect suspicious code in open source repositories using AI, helping maintainers by automating the identification of potentially harmful code.
This elaborate will concentrate mostly on Trojan horse attack, the technique used to obfuscate them, and how AI, specifically Large Language Models (LLMs), can be employed to detect those kind of attacks.
The study also explores the use of the Perplexity measure. Initial experiments suggested that Perplexity could effectively flag anomalies. However, the results also revealed its limitations, particularly in distinguishing between truly malicious behavior and unconventional, yet benign, coding styles.
Lastly, state-of-the-art models were evaluated for their ability to detect malicious patterns in codebases using zero-shot classification. Among them, the LLamaguard 3 model, despite its compact size, demonstrated remarkable capabilities, outperforming larger models. It achieved an accuracy of 98.04% and an F1-score of 91.73% on the evaluated dataset.