Machine Learning for Automated Software Vulnerability Detection

Machine Learning for Software Vulnerability Detection Project Overview

Software vulnerabilities remain one of the largest security risks in modern software systems. Traditional vulnerability detection tools rely heavily on manual rule creation or static analysis techniques that struggle to capture complex semantic patterns in code.

This project explores how machine learning models can automatically detect software vulnerabilities directly from source code by learning patterns associated with vulnerable code structures.

We evaluate different machine learning approaches that use structural representations of code, such as Abstract Syntax Trees (ASTs) and control-flow information, to classify code as vulnerable or safe and potentially identify vulnerability categories such as CWE types.

Our project builds on recent research demonstrating that deep learning models can learn semantic patterns in code that traditional static analysis tools miss.

Research Papers / Techniques Implemented

Our project is based on techniques proposed in the following papers:

Key Papers

VulDeePecker (NDSS 2018) Introduces deep learning for vulnerability detection using code gadgets.

Devign (NeurIPS 2019) Uses graph neural networks to analyze semantic structures in code.

SySeVR (IEEE TDSC 2022) Framework for extracting vulnerability-related code representations.

Chakraborty et al. (ACM Computing Surveys 2021) A comprehensive survey of deep learning methods for vulnerability detection.

Technique We Are Implementing

Our implementation focuses primarily on ideas from Devign and SySeVR, specifically:

Representing source code as Abstract Syntax Trees (ASTs)

Extracting structural features from ASTs

Training machine learning models to classify vulnerabilities

Minimum Viable Product (MVP)

The MVP will be a working pipeline that automatically analyzes source code and predicts whether the code contains a vulnerability.

Pipeline

Collect vulnerability dataset (Devign dataset)

Parse source code into Abstract Syntax Trees

Extract structural features from the AST

Train machine learning models

Predict whether code is vulnerable or safe

Technical Stack Programming Language

Python

Libraries / Tools

Tree-sitter – parse source code into ASTs

Scikit-learn – machine learning models

XGBoost – gradient boosting classifier

NetworkX – graph representations of code

Pandas / NumPy – data processing

Jupyter Notebook – experimentation

Infrastructure

GitHub – version control

Python virtual environments

Machine Learning Models

We will compare multiple models:

Random Forest

XGBoost

Graph-based learning approaches (future extension)

These models will be trained to classify code as:

Vulnerable

Safe

Optional Extension

Predict vulnerability categories (CWE types).

Dataset

We plan to use the Devign dataset, which contains real-world vulnerable and non-vulnerable code examples extracted from open-source repositories.

Dataset Features

Labeled vulnerable functions

Associated CWE categories

Real-world vulnerability patches

Success Criteria

The project will be considered successful if we can demonstrate that our model can:

Automatically analyze source code

Identify vulnerable code patterns

Achieve meaningful detection accuracy

Target Metric

Detecting approximately 70–80% of vulnerabilities in the Devign dataset.

Evaluation Metrics

Accuracy

Precision / Recall

F1 Score

Contribution to the Security Community

Automated vulnerability detection tools can assist developers by identifying security flaws earlier in the development process.

This project contributes to the broader security ecosystem by:

Exploring machine learning approaches to static analysis

Demonstrating automated vulnerability classification

Potentially contributing ideas to developer tooling and security frameworks such as OpenSSF

Project Timeline Contributors Jacob Merrill Responsibilities

Dataset preparation

AST parsing pipeline

Feature extraction

Jake Keleher Responsibilities

Model training

Model evaluation

Experimentation