Releases: zumpious/towards-efficient-complementary-security-analysis-using-large-language-models
Releases · zumpious/towards-efficient-complementary-security-analysis-using-large-language-models
Release list
v1.0.3 – Update CITATION.cff for standardized citation
This patch updates the CITATION.cff file to conform to CFF 1.2.0, fixes YAML syntax issues (quoting fields and removing invalid characters), and adds the type: dataset field for proper citation metadata.
Add CITATION.cff
This release adds a CITATION.cff file to provide standardized citation metadata for the repository.
Zenodo Release
This release is being created now to trigger Zenodo’s GitHub integration—since our initial tag was made before syncing the repo, Zenodo didn’t automatically import it.
v1.0.0 – Initial release of experimental results & notebooks
-
Data (
/data)- OWASP Benchmark (spotbugs_dataset.pkl)
- Source: OWASP Benchmark
- 2,740+ test cases across 11 vulnerability areas, analyzed with SpotBugs + FindSecBugs
- Train split (~80% of findings) for preliminary studies & few-shot example generation
- Test split (~20% of findings) for validation and cross-model comparison
- See datasets.md for full dataset details
- OWASP Benchmark (spotbugs_dataset.pkl)
-
Preliminary Studies (
/data/preliminary_study)- Contextual Information Analysis
- Impact of SpotBugs report vs. CWE database context on LLM assessments
- Conducted on the train split
- README & details
- Prompting Techniques Comparison
- Few-Shot, Chain-of-Thought (CoT), and Self-Consistency (SC) with GPT-3.5 Turbo
- Conducted on the train split
- README & details
- Contextual Information Analysis
-
Main Research Findings (
/data/towards_efficient_complementary_security_analysis)- Evaluation of multiple LLM families (Qwen, GPT, Phi, Llama) on:
- OWASP test split (403 findings)
- Real-world Mnestix dataset (114 findings)
- Full write-up & data
- Evaluation of multiple LLM families (Qwen, GPT, Phi, Llama) on:
-
Supporting Code (
/src)- Few-shot example scripts: few_shot_examples.py
- Prompt templates: prompt_templates.py
Feel free to download the JSON files and open the notebooks to reproduce our analyses!