Skip to content

Releases: AbdaullahAG/SemGuard

SemGuard v2.0.0 β€” Expanded Arabic Dataset (807 examples) & AGPLv3

Choose a tag to compare

@AbdaullahAG AbdaullahAG released this 01 Aug 17:07

What's New in v2.0.0

  • 🌍 Arabic Security Dataset expanded from 319 β†’ 807 validated examples across 7 threat categories
  • πŸ€– LLM-as-Judge validation pipeline using three independent judges (GPT-4o, Grok-4, Llama 3.3 70B), achieving Fleiss' ΞΊ = 0.839
  • πŸ“Š Performance jump: mean F1 improved from 0.872 β†’ 0.989 (+13.7%) after retraining
  • πŸ” New finding: quantitative analysis of 527 rejected examples reveals a 98.2% inter-judge disagreement rate for the impersonation category
  • 🧹 Repository restructured: separated dataset-generation dependencies (requirements-dataset.txt) from core runtime (requirements.txt), organized dataset_pipeline/
  • βš–οΈ License changed to AGPLv3 to ensure network-deployed derivatives remain open

Paper

Accepted at IEEE AEECT 2026: "SemGuard: A Triple-Anchor Semantic Security Gateway for Multilingual Prompt Attack Detection in Large Language Models"

Dataset

Full dataset (including judge votes and disagreement corpus): HuggingFace