Repository navigation
v3.0.0 — domain expansion + honest-benchmark hardening
spanguard v3.0.0 — domain expansion + honest-benchmark hardening
New
- Corpus v3 (
training/gen_corpus_v3.py): 14,000 docs, 5 languages (en/de/hu/fr/bn), 16 families — adds the 2026 attack surface: MCP tool-description poisoning (MCPTox class), agent skill/config files (Skill-Inject class), buried multi-hop payloads, markdown-image exfiltration, polite social engineering; benign traps follow red-team findings + NotInject-style FP bait. Seed-locked, hash manifest committed. - spanguard-char-v3 (0.79M, HF:
Nabidnur/spanguard-char-v3): extended-domain char guard — 0.4% escape / span F1 0.996 on the five new families vs 11.6% / 0.467 for char-v2; the old-OOD regression (handwritten 4.2% → 12.5%) is published, not hidden. Both models ship. - spanguard-span-mdistil-v2 (135M, HF): corpus-v2 retrain with boundary-weighted loss (×3 at span edges) — public-probe over-strip 34.2% → 18.1%, hand-written over-strip 11.3% → 6.9%, escape trade 29.2% → 33.3% published as the precision/recall shift it is.
- Tool-definition scanning (gateway +
spanguard tools scan|pin): OpenAI tools array / MCP descriptions scanned with rules+heuristics; rug-pull detection via operator-controlled description-hash pin file. (MCPTox: 36.5% avg poisoning success — the gateway now covers the tool-definition surface.) - Typed markers
[stripped:override]/[stripped:exfil]/ … (G): the agent can tell the user what was removed. Config:SPANGUARD_TYPED_MARKERS=0for legacy. - Adaptive red-team loop (
training/redteam_adaptive.py, idea A): LLM attacker mutates payloads against the live guard — first cut: 25 real mutations, 0 escapes in 30 attempts (eval/results/redteam-v3-0300/).
Fixed (external-review hardening)
- Evidence consistency: teacher pass stated as it ran (1,150 calls, $3.91, 0 teacher-only spans →
planted ∪ teacher == planted); latency claims now quote committed p50s (fast 277 ms, char ~2 ms, rules 0.2 ms, horizon-small 87.5 ms); public-deepset reframed as a benign-only over-strip probe (escape column n/a); Wilson 95% CIs on headline numbers (char vs fast non-overlapping). - SPEC: XLM-R 278M → distilbert 135M; test/rules counts match the code (21 tests, 18 rules); Prompt Guard 2 marked license-gated (403) with published figures cited instead of a fake head-to-head.
- Gateway: async httpx client; Bearer-by-default upstream auth (
SPANGUARD_UPSTREAM_AUTH);/guard+/v1/*optional token auth; response tool-message cleaning actually walkschoices[*].message;/healthzreports actually-loaded detectors + degraded reasons (no silent rules-only downgrade). - 21 tests green, ruff 0 errors, CI badge honest.
Positioning
2026 field table (Meta PromptGuard 2 / LlamaFirewall, tool-output Sanitizer, buried-injection report, Presidio PII playbook, NotInject) + stated roadmap (task-conditioned stripping, taint-linked toolguard, conformal thresholds, NotInject/AgentDojo columns).