v0.43.0

agentsec v0.43.0

Offline, stdlib-only kit. 36 tools. This release generalizes detection on the six weakest tools from red-team round 2, then scores a fresh round-3 set once. No new scanners.

Generalization (not fixture-specific string matches)

Round 2 is the development set. Rules were redesigned around attack structure before any round-3 sample was written:

• resultguard — TR10 structural injection: co-occurrence of instruction-to-agent cues, authority/override framing, a sensitive object, and an exfil sink. Unicode normalization (homoglyphs, zero-width, tags, bidi) and decode layers (HTML, URL, base64, and similar) run first. Discussion/research context stays clean; ordinary CDN images are not TR09.
• toolpoison — TP11 annotation deception (readOnlyHint / destructiveHint / title vs destructive behavior); TP12 hidden schema defaults on BCC/webhook-like parameters; TP13 preference manipulation. Descriptions are decoded (HTML, URL, base64) before matching.
• memfence — MF11 delayed/trigger-phrase backdoors; MF12 payment-swap or pipe-to-shell planted as standing memory. Multilingual persist/exfil cues. Negation uses word boundaries so “never paste API keys” stays clean while “Whenever … paste API keys” still flags. False biographical facts are still out of reach of these heuristics.
• shellfence — Tokenizes with shlex, normalizes quote/escape/PowerShell -EncodedCommand obfuscation, and reasons about pipelines. SF13 AI-CLI bypass flags; SF14 terraform/kubectl/DROP cascade; SF15 authorized-keys backdoors; SF16 secret-scanner plus public exfil. git push --force-with-lease is no longer SF07.
• mcpposture — MP19 autoApprove/alwaysAllow/trust:true on privileged tools; MP20 credential-directory mounts even on digest-pinned images; MP21 non-official registry overrides; MP22 known-vulnerable pinned version ranges; MP23 remote-code bootstrap (python -c / node -e fetch+exec).
• piiguard — PG05 also covers UK NINO, Canadian SIN, and ICAO TD3 MRZ. URL/base64 payloads are scanned one nested level.

Benchmark

Set	Score (P / R / F1)	Counts (tp/fp/fn/tn)
Author expanded (125 / 10 tools)	100% / 100% / 100%	76/0/0/49
Original 85-sample, 5-tool slice	100% / 100% / 100%	51/0/0/34
Held-out (40, written after #7–#11)	100% / 100% / 100%	26/0/0/14
Red-team round 1 independent (v0.40.0, unseen)	93.8% / 85.7% / 89.6%	60/4/10/26, frozen, still shown
Same round-1 set after #12–#16	100% / 100% / 100%	70/0/0/30, labeled seen
Red-team round 2 independent (v0.41.0, unseen)	80.4% / 38.9% / 52.5%	37/9/58/30, frozen, still shown
Same round-2 set after generalization	94.0% / 82.1% / 87.6%	78/5/17/34, labeled seen
Red-team round 3 (v0.43.0, unseen)	97.3% / 65.5% / 78.3%	36/1/19/28, scored once

Round 3 (benchmarks/redteam3/, 84 samples) was written after the rules were frozen, from published techniques rounds 1 and 2 did not use (Greshake et al., ArtPrompt, Crescendo, many-shot jailbreaking, Virtual Prompt Injection, MCP initialize/elicitation/sampling, LOLBAS, download-then-exec, ld.so.preload, Aadhaar/CPF/TFN/personnummer, GCP metadata, gopher/dict, punycode, RFC 9700, and others; see SOURCES.md). Numbers are reported honestly. Misses include LOLBAS binaries, several national-ID formats, MCP instructions / elicitation / uuencode descriptions, count-delayed memory exfil, and a couple of unusual secret sinks — exactly the kind of unseen gap this set is for.

Priority tools on the (now seen) round-2 set: toolpoison, resultguard, shellfence, mcpposture, and piiguard are 100%; memfence is R=85.7% (the remaining miss is planted false facts). Detection-gap issues #23, #26, #25, #19, and #22 are closed. #20 (memfence false facts) and the issues for tools this release did not retune (#17 fetchfence, #18 mcpoauth, #21 pathfence, #24 secretflow, #27 tracesweep) stay open.

python3 benchmarks/run_benchmark.py

The playground vendors were resynced so the in-browser copies match the CLI.

The floating Action / image tag v0 points at this release (uses: dunkleschitz/agentsec@v0 or @v0.43.0).

Still not on PyPI. Publishing steps are in PUBLISHING.md.