perf: Copy for Signal/Severity, TF-IDF single-pass, deferred schema paths, shared test helpers#61
Merged
Merged
Conversation
…aths, shared test helpers
- Signal and Severity now derive Copy — eliminates .clone() in the hot-path
scan_with_automaton (called for every tool) and in tfidf scan_tfidf_with.
- TF-IDF vocab-overlap guard merged into the term-frequency counts pass:
previously two separate O(tokens) loops; now one pass builds counts and
tracks vocab_hits simultaneously, enabling earlier exit with no extra work.
- scan_schema path allocation deferred: format!("{path}.{key}") is now only
called when a content-bearing key is found; leaf structural scalars
("type": "string", "format": "email") skip the allocation entirely.
- tool() and tool_no_desc() moved from inline description.rs tests to
testutil.rs so all scanner test modules can share them without duplication.
- Remove misleading #[allow(dead_code)] from EmbeddedInstruction — it IS used
extensively in response.rs; the annotation was a false suppression.
- bench/README.md updated with v0.11 detection numbers (440/485 = 90.7%),
accurate AC pattern count (161), performance notes section, and corrected
embedded_instruction signal entry.
https://claude.ai/code/session_01G4f8mN9SeSHSGY1dWfFzih
- Overall detection rate: 84.7% → 90.7% (440/485) - Per-paradigm: T1 77.9%→84.4%, T2 79.7%→84.6%, T3 91.1%→97.7% - Pass count: three → four (TF-IDF pass added in v0.10) - AC pattern count: 155 → 161 - Roadmap: mark v0.10 and v0.11 ✅ Done, rename v0.11 GitHub Action → v0.11a - Upcoming milestone detail updated to reflect completed vs planned work https://claude.ai/code/session_01G4f8mN9SeSHSGY1dWfFzih
…restore risk-category breakdown Rename throughout fixtures, scripts, source comments, and docs: Template-1 → Unrelated Prerequisite (_meta.paradigm: "unrelated-prerequisite") Template-2 → Fake Enabling Prerequisite (_meta.paradigm: "fake-enabling-prerequisite") Template-3 → Argument Hijacking (_meta.paradigm: "argument-hijacking") Files updated: bench/mcptox_actual.json, bench/mcptox_representative.json, bench/run.sh, bench/regenerate_actual.py, bench/README.md, README.md, src/fuzzer/description.rs, src/fuzzer/mod.rs, src/fuzzer/tfidf.rs, src/corpus/loader.rs, corpus/tool_poisoning/TPA-013/014/015.json. bench/run.sh now outputs a "By risk category" breakdown for the actual fixture (all 485 tools carry _meta.risk_category — no regeneration step needed). bench/README.md and README.md updated with current v0.11 risk-category numbers. https://claude.ai/code/session_01G4f8mN9SeSHSGY1dWfFzih
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
$(cat <<'EOF'
Summary
SignalandSeverityderiveCopy— eliminates.clone()inscan_with_automaton(called for every tool scan) andscan_tfidf_with. Both are unit-variant enums;Copyis the correct trait.countsHashMap is built while simultaneously trackingvocab_hits, so the early-exit fires with no redundant work.scan_schema()previously calledformat!("{path}.{key}")for every key in every schema object, including structural leaf scalars that immediately returnvec[]. Path string is now only allocated when a content-bearing key (description,title,enum, etc.) is encountered.tool()andtool_no_desc()moved from inlinedescription.rstests tosrc/testutil.rs(the designated home for shared test infrastructure per CLAUDE.md), eliminating duplication for any future scanner test modules.#[allow(dead_code)]onEmbeddedInstruction— it is used extensively inresponse.rs; the annotation was incorrect suppression.bench/README.md— updated with v0.11 detection numbers (440/485 = 90.7%, up from 432/485 = 89.0%), accurate AC pattern count (161), a new Performance Notes section documenting each optimization, and corrected signal table entry forembedded_instruction.Detection numbers (v0.11)
Test plan
cargo test— all 210 tests passcargo clippy -- -D warnings— zero warningscargo build --release— builds cleanly./bench/run.sh— 440/485 actual, 44/44 representative, 0/20 FPhttps://claude.ai/code/session_01G4f8mN9SeSHSGY1dWfFzih
EOF
)
Generated by Claude Code