Skip to content

fix(security-guidance): add exclude_substrings to cut false positives#61373

Open
zhang-liz wants to merge 2 commits into
anthropics:mainfrom
zhang-liz:fix/security-guidance-exclude-substrings
Open

fix(security-guidance): add exclude_substrings to cut false positives#61373
zhang-liz wants to merge 2 commits into
anthropics:mainfrom
zhang-liz:fix/security-guidance-exclude-substrings

Conversation

@zhang-liz

Copy link
Copy Markdown

Summary

The security-guidance plugin's content-based rules use plain substring matching, which produces well-known false positives for several common identifier and call shapes:

This PR adds an optional exclude_substrings field to the SECURITY_PATTERNS schema. A rule matches only if at least one occurrence of its trigger substring is not inside any excluded substring, so:

  • Files containing only excluded shapes are skipped (no warning)
  • Files containing both an excluded shape and a real risk shape still fire on the real one
  • Rules without exclude_substrings behave exactly as before (backward compatible)

Schema follows the design proposed by the issue author in #55464.

Case matrix

Content Trigger Before After
value = ast.literal_eval(raw) eval( flagged not flagged
result = eval(user_input) eval( flagged still flagged
ast.literal_eval(a); eval(b) eval( flagged still flagged (on the real eval(b))
db.exec(schema) exec( flagged not flagged
child_process.exec(cmd) exec( flagged still flagged
\exec(query)`` in markdown exec( flagged not flagged
new Functionality new Function flagged not flagged
new Function('return ' + x) new Function flagged still flagged
import pickle; pickle.dump(o,f) pickle flagged not flagged
import pickle; pickle.load(f) pickle flagged still flagged

Test plan

  • python3 -m unittest plugins/security-guidance/hooks/test_security_reminder_hook.py — 25 tests, all pass locally
  • Path-based GitHub Actions workflow rule still fires
  • Existing content rules with no exclusions (dangerouslySetInnerHTML, document.write, innerHTML, os.system) unchanged

Closes / refs

🤖 Generated with Claude Code

zhang-liz and others added 2 commits May 21, 2026 22:23
Adds an optional `exclude_substrings` field to the SECURITY_PATTERNS
schema. A rule matches only if at least one occurrence of its trigger
substring is NOT inside any excluded substring, so the existing
substring API stays simple while common false-positive shapes can be
suppressed without disabling the rule.

Populated for four rules whose triggers were the most over-matching:

- eval_injection: excludes ast.literal_eval / literal_eval / safe_eval
- child_process_exec: excludes db.exec, stmt.exec, regex .exec(/...),
  and a handful of common doc/code-fence shapes for bare 'exec('
- new_function_injection: excludes 'new Functionality',
  'new FunctionComponent', and related identifier prefixes
- pickle_deserialization: excludes 'import pickle', 'pickle.dump' and
  similar non-deserialization shapes, plus 'pickled' / 'pickle_jar'

Real risk shapes (eval(userInput), child_process.exec, new Function(,
pickle.load) continue to match as before.

Closes anthropics#55464
Refs anthropics#46720

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Covers _exclude_covers_all_matches coverage logic and the four
rules updated in the previous commit, plus a regression set for the
path-based GitHub Actions workflow rule and the unchanged content
rules (dangerouslySetInnerHTML, document.write, innerHTML, os.system).

Run: python3 -m unittest plugins/security-guidance/hooks/test_security_reminder_hook.py

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

security-guidance plugin: eval_injection rule falsely flags ast.literal_eval()

1 participant