Skip to content

agent-audit v0.7.0 - False Positive Reduction

Choose a tag to compare

@HeadyZhang HeadyZhang released this 05 Feb 06:24
· 55 commits to master since this release

agent-audit v0.7.0 - False Positive Reduction

Benchmark Results

  • T5 (deepagents): 142 → 88 findings (38% reduction) ✅
  • T9 (crewAI): 713 → 183 findings (74% reduction) ✅
  • OWASP Coverage: 10/10 ✅
  • All 881 tests pass ✅

New Features

1. Dangerous Operation Analyzer

  • New module: analysis/dangerous_operation_analyzer.py
  • Only triggers AGENT-034 when parameters flow to dangerous operations
  • Recognizes safe tool patterns (get_, fetch_, list_, search_, etc.)

2. Framework Internal Path Detection

  • New module: analysis/framework_detector.py
  • Reduces confidence for findings in framework paths (crewai/, langchain_core/)
  • T9 AGENT-004: 286 → 1 (99.6% reduction)

3. Test File Confidence Reduction

  • Returns low confidence (0.30) for test files
  • Prevents false positives from test fixtures and mocks

4. Finding Deduplication

  • Added _deduplicate_findings() in engine.py
  • Removes AGENT-027 when AGENT-010 already fires on same line
  • Prevents duplicate ASI-01 findings

Files Changed

  • analysis/dangerous_operation_analyzer.py (new)
  • analysis/framework_detector.py (new)
  • analysis/semantic_analyzer.py (modified)
  • scanners/python_scanner.py (modified)
  • rules/engine.py (modified)