Skip to content

feat(deps-audit): dependency vulnerability scan via OSV.dev API (closes #158)#162

Merged
Wolfvin merged 2 commits into
mainfrom
feat/issue-158-deps-audit-osv
Jul 3, 2026
Merged

feat(deps-audit): dependency vulnerability scan via OSV.dev API (closes #158)#162
Wolfvin merged 2 commits into
mainfrom
feat/issue-158-deps-audit-osv

Conversation

@Wolfvin

@Wolfvin Wolfvin commented Jul 2, 2026

Copy link
Copy Markdown
Owner

Closes #158

Implements codelens deps-audit <path> — a pure-Python dependency vulnerability scanner that queries the OSV.dev batch API for known CVEs across PyPI, npm, and crates.io ecosystems.

What was done

New files:

  • scripts/commands/deps_audit.py — thin CLI wrapper (follows secrets.py pattern: add_args + execute + register_command)
  • scripts/dep_audit_engine.py — engine module with:
    • Manifest/lock-file parsers for requirements.txt, pyproject.toml, Pipfile, package.json, package-lock.json, yarn.lock, pnpm-lock.yaml, Cargo.toml, Cargo.lock
    • OSV batch API client (POST /v1/querybatch, 1000-query chunks) with retry+backoff on 429/5xx, vuln-detail caching (1 GET per unique vuln ID)
    • Pure-Python CVSS v3.1 base-score computation (no external library) → severity bucket
    • Graph persistence: dependency_vuln nodes + HAS_VULN edges; idempotent re-scan (deletes previous findings for the same source file before re-inserting)
    • Graceful failure: network errors never raise; partial failures surfaced via api_errors field
  • tests/test_deps_audit.py — 47 tests (parsing, offline mode, ecosystem filter, lock-file preference, mocked OSV queries, CVSS vector parsing, severity filter, graph persistence idempotency, edge cases, spec-string helpers)

Schema additions (scripts/graph_model.py):

  • NODE_TYPE_DEPENDENCY_VULN = "dependency_vuln"
  • EDGE_TYPE_HAS_VULN = "HAS_VULN"
  • No DDL change — node_type column is TEXT, accepts new value as-is

Dispatch wiring (scripts/codelens.py):

  • Added deps-audit to _LIST_COMMANDS (smart --top default)
  • Added deps-audit to _SORT_STRATEGIES (sort by severity desc)

Docs sync:

  • sync_command_count.py --apply run — command count 70 → 71, MCP tools 68 → 69
  • README, SKILL.md, SKILL-QUICK.md auto-updated with new count
  • README command table manually extended with deps-audit row
  • SKILL-QUICK Security section manually extended with deps-audit entry

Definition of Done checklist (issue #158)

  • deps-audit command registered in scripts/codelens.py dispatch table
  • Auto-detects requirements.txt, pyproject.toml, Pipfile, package.json, Cargo.toml (+ their lock files for pinned versions)
  • Queries OSV batch API (api.osv.dev/v1/querybatch), handles pagination (1000-query chunks) and rate limits (Retry-After + exponential backoff on 429/5xx)
  • Findings stored in SQLite as dependency_vuln node type (+ HAS_VULN edge to source file)
  • --severity filter: low, medium, high, critical (includes higher severities)
  • --ecosystem filter to limit to one package manager
  • Standard --format support: json, compact, markdown, ai, sarif (auto-added by codelens.py global --format)
  • Zero external binary dependency — pure Python + urllib + json + sqlite3 + tomllib
  • sync_command_count.py --apply run after adding command (count: 71 ✓)
  • SKILL.md / SKILL-QUICK.md / README.md updated with new command

Pre-flight (per _skills/pre-flight/SKILL.md)

  • Identity: Executor·Pragmatist
  • Repo synced: main up-to-date (no rebase conflict)
  • Open PRs checked: no collision (no PR touches deps-audit area)
  • Merged PRs 24h: none
  • Files read before edit: secrets.py, vuln_scan.py, codelens.py dispatch, graph_model.py schema, sync_command_count.py, secrets_engine.py for engine pattern, test_secrets_engine.py for test pattern

Findings (out-of-scope observations, per pre-flight §Temuan Aneh)

  1. CONTEXT.md is stale — says command count is 63, but actual runtime count was 70 before this PR (now 71). The CONTEXT.md table also still lists issues [PROPOSAL] Rule pattern engine — Semgrep-compatible YAML on tree-sitter AST (Layer 2 of #43 hybrid) #46-[FEATURE] ast-grep optional accelerator — bundle binary for ~3x rule matching speedup #68 as "jangan touch" but multiple workers (including this PR) have already worked on Agent UX: Confidence scores, delta output, and dependency graph between findings #5, [FEATURE] Cypher-like graph query engine for MCP tool query_graph #9, [FEATURE] Semantic vector search via semantic_query MCP tool #11, [FEATURE] Cross-repo intelligence: link edges across multiple indexed repos #15, [FEATURE] Architecture Decision Records (ADR) via manage_adr MCP tool #16, [FEATURE] Taint analysis depth — unified cross-file engine + persistence + library approximation + debug trace #49, [FEATURE] Output formatters expansion — junit_xml, emacs, vim, gitlab_sast, gitlab_secrets, jsonl, toon, dataflow-traces #52, [FEATURE] Distribution & packaging overhaul — PyPI + Docker + Homebrew + binary + signing #54, [FEATURE] CI/CD integration suite — baseline scan, diff scan, strict mode, manifest tests, golden snapshots #57, [FEATURE] Homepage + design doc process — VitePress site + docs/design/ + docs/plans/ convention #67, [FEATURE] ast-grep optional accelerator — bundle binary for ~3x rule matching speedup #68. Suggest a separate context-refresh PR.
  2. Existing vuln-scan command overlaps with deps-auditvuln-scan already queries OSV.dev but uses native audit tools (npm audit, cargo audit, pip-audit) and parses more lock-file formats. deps-audit is the pure-Python subset per issue [FEAT] deps-audit command — dependency vulnerability scan via OSV API #158 spec. They coexist intentionally: vuln-scan is heavier (binary deps, cache layer), deps-audit is lightweight (zero binary deps, graph persistence). Future consolidation may make sense — leaving to BOS.
  3. sync_command_count.py reported NO_MATCH for mcp_server.py and test_integration.py — the regex rules in sync_command_count didn't find the count pattern in those files. Not a regression introduced by this PR; flagged for awareness.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

Wolfvin added a commit that referenced this pull request Jul 3, 2026
)

Issue #172 requested verification that sync_command_count.py was run on
PR #162 (deps-audit), not manual count bump.

What was done:
- Rebased feat/issue-158-deps-audit-osv on latest main (39 commits behind)
- Resolved conflicts in 6 doc files (all were command-count numbers):
  * README.md, SKILL.md, SKILL-QUICK.md, pyproject.toml, skill.json,
    scripts/graph_model.py
  * Took main's version for count numbers, then re-applied my changes
    (graph_model NODE_TYPE_DEPENDENCY_VULN/EDGE_TYPE_HAS_VULN constants,
    README deps-audit row, SKILL-QUICK deps-audit entry)
- Ran `python3 scripts/sync_command_count.py --apply`:
  * Command count: 76 (was 71 in original PR — main has moved +5 commands
    from other merged PRs: orient #165, large-file-threshold #163, etc.)
  * MCP tools: 74 (56 static + 18 dynamic)
- Verified `sync_command_count.py --check` reports 'All documentation
  files are in sync with COMMAND_REGISTRY'
- Re-ran deps_audit tests: 47 passed, 0 failed

Definition of Done (issue #172):
- [x] PR #162 rebased on latest main
- [x] sync_command_count.py --apply run — output counts match docs files
- [x] Counts corrected: 71 → 76 commands, MCP tools updated accordingly
- [x] PR #162 is mergeable (no conflicts, tests pass)

Note: sync_command_count.py works fine when invoked directly (it imports
COMMAND_REGISTRY without going through codelens.py main). However,
`codelens.py --command-count` currently crashes with an argparse
'conflicting option string: -f' error — this is a pre-existing bug in
main unrelated to this PR (the `affected` command defines `-f --filter`
which conflicts with the global `--format -f` alias). Flagged in PR
description for BOS awareness but NOT fixed here (out of scope for #172).
@Wolfvin Wolfvin force-pushed the feat/issue-158-deps-audit-osv branch 2 times, most recently from 2fca072 to b0ba5e7 Compare July 3, 2026 03:29
@Wolfvin Wolfvin merged commit 172ad4d into main Jul 3, 2026
0 of 7 checks passed
@Wolfvin Wolfvin deleted the feat/issue-158-deps-audit-osv branch July 3, 2026 03:30
Wolfvin added 2 commits July 3, 2026 10:30
#158)

Add new `deps-audit` command — pure-Python dependency audit that queries
the OSV.dev batch API (`/v1/querybatch`) for known CVEs across three
ecosystems: PyPI, npm, crates.io.

Files added:
- scripts/commands/deps_audit.py — thin CLI wrapper (follows secrets.py pattern)
- scripts/dep_audit_engine.py — engine: manifest/lock parsing, OSV client,
  CVSS v3 severity bucketing, graph persistence, recommendations
- tests/test_deps_audit.py — 47 tests covering parsing, offline mode,
  ecosystem filter, lock-file preference, mocked OSV queries, CVSS vector
  parsing, severity filter, graph persistence (idempotent), edge cases

Schema additions (graph_model.py):
- NODE_TYPE_DEPENDENCY_VULN = 'dependency_vuln'
- EDGE_TYPE_HAS_VULN = 'HAS_VULN'
- No DDL change — node_type column is TEXT, accepts new value as-is

Features per issue #158 DoD:
- Auto-detects requirements.txt / pyproject.toml / Pipfile (PyPI),
  package.json + package-lock.json/yarn.lock/pnpm-lock.yaml (npm),
  Cargo.toml + Cargo.lock (crates.io)
- Lock file preferred over manifest when both present (pinned versions)
- OSV batch API with 1000-query chunks, retry+backoff on 429/5xx
- Vuln detail cache (1 GET per unique vuln ID across the run)
- Severity bucketed from database_specific.severity OR CVSS v3 vector
  (pure-Python CVSS base score computation, no external library)
- --severity filter (includes higher severities — high returns high+critical)
- --ecosystem filter (PyPI / npm / crates.io)
- --offline mode (skip API, return packages-only with status='offline')
- Findings persisted as dependency_vuln graph nodes + HAS_VULN edges;
  idempotent re-scan (deletes previous findings for the same source file)
- Standard --format support (json / markdown / ai / sarif / compact)
- Zero external binary deps — pure stdlib (urllib, json, sqlite3, tomllib)
- Network failures graceful: status stays 'ok', api_errors field surfaces
  partial failures, NEVER raises

sync_command_count.py --apply run: command count 70 -> 71, MCP tools 68 -> 69.
README/SKILL/SKILL-QUICK updated with new command entry.

Tests: 47 passed (deps_audit), 235 passed + 39 skipped (broader regression
sweep across command_registry, command_count, graph_model, persistent_registry,
registry, secrets_engine, vuln_staleness, config_secret_redaction).
)

Issue #172 requested verification that sync_command_count.py was run on
PR #162 (deps-audit), not manual count bump.

What was done:
- Rebased feat/issue-158-deps-audit-osv on latest main (39 commits behind)
- Resolved conflicts in 6 doc files (all were command-count numbers):
  * README.md, SKILL.md, SKILL-QUICK.md, pyproject.toml, skill.json,
    scripts/graph_model.py
  * Took main's version for count numbers, then re-applied my changes
    (graph_model NODE_TYPE_DEPENDENCY_VULN/EDGE_TYPE_HAS_VULN constants,
    README deps-audit row, SKILL-QUICK deps-audit entry)
- Ran `python3 scripts/sync_command_count.py --apply`:
  * Command count: 76 (was 71 in original PR — main has moved +5 commands
    from other merged PRs: orient #165, large-file-threshold #163, etc.)
  * MCP tools: 74 (56 static + 18 dynamic)
- Verified `sync_command_count.py --check` reports 'All documentation
  files are in sync with COMMAND_REGISTRY'
- Re-ran deps_audit tests: 47 passed, 0 failed

Definition of Done (issue #172):
- [x] PR #162 rebased on latest main
- [x] sync_command_count.py --apply run — output counts match docs files
- [x] Counts corrected: 71 → 76 commands, MCP tools updated accordingly
- [x] PR #162 is mergeable (no conflicts, tests pass)

Note: sync_command_count.py works fine when invoked directly (it imports
COMMAND_REGISTRY without going through codelens.py main). However,
`codelens.py --command-count` currently crashes with an argparse
'conflicting option string: -f' error — this is a pre-existing bug in
main unrelated to this PR (the `affected` command defines `-f --filter`
which conflicts with the global `--format -f` alias). Flagged in PR
description for BOS awareness but NOT fixed here (out of scope for #172).
@sonarqubecloud

sonarqubecloud Bot commented Jul 3, 2026

Copy link
Copy Markdown

Wolfvin added a commit that referenced this pull request Jul 3, 2026
)

Issue #172 requested verification that sync_command_count.py was run on
PR #162 (deps-audit), not manual count bump.

What was done:
- Rebased feat/issue-158-deps-audit-osv on latest main (39 commits behind)
- Resolved conflicts in 6 doc files (all were command-count numbers):
  * README.md, SKILL.md, SKILL-QUICK.md, pyproject.toml, skill.json,
    scripts/graph_model.py
  * Took main's version for count numbers, then re-applied my changes
    (graph_model NODE_TYPE_DEPENDENCY_VULN/EDGE_TYPE_HAS_VULN constants,
    README deps-audit row, SKILL-QUICK deps-audit entry)
- Ran `python3 scripts/sync_command_count.py --apply`:
  * Command count: 76 (was 71 in original PR — main has moved +5 commands
    from other merged PRs: orient #165, large-file-threshold #163, etc.)
  * MCP tools: 74 (56 static + 18 dynamic)
- Verified `sync_command_count.py --check` reports 'All documentation
  files are in sync with COMMAND_REGISTRY'
- Re-ran deps_audit tests: 47 passed, 0 failed

Definition of Done (issue #172):
- [x] PR #162 rebased on latest main
- [x] sync_command_count.py --apply run — output counts match docs files
- [x] Counts corrected: 71 → 76 commands, MCP tools updated accordingly
- [x] PR #162 is mergeable (no conflicts, tests pass)

Note: sync_command_count.py works fine when invoked directly (it imports
COMMAND_REGISTRY without going through codelens.py main). However,
`codelens.py --command-count` currently crashes with an argparse
'conflicting option string: -f' error — this is a pre-existing bug in
main unrelated to this PR (the `affected` command defines `-f --filter`
which conflicts with the global `--format -f` alias). Flagged in PR
description for BOS awareness but NOT fixed here (out of scope for #172).
Wolfvin added a commit that referenced this pull request Jul 3, 2026
)

Issue #172 requested verification that sync_command_count.py was run on
PR #162 (deps-audit), not manual count bump.

What was done:
- Rebased feat/issue-158-deps-audit-osv on latest main (39 commits behind)
- Resolved conflicts in 6 doc files (all were command-count numbers):
  * README.md, SKILL.md, SKILL-QUICK.md, pyproject.toml, skill.json,
    scripts/graph_model.py
  * Took main's version for count numbers, then re-applied my changes
    (graph_model NODE_TYPE_DEPENDENCY_VULN/EDGE_TYPE_HAS_VULN constants,
    README deps-audit row, SKILL-QUICK deps-audit entry)
- Ran `python3 scripts/sync_command_count.py --apply`:
  * Command count: 76 (was 71 in original PR — main has moved +5 commands
    from other merged PRs: orient #165, large-file-threshold #163, etc.)
  * MCP tools: 74 (56 static + 18 dynamic)
- Verified `sync_command_count.py --check` reports 'All documentation
  files are in sync with COMMAND_REGISTRY'
- Re-ran deps_audit tests: 47 passed, 0 failed

Definition of Done (issue #172):
- [x] PR #162 rebased on latest main
- [x] sync_command_count.py --apply run — output counts match docs files
- [x] Counts corrected: 71 → 76 commands, MCP tools updated accordingly
- [x] PR #162 is mergeable (no conflicts, tests pass)

Note: sync_command_count.py works fine when invoked directly (it imports
COMMAND_REGISTRY without going through codelens.py main). However,
`codelens.py --command-count` currently crashes with an argparse
'conflicting option string: -f' error — this is a pre-existing bug in
main unrelated to this PR (the `affected` command defines `-f --filter`
which conflicts with the global `--format -f` alias). Flagged in PR
description for BOS awareness but NOT fixed here (out of scope for #172).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEAT] deps-audit command — dependency vulnerability scan via OSV API

1 participant