Skip to content

v0.2.0 — M2 Report understanding

Choose a tag to compare

View all tags
@SelfishCoconut SelfishCoconut released this 14 Jul 09:17
· 138 commits to main since this release
v0.2.0
72ed61a
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

M2 — Report understanding

Second milestone: turn a real penetration-test PDF into schema-validated findings, with a model-agnostic LLM backend. Tagged at the M2 close-out commit (72ed61a); the already-merged M3 work (FR-04, FR-05) is intentionally excluded and ships in v0.3.0.

Features

  • FR-01 — PDF report ingestion (#6, #42): src/revalid/pdf.py (pdfplumber, ADR-0007) extracts per-page/full text plus best-effort finding candidates and fails closed on non-PDF/corrupt/no-text input. Validated on the real OWASP Juice Shop evaluation report (Nozipho Mthimunye's public write-up, credited and kept local): 11 pages → 8 clean finding candidates with no manual preprocessing.
  • FR-03 — LLM finding extraction (#8, #45): src/revalid/extract.py (Pydantic AI, ADR-0009) — per-candidate agent with a list[ExtractedFinding] schema gate; invalid output is flagged, not persisted; extraction lineage kept for the audit trail (NFR-02). make demo-extract runs the FR-01→FR-03 pipeline.
  • FR-13 — Model-agnostic LLM config (#18, #47): src/revalid/llm.py (ADR-0010) — the backend is selected by REVALID_LLM_MODEL (default Claude; ollama:<model> + OLLAMA_BASE_URL for a local fallback). Switching is configuration-only; a live-Ollama system test skips when no server is reachable.

Milestone acceptance (Done-when)

The real Juice Shop PDF yields ≥90% well-formed findings — met: a full FR-01→FR-03 run on the local ollama:qwen3.5:9b backend produced 8/8 (100%) well-formed findings; the 4th of an earlier synthetic set was correctly held back by the schema gate on a weaker model.

Decisions ratified

ADRs 0007 (pdfplumber), 0009 (LLM extraction), 0010 (model-agnostic config) ratified proposed → accepted as part of this close-out (docs PR #52 on main; the ratification commit post-dates this tag, so the tree at v0.2.0 still shows them proposed).

Full range: v0.1.0..v0.2.0 — see the commit list below.

Assets 2
Loading