Skip to content

v1.6.0: optional docs-scan mode + conditional reliability severity

Choose a tag to compare

@matbcassis matbcassis released this 22 Jun 14:47

This release bundles everything since v1.3.0 (1.4.0, 1.5.0, 1.6.0). The headline is optional docs-scan mode: the audit can now map documentation that lives outside the dbt layer — repo docs/, runbooks, READMEs, or a user-pointed source — and report where context duplicates, drifts from the code, goes stale, or points off-repo. dbt-only remains the default; docs mode is opt-in.

Docs-scan mode (1.4.0)

  • New scripts/docs_scan.py — deterministic, near-zero tokens. Reuses the Step 2a inventory, so the project isn't re-parsed. Emits identifier coverage, column drift, multi-home candidates, external pointers, staleness flags, and a hard-capped LLM queue.
  • New phases/docs.md light LLM pass: adjudicates agree/differ on multi-home snippets and doc column claims. Cost scales with flagged rows (tens), not doc volume.
  • Severity follows agent-answer reliability, not evidence provenance: a multi-home contradiction is a Blocker only when the dbt layer has no authoritative definition to fall back on.

Accuracy fixes (1.5.0)

Validated on the tuva healthcare package (1,182 models) plus a full-funnel run:

  • Generic key/value tables no longer mis-read as column claims; duplicate docs deduped; semantic-layer metrics/measures recognized as authoritative homes.
  • Phantom-column false positives suppressed on Jinja column-capture blocks and column-spreading macros; measure_agg_mismatch SUM-of-flag and model_scope_contradiction disclosed-filter false positives fixed.
  • Nested-repo doc auto-discovery; manifest-generated docs detected and skipped for drift.

Conditional reliability severity (1.6.0)

  • The docs-vs-dbt contradiction rule now carries two labels by agent grounding model: dbt-pinned contradictions are a Blocker for a repo-grounded agent (Claude Code / Cursor / repo-RAG, which sees both sides) and Hygiene for a metadata-grounded agent. Every report declares which assumption its severities are read for.

Test suite expanded 79 → 114 across the three versions. dbt-only audits are unaffected — docs mode is entirely opt-in.