Skip to content

pumllint 0.5.0 — maturity scoring

Choose a tag to compare

View all tags
@fdurieux fdurieux released this 22 Jul 21:38
v0.5.0
a4569f0

Maturity scoring (pumllint score)

pumllint now aggregates its rule findings into a 360° maturity level per diagram — from 1 (Sketchy) to 5 (Generation-ready) — with a prescriptive gap report listing exactly which findings block promotion to the next level.

python -m pumllint score diagrams/ --min-level 4      # CI gate: exit 1 below Level 4
python -m pumllint score diagrams/ --profile codegen  # Level 5 requires this profile
python -m pumllint score diagrams/ --check-syntax     # also run plantuml -checkonly
  • Seven scoring dimensions (semantic, completeness, consistency, traceability, readability, ambiguity + an external syntax gate), every rule annotated in the catalog.
  • Integrity caps C1–C7 — blocker cap, syntax gate, sacrificed-dimension cap, vacuity guards (an empty diagram cannot score), and Level-5 binding to the codegen profile so the claim can't be made without those rules running.
  • Gap reports in text/JSON/SonarQube output: the minimal set of fixes to reach the next level.
  • XD cross-diagram pack: participant identity checked across the whole lint batch (kind/stereotype conflicts, case collisions), majority-wins attribution.

Calibrated and evidence-backed

  • Defaults (K=50, signal-proportional dimension weights, severity ladder) validated against a generated mutation/synthetic/wild corpus — zero monotonicity violations in every configuration tested — and frozen behind golden-score regression tests (SCORING.md §9).
  • The maturity→codegen relationship is measured, not asserted (EVIDENCE.md): across 75 generation runs with an independent LLM judge, maturity scores correlated with generated-code fidelity (r ≈ 0.49), and diagrams below Level 2 degraded sharply — fidelity down by roughly a third, invented business logic doubled. The --min-level gate keeps those diagrams out.

Also in this release

  • Executable spec extended: SCORING.md's Gherkin runs in the test suite alongside RULES.md.
  • Parser tolerates a UTF-8 BOM before @startuml (found by wild-corpus harvesting).
  • Hardened by a 57-agent max-effort review: 15 findings fixed, including cross-diagram attribution for unterminated blocks, a vacuous-pass hole in the CI gate, and shell-splitting for syntax_command strings.
  • Zero-dependency promise preserved: the scorer is stdlib-only; 249 pytest + 178 zero-dep-runner tests green.

Full scoring model: SCORING.md · Rules: RULES.md · Experiment record: EVIDENCE.md

Assets 2
Loading