Skip to content

pumllint 0.18.0 — scoring platform complete: ratchet CI, 5 diagram types, measured evidence, schemas

Choose a tag to compare

@fdurieux fdurieux released this 24 Jul 12:38

Cumulative release: everything since 0.5.0 (thirteen versions), organized by theme. Zero runtime dependencies throughout; Python ≥ 3.11.

Scoring & CI trust

  • Model-set aggregate score (0.6.0) — every report ends with the set verdict: worst per-diagram level (the set is only as trustworthy as its weakest diagram) plus an element-weighted composite. --min-level gates on it.
  • Baseline / ratchet mode (0.6.0) — record today's per-diagram levels once, then fail CI only on regression; --update-baseline accepts the status quo. Trend/delta annotations in the text and json reports (0.7.0).
  • Maturity badge (0.7.0)score -f badge emits shields.io endpoint JSON for the model-set level.
  • GitHub Action + pre-commit hooks (0.8.0) — composite action mirroring the CLI (uses: fdurieux/pumllint@v0.18.0), plus pumllint / pumllint-score hooks; both dogfooded in this repo's CI.
  • HTML architect report (0.15.0)score -f html: a single self-contained page (no scripts, no external requests, deterministic), model-set verdict first, per-diagram cards worst-first with dimension bars, gap report and baseline trends.

Coverage: 42-rule catalog across five diagram types

  • Class diagrams (0.9.0) — parser + CLS001–005 (empty classes, orphans, missing multiplicities/types, generalization cycles).
  • State machines (0.10.0) — parser + STA001–003 (unreachable states, no initial/final, unlabelled transitions).
  • Use-case direction (0.11.0) — UC003 catches <<include>>/<<extend>> arrows pointing the wrong way.
  • Traceability & readability (0.12.0) — GEN006 owner-tag and GEN007 requirement-link (convention-gated: dormant until you configure your project's pattern), GEN008 note-density, GEN009 max-elements, SEQ011 max-messages.
  • Cross-type entity identity (0.13.0) — XD004/XD005 catch the same entity named or stereotyped inconsistently across sequence, use-case, class and activity diagrams.
  • Calibration everywhere (0.14.0, 0.17.0) — the mutation-ladder corpus and the frozen golden-score contract now cover every diagram type (97 scored units).

Evidence

  • Deepened codegen experiment (0.17.0) — 243 generation runs across two generator models and two independent LLM judges (EVIDENCE.md): controlling for scenario difficulty, per-diagram correlation between the composite score and generated-code fidelity is r ≈ 0.65–0.70, stable across models. The below-Level-2 cliff reproduces — and steepens under a smaller generator model. Judges agree on ranking (r = 0.715).

Auto-fix & contract

  • pumllint fix (0.16.0) — auto-remediation for exactly the mechanical findings where nothing has to be invented: diagram names, humanized titles, implicit participant declarations. --dry-run prints the diff and exits 1 when fixes are pending (CI check mode); the Action accepts command: fix. Labels, guards and multiplicities stay human decisions.
  • JSON report schemas (0.18.0) — the -f json shapes for lint and score are now a pinned public contract: JSON Schemas (draft 2020-12) ship inside the package, pumllint schema {lint,score} prints them, and the test suite validates every emittable report shape against them.

Documentation

  • Audience-split guides (post-release)docs/ now carries role-specific guides: the management case (with the measured evidence), setup & CI integration, understanding findings & scores, and a step-by-step rule-writing guide walked through a real shipped rule.

Level 5 means method-convention complete — the diagram-side preconditions for faithful code generation, bound to the codegen profile. Full details: README, SCORING.md, RULES.md, ROADMAP.md.