Skip to content

Standards plugins: severity-in-tag rule model and domain-skill shape (family redesign) #8

Description

@vircung

Defines the target convention for every Standards plugin of this
marketplace (today: salesforce-standards, python-standards; any
future domain plugin). #6 and #7 track per-plugin adoption; this ticket
owns the standard itself and its shared pieces.

Motivation

Dogfooding the salesforce review path against real codebases showed
severity grading is path/attention dependent in both directions
(over-grade in the review's primary domain, under-grade in secondary
ones; the same rule graded Critical in one run and Minor in another).
Root cause: two competing authorities — the code-review skill's abstract
step-3 severity definition vs the area skills' ## Review severities
roll-ups — and severity living far from the rule text the reviewer
actually reads. python-code-review carries a verbatim-identical
abstract, so the same latent defect exists family-wide.

The model

  • Severity lives in the rule tag. Every rule carries
    (id: <rule-id>; severity: critical|important|minor; source: …) at its
    definition site. One id, one absolute severity. The ## Review severities roll-up sections are removed — the tag is the single
    source of truth (no second surface to drift).
  • Sub-rules. A rule whose violations grade differently splits into
    sub-rules with dot-suffixed sub-ids, each with its own absolute
    severity, e.g. apex-test-assertions.no-state-check (critical) vs
    apex-test-assertions.legacy-assert (minor). The group id always
    keeps its own severity — the default for findings matching the rule
    but no listed sub-rule. Sub-rule entries live in the rule's own
    section, tagged, co-located with the prose.
  • The kind facet on critical rules. Every rule or sub-id with
    severity: critical also carries kind: defect|hardening in its tag —
    defect for genuine runtime/live-risk failures, hardening for
    standards-mandated protections graded critical by house policy. Kind
    is a property of the rule, assigned once at authoring time and read
    off like severity — never a per-finding judgment (a per-run call would
    reintroduce the grading variance this redesign removes). Full tag
    grammar: (id; severity; kind — required at critical, absent otherwise; source). Rationale: in dogfooding, reviewers spontaneously
    and inconsistently split their Critical lists into "fix first" vs
    "house standard" — the facet encodes that distinction as data. The
    precedent holds: SonarQube abandoned its flat 5-severity ladder and
    resolved the same conflation with an orthogonal per-rule facet. The
    three severity levels themselves stay unchanged.
  • Grading cascade (each *-code-review skill): match a sub-rule →
    its severity (cite its sub-id); match a rule without a sub-rule → the
    group severity; no matching rule → the finding is still reported and
    counted, cited as rule: none, graded by the authoring rubric — a
    plausible-looking id is never fabricated; a critical-graded
    rule: none finding is always kind: defect (hardening requires a
    mandating standard, which a candidate gap by definition lacks). When
    a matching rule exists, citing its specific id is mandatory (a bare
    standard: citation is not a valid finding).
  • Authoring rubric. The former step-3 abstract (corrupts data /
    breaks security-sharing / governor on a bulk path → critical; forces
    rework or hides bugs → important; naming, style, docs → minor) moves
    from grading time to authoring time: it guides the severity and kind
    assigned when writing or editing a rule, and grading rule: none
    findings. Reviewers never use it to overrule a tag.
  • Reports. The findings: { critical: N, important: N, minor: N }
    frontmatter and the Critical → Important → Minor layout stay
    byte-compatible. Kind surfaces additively: an inline label on critical
    findings and a prose breakdown in the Summary headline (e.g.
    "critical: 33 — 12 defect, 21 hardening"). No new counts until a
    machine consumer exists.
  • Candidate-gap offers. rule: none findings are candidate standards
    gaps. After the report, the reviewer surfaces them in its reply and
    offers — never performs unprompted: (a) when the reviewed project keeps
    a Project-memory store (probe docs/memory/INDEX.md and
    .claude/memory/INDEX.md), parking them as an idea entry — explicit
    guidance on store choice wins; with both stores present the offer asks
    the developer which one; never an offer to create a store; (b) drafting
    an upstream report to the plugin's source repository, resolved at offer
    time from the installed marketplace's source metadata (a non-public
    source degrades the offer to a generalized draft with no filing
    target) — generalized (violation class + proposed rule), never the
    reviewed project's code, identifiers, or name, with the full draft
    shown for approval before anything is filed. No store — only the
    upstream offer.

Where the pieces live

  • Runtime-facing parts (rule: none citation, the kind labels in
    reports, the offers) — canonical text in the working-process
    review-reports contract; each Standards plugin's inline fallback
    carries a strict subset for standalone installs (existing pattern).
  • The authoring convention (tag grammar, dot sub-ids, group defaults,
    kind assignment, rubric) — a repo-level authoring rule in
    .claude/rules/, binding this repo's plugin authoring.
  • Family vocabulary (kind, defect, hardening, the tag grammar) —
    entries in docs/domain/glossary.md, so the plugins cannot drift
    terminology.

Adoption

Out of scope

  • *-plan-review checklists (separate grading surface; cross-references
    re-checked only).
  • Changes to the three severity levels (the kind facet deliberately
    replaces a fourth level — see Motivation and the model).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions