Tool Confidence Level (TCL) qualification dossier — workstream A

[avrabe]

Why now

docs/design/iso26262-artifact-mapping.md row 32 has flagged the TCL/TD gap since v0.4. safety/stpa/tool-qualification.yaml already self-positions rivet at "TCL 1 (highest)" — but uses DO-330 numbering on a 26262 acronym (lower TCL number = lower demand in 26262). The existing wording is itself an audit-flag in our own dogfood.

A new design note worked through the standards (ISO 26262 §11 / IEC 61508 / DO-330 / EN 50128 / ISO/PAS 8800), Polarion's TI2/TD1 → TCL1 posture, the competitor baseline, and what rivet would have to do to defend a stronger claim:

• docs/design/tool-confidence-level.md

The conclusion: rivet's right target is TCL1 in 26262 units but with the evidence bundle of a TCL2 tool, because the TD1 claim cannot rest on "human reviewer caught it" once the upstream author is an AI agent. The TD-raising substitutes (validate, oracles, cited-source hashing, docs-check, schema-migrate snapshot/abort) are already shipped — they just need to be packaged as a dossier.

Workstream A — one developer-week, in priority order

A1. Rename the existing STPA's TCL annotation. Change safety/stpa/tool-qualification.yaml line 14 from "TCL 1 (highest)" to "TCL 3 (highest demand)" to match ISO 26262-8 numbering. (~1h)

A2. Add tool-confidence typed artifact to schemas/iso-26262.yaml. Fields: ti: TI1..TI2, td: TD1..TD3, derived tcl, link-fields for qualification-evidence. Closes iso26262-artifact-mapping.md §C row 32. (~4h)

A3. Add ai-found-defect artifact type to schemas/common.yaml plus defect-against and corrects link types. This is the type that operationalises the user-facing requirement "any error found while AI is working would get reflected into the safety case". (~4h, schema sketch in §6.2 of the design note)

A4. Write docs/design/tool-qualification-dossier.md and a matching rivet docs tool-qualification topic. Sections: TOR, Use Cases, Verification Plan, Configuration Baseline, Known Limitations, Tool Error Detection & Reporting Process, TCL/TQL Position Statement per Regime. (~1-2 days)

A5. Add rivet --qualification-mode flag and rivet stats --qualification to emit a configuration-baseline manifest (binary version, schema hashes, oracle inventory, cargo vet status). (~1 day)

Decision the user should make first

Which regime is the lead for the dossier — ISO 26262, DO-178C, or IEC 62304? The other two follow as cross-walks, but picking one governs numbering, dossier structure, and which customer pilot to prioritise.

Out of scope (intentionally)

• Customer-project-level qualification — that is the customer safety manager's call. Rivet ships the dossier.
• Numeric-safety-output features (SPFM/LFM/PMHF) — broader iso-26262 schema completion (gap §C rows 2/20/21).
• E-signature / 21 CFR Part 11 — explicitly disclaimed in ai-safety-cyber-hitl.md §5.2.

References

• docs/design/tool-confidence-level.md — this workstream's design note
• docs/design/iso26262-artifact-mapping.md §C row 32
• docs/design/ai-safety-cyber-hitl.md §5
• safety/stpa/tool-qualification.yaml
• schemas/iso-pas-8800.yaml, schemas/score.yaml, schemas/en-50128.yaml

Refs: FEAT-001

[avrabe]

Triage: workstream A1–A5 already reads as a near-AC ladder, but there is no explicit ## Acceptance Criteria section, and A4 is gated on a regime decision that hasn't been made yet. To pick this up under the oracle-gated workflow (ref) please add:

## Acceptance Criteria

- [ ] **A1** `safety/stpa/tool-qualification.yaml` line 14 reads `TCL 3 (highest demand)` (ISO 26262-8 numbering); `rivet validate` clean
- [ ] **A2** New `tool-confidence` typed artifact in `schemas/iso-26262.yaml` with fields `ti: TI1..TI2`, `td: TD1..TD3`, derived `tcl`, link-field `qualification-evidence`; round-trips through `rivet validate`; `iso26262-artifact-mapping.md` §C row 32 marked closed
- [ ] **A3** New `ai-found-defect` artifact type in `schemas/common.yaml` plus `defect-against` and `corrects` link types; fixture covers both link types and schema-test passes
- [ ] **A4** `docs/design/tool-qualification-dossier.md` and `rivet docs tool-qualification` topic exist with the seven required sections (TOR, Use Cases, Verification Plan, Configuration Baseline, Known Limitations, Tool Error Detection & Reporting, TCL/TQL Position Statement per regime); `rivet docs check` clean
- [ ] **A5** `rivet --qualification-mode` flag and `rivet stats --qualification` emit a configuration-baseline manifest (binary version, schema hashes, oracle inventory, `cargo vet` status); golden test against a fixture; manifest is deterministic
- [ ] Commit trailers `Implements: REQ-002` (STPA), `Implements: REQ-010` (schema), `Implements: REQ-007` (CLI)

Decision needed on scope, not bullets: the issue body asks "which regime is the lead — ISO 26262, DO-178C, or IEC 62304?" That answer governs A4's numbering and section ordering; without it A4 ships in two flavors. Please decide that and update A4's AC bullet to name the regime.

A1 is small enough to land independently as a one-line fix PR if useful — happy to take that as a sub-issue if the rest is gated on the regime decision.

---

Generated by Claude Code — issue-triage agent run 2026-05-01.

---

Generated by Claude Code

[avrabe]

User decision: ISO 26262 is the lead regime

Per maintainer feedback (1 May): ISO 26262 (automotive) confirmed as the lead. DO-178C / DO-330 and IEC 62304 follow as cross-walks.

Implications

• Dossier structured around ISO 26262 Part 8 §11 (TI/TD/TCL); §11 cross-walked to DO-330 TQL-1..5 and IEC 62304 Class A/B/C as appendices.
• First pilot customer should be in automotive — sharpest peer-comparison with Polarion.
• tool-confidence typed artifact defaults to ISO 26262 vocabulary; cross-regime mappings are non-default fields.
• TCL annotation in safety/stpa/tool-qualification.yaml line 14 currently says "TCL 1 (highest)" — mixes DO-330 numbering (TQL-1 is highest rigor) with ISO 26262 acronym (TCL1 is LOWEST confidence). Fix by writing "TCL1" with cross-walk note.
• Order of operations: A1 (1h) → A2 (4h) → A3 (4h) → A4 (~2 days) → A5 (1 day). ~1 dev-week as scoped.

[avrabe]

Workstream A1 in flight — PR #257.

Per the 2026-05-01 17:14 decision (ISO 26262 lead, fix as "TCL1" with cross-walk note), the STPA header in safety/stpa/tool-qualification.yaml is rewritten to ISO 26262-8 numbering with an explicit DO-330 cross-walk paragraph. Comment-only change; rivet validate error set is byte-identical to the pristine-main baseline.

A2 (typed tool-confidence artifact in schemas/iso-26262.yaml) and A3 (ai-found-defect type in schemas/common.yaml with defect-against / corrects link types) are the natural next slices and will land as separate focused PRs. A4 (dossier doc) and A5 (--qualification-mode flag + rivet stats --qualification) remain after that.

This issue stays open until the full workstream lands.

---

Generated by Claude Code — issue-triage agent run 2026-05-01.

---

Generated by Claude Code