Embedded (builtin) schemas are binary-versioned, not pinned — upgrading rivet silently changes validation (new diagnostics on unchanged artifacts)

[avrabe]

Reported by the maintainer; confirmed in code.

Symptom

A project using a builtin schema (schemas: [common, aspice, …] with no local schemas/ dir) sees new validation issues appear on unchanged artifacts after upgrading rivet — the same rivet validate that passed now flags things.

Root cause

embedded::load_schemas_with_fallback (rivet-core/src/embedded.rs:250) resolves each schema name by preferring an on-disk <schemas_dir>/<name>.yaml, else falling back to the embedded (compiled-into-the-binary) copy. So with no local schemas, the schema you validate against is whatever shipped in that rivet build. Every release that tightens or adds rules to a builtin schema silently changes validation semantics for every embedded-schema consumer — recent examples that would do exactly this: the status enum on common (status-allowed-values), the coverage-rule-consistency meta-check, prose-mention-without-typed-link, the ASPICE gate changes.

Schemas carry a version: field (common 0.1.0, aspice 0.2.0), but it's cosmetic — there's no pin in rivet.yaml, no min-rivet-version/version gate, and no drift detection. So the change is completely silent.

This is working-as-coded but a real stability gap. Options:

1. Mitigation that already works today: vendor the schemas into the project — rivet init can write schemas/*.yaml, and the loader prefers on-disk over embedded — so they're pinned in the project's git and immune to binary upgrades. Worth documenting prominently as the recommended posture for projects that need stable validation across upgrades. (Does rivet init vendor schemas by default? If not, an --vendor-schemas flag would make this one step.)
2. Pin + detect: let rivet.yaml record the schema version(s) it expects and have validate warn when the embedded schema version differs (turn the cosmetic version: into a real gate).
3. Surface: rivet validate could print the schema name@version set it used (and rivet schema list could show embedded-vs-vendored + version), so an upgrade-induced change is at least visible.

Recommend (1) documented now + (3) as a small surfacing change; (2) is the fuller fix (a maintainer call on the pin syntax). Found while researching the report — can implement the surfacing (3) and a docs section for (1) if you want.

[avrabe]

Triage pass — premise confirmed against current main:

• load_schemas_with_fallback (rivet-core/src/embedded.rs:250) does prefer on-disk over embedded (line 258 → 261), so option 1 of the body (vendor the schemas into the project) is mechanically supported today; what's missing is the discovery surface that tells a downstream maintainer this is the recommended posture.
• The body identifies three options (1: document/vendor; 2: pin-and-detect; 3: surface schema source/version in validate / schema list). These are independent decisions with different blast radii — (1) is docs-only, (2) is a behaviour-changing schema-version-gate, (3) is an output-format change. Picking which lands in the first PR is a maintainer call and the issue can't be picked up under the "one PR per issue, keep PRs focused" rule until that's pinned.

Could you add an explicit ## Acceptance Criteria block selecting one of the three (or saying "options 1+3 together, option 2 deferred to a separate issue", which would be my read of the body's own recommendation)? Sketch for the most tractable starting slice (1 + 3, deferring 2):

## Acceptance Criteria — first slice
- [ ] **Surface (3):** `rivet validate` summary output includes a `schemas:` line listing each schema in use as `<name>@<version> (embedded|on-disk:<path>)`, so an upgrade-induced semantic change is at least visible in the failing-validate output that surfaces it (golden-file test on a fixture project covers both an embedded-only and a vendored case)
- [ ] **Surface (3):** `rivet schema list` (and/or `rivet schema show <name>`) prints the same `embedded|on-disk` source + version for each schema (golden test)
- [ ] **Docs (1):** `docs/schemas.md` (or a new `docs/upgrade-stability.md`) adds an "Upgrade-stable validation" section recommending vendoring schemas into `schemas/` for projects that need stable validation across rivet upgrades, with the `rivet init` command/flag and the loader's "prefer on-disk over embedded" behaviour both linked from there
- [ ] (Optional, scope-flag if doing in this PR) `rivet init` gains `--vendor-schemas` that copies the requested embedded schemas into the project's `schemas/` dir so step (1) is one command, not a manual file copy
- [ ] **Explicitly out of scope (file separately):** option (2) — `rivet.yaml` schema-version pin + drift-warning. It's the fuller fix but a behaviour-changing semantic, deserves its own issue and isn't blocked by this PR (a project that's vendored its schemas via (1) is already drift-proof against binary upgrades, so (2) becomes optional rather than urgent)
- [ ] Commit trailer: `Implements: REQ-010` (schema-driven validation) + `Refs: REQ-NNN` if a new REQ tracks the surfacing; `rivet validate` PASS; `cargo test -p rivet-core -p rivet-cli` green

If you'd rather start with option (2) (the pin-and-detect mechanism) instead, that's also tractable — but the AC bullets would be different (version field becomes load-bearing instead of cosmetic; need to define warn-vs-error policy and the min-rivet-version syntax). Say which path and I'll re-shape if needed.

Process side-rail: https://pulseengine.eu/blog/ is still HTTP 503 (per the second finding on #420), so the binding workflow guidance can't be consulted and no PR will land from this run regardless. Comment is the AC ask only.

Not implemented this run.

---

Generated by Claude Code — issue-triage agent run 2026-06-03.

---

Generated by Claude Code

[avrabe]

Verified against the code — the report is valid, and here's the precise gap:

What exists: rivet schema info <name> already surfaces a schema's version (e.g. rivet schema info common prints Schema: common / Version: 0.1.0). So per-schema version is introspectable.

The actual gap (root cause of "issues appear across versions"): load_schemas_with_fallback (rivet-core/src/embedded.rs:250) resolves each schema as on-disk if <schemas_dir>/<name>.yaml exists, else the compiled-in embedded copy — and never reports which source it used. So when you rely on builtin schemas (no local schemas/ dir), upgrading the rivet binary silently swaps the embedded schema underneath you, and neither validate nor schema info tells you "this came from the embedded copy at version X". New diagnostics then look like they appeared from nowhere.

Minimal fix (low-ripple, additive — no signature change to the loader): a read-only schema_sources(names, schemas_dir) -> [(name, OnDisk(path) | Embedded | Missing)] helper, surfaced as a one-line source annotation in validate and schema info — e.g. Schemas: common@0.1.0 (embedded), stpa@0.3.0 (./schemas/stpa.yaml). That makes embedded-vs-pinned visible at a glance.

Mitigation available today: vendor the schemas locally (rivet schema export / copy the embedded YAML into <project>/schemas/), then the on-disk copies win and are pinned in git — immune to binary drift.

Filing this as REQ-176 (draft) and will pick it up in a dedicated PR; flagging the design here for maintainer input on the surfacing point (validate header vs schema info vs a new schema sources).

[avrabe]

Follow-up to #442: opened #443 adding rivet schema sources, which lists every active schema as on-disk (pinned) / embedded (binary-versioned) / missing — the at-a-glance view of whether you're exposed to binary-version drift. It resolves without loading the schema graph, so it still reports missing when a schema fails to load. Between schema info (per-schema) and schema sources (whole set), the provenance is now introspectable; the remaining open idea is surfacing it inside validate output, which I left for maintainer input on the surface point.

[avrabe]

Status: the core of this is resolved on main. rivet schema info <name> reports the source (REQ-176) and rivet schema sources lists every active schema as on-disk (pinned) / embedded (binary-versioned) / missing (REQ-177) — so a project can now see at a glance whether it's exposed to embedded-schema drift, and the mitigation (vendor schemas under schemas/ to pin) is documented. The one optional idea still open is surfacing the source inline in validate output, which is a maintainer call on noise vs. signal. Suggest closing once that's decided, or repurposing this issue to just the validate-surfacing nicety.

[avrabe]

Triaged + implemented from the hourly dogfooding loop — valid, confirmed in code (embedded::load_schema_contents prefers on-disk <schemas_dir>/<name>.yaml, else the embedded copy; the version: field was parsed but never surfaced).

Shipped your option 3 (surface) in PR #483. rivet validate now reports the exact schema set it used:

Schemas: common@0.1.0 (on-disk), dev@0.1.0 (on-disk), …, stpa-dev.bridge@0.1.0 (on-disk)

plus a schemas array (name/version/source/path) in --format json. An embedded-schema project shows (embedded), so an upgrade-induced change is visible at the point it bites; a vendored project shows (on-disk) = pinned.

Implementation note: I built it on the existing schema_sources classifier (REQ-177, which already backs rivet schema sources) rather than a parallel one — resolve_schema_provenance just adds the parsed version + auto-discovered bridges, keeping one on-disk/embedded vocabulary across both surfaces.

On your other two options:

• Option 1 (vendor schemas, documented): worth noting the rivet repo itself already vendors schemas/*.yaml, and the validate output now shows that posture (on-disk) — so the recommended-posture story is partly self-demonstrating. A docs paragraph + a rivet init --vendor-schemas (or default-vendor) is still a clean follow-up; I left it out of this PR to keep it focused.
• Option 2 (pin + gate): there's already a min-rivet-version field on SchemaMetadata (unused) — so part of the machinery exists — but turning the cosmetic version: into a hard gate, and the rivet.yaml pin syntax, is the maintainer call you flagged. Happy to implement once you pick the shape.