feat(cli): v0.4.1 CLI polish — fail-on, stats counts, coverage gate, JSON schemas#177
Merged
feat(cli): v0.4.1 CLI polish — fail-on, stats counts, coverage gate, JSON schemas#177
Conversation
New flag on `rivet validate`: --fail-on error (default, current behavior), --fail-on warning, --fail-on info. Exit code 1 when any diagnostic at or above the given severity is emitted. Lets CI tighten the traceability gate over time without forcing every warning to be promoted in the schema. Tests cover all three outcomes: default --fail-on error on a warning-only project exits 0, --fail-on warning on the same project exits 1, and an invalid value is rejected with a clear error message. Implements: REQ-007 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`rivet stats --format json` now exposes the same severity breakdown as `rivet validate --format json` (new fields: errors, warnings, infos). Removes the need for consumers to make a second validate call just to get diagnostic counts when rendering a dashboard or CI summary. Existing fields (total, types, orphans, broken_links) are unchanged — additive only, backward-compatible. The text output gains a trailing "Diagnostics: N error(s), ..." summary line so the human-readable form agrees with JSON. Tests: one asserting the new fields are present and numeric; a cross-command test asserting stats and validate agree on the counts for the current project. Implements: REQ-007 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The --fail-under flag already gated exit code on overall coverage.
This commit hardens and documents the CI-gate use case:
- JSON output echoes a new `threshold: { fail_under, passed }`
block when the flag is set, so consumers can distinguish a clean
run from a gated failure without parsing stderr.
- Text output prints a "✔ coverage N.N% meets threshold M.M%" line
on success to match the existing failure message.
- JSON output now carries `"command": "coverage"` for consistency
with the rest of the --format json envelopes.
Tests: --fail-under 0 passes, --fail-under 101 fails, no flag is
report-only, and JSON carries the threshold echo.
Implements: REQ-007
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds draft-2020-12 JSON Schemas describing every `--format json` CLI output so downstream consumers can validate against a machine-readable contract instead of reverse-engineering field layouts: schemas/json/validate-output.schema.json schemas/json/stats-output.schema.json schemas/json/coverage-output.schema.json schemas/json/list-output.schema.json Schemas are hand-written (rivet has no `schemars` dependency today — grep of workspace Cargo.toml files returned zero hits — and pulling it in just for four small schemas is heavier than the schemas themselves). Two new subcommands under `rivet schema` surface the schemas: rivet schema list-json # enumerate shipped schemas + paths rivet schema get-json <name> # print path for one rivet schema get-json <name> --content # print schema content Tests cover: - every shipped schema file is valid JSON with required metadata - `schema list-json --format json` lists all four, all files exist - `schema get-json <name>` round-trips path-and-content for all four - an unknown name is rejected with a helpful error - the actual `rivet validate` / `rivet stats` JSON output contains every `required` field declared in the corresponding schema — so future field drift fails CI instead of silently breaking consumers Implements: REQ-007 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
avrabe
force-pushed
the
feat/v041-cli-polish
branch
from
a284339 to
88c7f37
Compare
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
⚠️ Performance Alert ⚠️
Possible performance regression was detected for benchmark 'Rivet Criterion Benchmarks'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.20.
| Benchmark suite | Current: 88c7f37 | Previous: a24c1b9 | Ratio |
|---|---|---|---|
store_lookup/1000 |
24926 ns/iter (± 80) |
19264 ns/iter (± 61) |
1.29 |
diff/1000 |
719897 ns/iter (± 15853) |
593193 ns/iter (± 3478) |
1.21 |
diff/10000 |
10557718 ns/iter (± 424327) |
7940709 ns/iter (± 870929) |
1.33 |
query/1000 |
6834 ns/iter (± 16) |
5527 ns/iter (± 12) |
1.24 |
This comment was automatically generated by workflow using github-action-benchmark.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Four small CLI polish items, one commit each:
rivet validate --fail-on <severity>— exit 1 when any diagnostic at or above the given severity is emitted. Defaults toerror(current behavior);--fail-on warninglets a project tighten its gate over time without having to promote every warning in the schema.rivet stats --format jsonincludes diagnostic counts — new fieldserrors,warnings,infos(same asrivet validate --format json). Removes the need for consumers to make two calls just to build a dashboard or CI summary. Existing fields unchanged — additive only.rivet coverage --fail-undergate polish — flag already existed; hardened JSON output (newthreshold: { fail_under, passed }block, added"command": "coverage"envelope), cleaner text output on pass, and dedicated CI-gate tests.--format jsonoutputs — ships draft-2020-12 JSON Schemas underschemas/json/forvalidate,stats,coverage,list. Two new commands:rivet schema list-jsonenumerates them andrivet schema get-json <name>prints the path (or content with--content). Schemas are hand-written; rivet has noschemarsdependency today and pulling it in for four small hand-maintained schemas wasn't worth the transitive cost.New fields added to JSON output
stats:errors,warnings,infoscoverage:command: "coverage",threshold: { fail_under, passed }(only when--fail-underset)Schemars vs hand-written
schemarswas not available in the workspace (grep of everyCargo.tomlreturned zero hits). The four schemas are hand-written.Test plan
cargo clippy -p rivet-cli --all-targets -- -D warnings— cleancargo test -p rivet-cli --test cli_commands— 50/50 pass (13 new tests added across the 4 commits)--fail-on(three cases), stats diagnostic counts match validate, coverage--fail-underthree cases, JSON schema self-consistency (shipped files are valid JSON, everyrequiredfield is emitted by the CLI)Commits
00e1a77feat(cli): add --fail-on flag to validateaef9d8cfeat(cli): include errors/warnings/infos in stats JSON output35ff164feat(cli): polish coverage --fail-under gatea284339feat(cli): publish JSON schemas for --format json outputs🤖 Generated with Claude Code