feat: verification pyramid — STPA-Sec tests, formal proof CI, Kani expansion#150
Merged
feat: verification pyramid — STPA-Sec tests, formal proof CI, Kani expansion#150
Conversation
Created 6 requirements satisfying SC-AI-001..006 using rivet batch. Fixed REQ-047/048/050 category from "security" to "non-functional". Warnings: 14 → 5. 689 artifacts, PASS. Trace: skip Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…rmal proof CI Close the gap between docs/verification.md and reality: implement 28 new tests across 3 files, fix production unwrap() panics, enable all formal verification tracks in CI, and add differential + operation-sequence property testing. Production fixes: - Replace store.get().unwrap() with safe patterns in validate.rs, matrix.rs, diff.rs, mutate.rs (12 sites) - matrix.rs: filter+unwrap → single-lookup filter_map CI (ci.yml): - Enable Kani bounded model checking (15 harnesses, was commented out) - Add Verus SMT verification via bazel test (rules_verus) - Add Rocq metamodel proof checking via bazel test (rules_rocq_rust) - Extend mutation testing to rivet-cli (was core-only) New test files: - stpa_sec_verification.rs: 16 tests for docs/verification.md §12 (XSS, commit traceability, git hook protection, document embeds) - differential_yaml.rs: 6 tests comparing rowan parser vs serde_yaml - proptest_operations.rs: 3 operation-sequence invariant tests Serve integration tests: - test_csp_header_present (SC-15) - test_reload_yaml_error_returns_error_response (SC-18) - test_reload_failure_preserves_state (SC-18) Implements: REQ-012, REQ-014, REQ-030 Verifies: REQ-004, REQ-017, REQ-020, REQ-032 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add 12 new Kani proof harnesses (16→27 total) targeting high-priority public functions that handle user input. Each harness proves panic-freedom over all bounded inputs via exhaustive symbolic execution. New harnesses: - commits: parse_commit_type, extract_artifact_ids, expand_artifact_range, parse_trailers — all prove panic-freedom for arbitrary ASCII input - store: upsert panic-freedom + retrievability after type change - diff: ArtifactDiff::compute panic-freedom for stores up to 3 artifacts - mutate: prefix_for_type, next_id, validate_link (missing source/target) - markdown: render_markdown panic-freedom, strip_html_tags correctness (output never contains angle brackets) Verifies: REQ-030 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
⚠️ Performance Alert ⚠️
Possible performance regression was detected for benchmark 'Rivet Criterion Benchmarks'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.20.
| Benchmark suite | Current: 9a27a53 | Previous: 21cd1e0 | Ratio |
|---|---|---|---|
store_insert/10000 |
14337713 ns/iter (± 1047691) |
10877938 ns/iter (± 720653) |
1.32 |
link_graph_build/10000 |
29583731 ns/iter (± 2673767) |
22296368 ns/iter (± 834679) |
1.33 |
validate/10000 |
11762980 ns/iter (± 1475744) |
9067860 ns/iter (± 248395) |
1.30 |
diff/10000 |
9066327 ns/iter (± 672761) |
7178257 ns/iter (± 183800) |
1.26 |
This comment was automatically generated by workflow using github-action-benchmark.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
…tions Fix #129: Replace hardcoded yaml-sections list (17 rivet-specific names) with yaml-section-suffix pattern matching. Any top-level YAML key ending with "-ucas" now auto-matches the uca type. Projects with custom section names (e.g., loom's isle-rewriter-ucas) are no longer silently dropped. Fix #130: Add nested artifact extraction from within parent items. Control-actions embedded inside controllers (control-actions: [{ca: ...}]) are now extracted as separate control-action artifacts with issued-by links back to the parent controller. Also recognize "ca" as an ID field alias for STPA control-action items. CI: Mark Kani/Verus/Rocq jobs as continue-on-error while toolchains are being stabilized in GitHub Actions. Fixes: REQ-002, REQ-004 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Mutation testing now covers both rivet-core and rivet-cli, which requires more time than the original 20-minute budget. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
avrabe
added a commit
that referenced
this pull request
Apr 21, 2026
Addresses three gaps found in the post-v0.4.0 dogfooding audit. **v0.4.0 shipped-work artifacts** — `artifacts/v040-features.yaml` was last touched 2026-04-12 and describes variant/PLE work (FEAT-106..114), not the verification pyramid that actually shipped on 2026-04-19. New file `artifacts/v040-verification.yaml` authors 4 design decisions (DD-052 four-layer verification pyramid, DD-053 suffix-based yaml-section matching, DD-054 non-blocking framing for formal CI jobs, DD-055 cfg-gate platform syscalls), 8 features (FEAT-115..122 covering Kani 27-harness expansion, differential YAML tests, operation-sequence proptest, STPA-Sec suite, suffix-based UCA extraction, nested control-action extraction, Zola export, Windows support), and 1 requirement (REQ-060 cross-platform binaries). Counts were verified against the actual codebase — 27 `#[kani::proof]` attrs in proofs.rs, 6 differential tests, 16 STPA-Sec tests. **Retroactive trailer map** — extended `AGENTS.md` with three more legacy orphans (51f2054 #126, f958a7e, 75521b8 #44), a new v0.4.0 PR-level section for #150/#151/#152/#153, and an honest "genuinely-unmappable" section calling out `ca97dd9f` (#95) whose `SC-EMBED-*` trailers point to artifacts that were never authored. **Verus Proofs → hard gate** — rules_verus PR #21 (merged as 5bc96f39) fixes the hub-repo's ambiguous `:all` alias by emitting proper `toolchain()` wrappers per platform. Updates the git_override pin from e2c1600a (Feb 2026, broken) to 5bc96f39 and removes `continue-on-error: true` from the Verus job. Implements: REQ-030, REQ-060 Refs: DD-052, DD-053, DD-054, DD-055, FEAT-115, FEAT-116, FEAT-117, FEAT-118, FEAT-119, FEAT-120, FEAT-121, FEAT-122 Verifies: REQ-030
avrabe
added a commit
that referenced
this pull request
Apr 21, 2026
Addresses three gaps found in the post-v0.4.0 dogfooding audit. **v0.4.0 shipped-work artifacts** — `artifacts/v040-features.yaml` was last touched 2026-04-12 and describes variant/PLE work (FEAT-106..114), not the verification pyramid that actually shipped on 2026-04-19. New file `artifacts/v040-verification.yaml` authors 4 design decisions (DD-052 four-layer verification pyramid, DD-053 suffix-based yaml-section matching, DD-054 non-blocking framing for formal CI jobs, DD-055 cfg-gate platform syscalls), 8 features (FEAT-115..122 covering Kani 27-harness expansion, differential YAML tests, operation-sequence proptest, STPA-Sec suite, suffix-based UCA extraction, nested control-action extraction, Zola export, Windows support), and 1 requirement (REQ-060 cross-platform binaries). Counts were verified against the actual codebase — 27 `#[kani::proof]` attrs in proofs.rs, 6 differential tests, 16 STPA-Sec tests. **Retroactive trailer map** — extended `AGENTS.md` with three more legacy orphans (51f2054 #126, f958a7e, 75521b8 #44), a new v0.4.0 PR-level section for #150/#151/#152/#153, and an honest "genuinely-unmappable" section calling out `ca97dd9f` (#95) whose `SC-EMBED-*` trailers point to artifacts that were never authored. **Verus Proofs → hard gate** — rules_verus PR #21 (merged as 5bc96f39) fixes the hub-repo's ambiguous `:all` alias by emitting proper `toolchain()` wrappers per platform. Updates the git_override pin from e2c1600a (Feb 2026, broken) to 5bc96f39 and removes `continue-on-error: true` from the Verus job. Implements: REQ-030, REQ-060 Refs: DD-052, DD-053, DD-054, DD-055, FEAT-115, FEAT-116, FEAT-117, FEAT-118, FEAT-119, FEAT-120, FEAT-121, FEAT-122 Verifies: REQ-030
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
unwrap()panics in validate.rs, matrix.rs, diff.rs, mutate.rs (12 sites → safe patterns)Test plan
cargo test --all— 1,085 tests pass, 0 failurescargo clippy --all-targets -- -D warnings— 0 errorsrivet validate— PASS (5 warnings, 0 errors)bazel testbazel test🤖 Generated with Claude Code