FUG-89: requirements-driven development workflow - #61
Open
issuefleet[bot] wants to merge 7 commits into
Open
Conversation
FUG-89 Build requirements driven development workflow
Following the analysis in FUG-88, translate all of the identified UNs, PRs, and RISKs into a machine-readable yaml format. Within the software implementation, include inline documentation in each module that references the PRs satisfied by the implementation. If there are mitigations for RISKs, they should each trace to derived PRs that describe the mitigation (those derived PRs in turn reference the RISKs mitigated). Within the test suite, annotate all test cases by the PRs (whether direct PRs or derived PRs) verified. Implement test traceability; all tests (software-only and HITL) output jUnit XML that contains extra tags that contain the traceability information. A final action in the workflow aggregates all jUnit XMLs into an HTML report that shows the entire UN/PR listing with the PASS/FAIL for validation/verification of each given the traceability information and the test results. |
|
Add the machine-readable requirements model (requirements/requirements.yaml + JSON schema) capturing UNs, PRs, RISKs and derived-PR risk mitigations seeded from the repo design docs, plus the traceability toolkit (//tools/traceability): - model.py: load + validate the model (structure + bidirectional satisfies/mitigates/mitigated_by cross-references). - pytest_requirements.py: @requirements marker -> per-testcase jUnit <property name=requirement> traceability tags. - pytest_runner.py: shared py_test entry writing jUnit to XML_OUTPUT_FILE. - junit.py / report.py / cli.py: parse tagged jUnit, join to the model, and render the HTML validation/verification/mitigation matrix (aggregate CLI). - annotations.py + //requirements:check_annotations: fail on any source annotation referencing an unknown PR id. Includes dogfooding unit tests and the //requirements:requirements_valid_test guard. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Route each Python py_test through the shared traceability runner (jUnit to XML_OUTPUT_FILE with the @requirements plugin) and tag each suite with the PRs it verifies via a module-level pytestmark: - pi/hitl, pi/server, pi/led_driver, pi/reconstruction, shared/simulator pytest_main.py -> traceability.pytest_runner shim; //tools/traceability dep. - HITL: sync (PR-5,35), improv (PR-9), mapping_trigger (PR-36), fx_bench (PR-7), map_upload (PR-13). Server: codebook (PR-1,30), proto_wire (PR-12,37), handler (PR-12), session (PR-6), app_integration (PR-2,6,12). Reconstruction: reconstruct (PR-2,32), rust_parity (PR-3), vio (PR-2). Simulator: roundtrip (PR-4). led_driver: graycode (PR-1,31,33). Verified under Bazel: bazel-testlogs test.xml now carries per-testcase requirement tags. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a 'Requirements: PR-…' line to every top-level module's BUILD docstring (firmware, pi/led_driver, web, pi/reconstruction, solver, shared/simulator, pi/hitl/harness, pi/server, fx_compiler, shared/protocol, tools), inverted from the model's per-PR 'modules' field. Extend the annotation scanner to cover BUILD/.bzl files so these module docs are validated against the model by //requirements:check_annotations too. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the traceability-report CI job (final workflow action): validate the model, fail on any unknown-PR annotation (check_annotations), run the software suites to populate tagged jUnit, aggregate every bazel-testlogs XML into traceability-report.html (UN/PR/RISK validation/verification/mitigation matrix), and upload it as an artifact. Add docs/requirements-driven-development.md documenting the model, the annotation conventions, the two traceability mechanisms, and how to extend per-case tags to the other languages. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add traceability.junit_writer (JUnitWriter) so non-pytest runners can emit the same requirement-tagged jUnit the aggregator consumes, and wire it into //pi/hitl/harness:e2e: each phase is a testcase tagged with the PRs it verifies (flash_boot -> PR-9,34; improv_provision -> PR-9; websocket_checks -> PR-5,35,11,12,34), written to XML_OUTPUT_FILE (or --junit-xml). This closes the 'all tests (software-only and HITL) output jUnit XML with traceability tags' requirement for the hardware path too. Round-trip test included. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the design-doc-seeded model with FUG-88's baseline verbatim: UN-1..8, PR-1..25 (direct, with FUG-88's traces), RISK-1..12. Capture each risk's mitigation (stated as prose in FUG-88) as a derived PR (PR-26..37) that mitigates the risk, with mitigated_by naming it back. Re-map every annotation to the new ids: each product suite's pytestmark, the HITL e2e phase tags, every module BUILD's Requirements line (inverted from the model), the toolkit's own self-references (-> PR-23 validation infra / PR-25 CI quality gates), and the docs examples. check_annotations + requirements_valid_test enforce consistency (94 references, all resolve). Also harden the aggregate CLI so a missing/empty jUnit path can never crash the report job (skip empty/nonexistent inputs; parse_file swallows OSError). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rebased onto current main (video-streaming HITL suite, UI rebrand, deploy changes). Annotate the merged-in pi/hitl test_video_bench (TouchDesigner video -> device-texture effect) with PR-10, PR-17, and tie the new Rust //tools/touchdesigner/stream_bench:stream_bench_test to PR-10 as coarse verification. check_annotations resolves all 96 references; the toolkit, validator, and all annotated suites (incl. the merged hitl suite) pass under Bazel; prek --all-files clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
issuefleet
Bot
force-pushed
the
agent/fug-89-build-requirements-driven-develo
branch
from
August 10, 2026 21:12
cac9795 to
58005eb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
FUG-89 — requirements-driven development workflow
Builds the end-to-end requirements-driven-development workflow: a machine-readable
requirements/risk model, inline traceability annotations across modules and tests,
per-testcase jUnit traceability tags for all suites (software-only and HITL),
and a CI action that aggregates every jUnit XML into an HTML report showing
PASS/FAIL validation and verification of the whole UN/PR/RISK listing.
Model content (FUG-88)
requirements/requirements.yamlis the FUG-88 "Draft requirements and riskassessment for Splanc" baseline translated verbatim:
satisfiesFUG-88's traceduser needs.
kind: derived) capturing each risk's mitigation(FUG-88 states these as prose); each
mitigatesits risk and the risk'smitigated_bynames it back.//requirements:requirements_valid_testenforcesthe bidirectional trace.
Requirements describing product areas this repo doesn't implement yet (e.g. some
AI-workflow surfaces) intentionally have no verifying test and surface as
UNVERIFIED in the report — that gap is the point of the report.
Machinery
//tools/traceability):@requirements("PR-…")pytest marker →per-testcase jUnit
<property name="requirement">tags; a shared runner writesjUnit to
$XML_OUTPUT_FILEso everypy_testcarries its tags into CI; aJUnitWriterlets the non-pytest HITL e2e harness emit the same tags; a jUnitparser + HTML report generator +
aggregateCLI produce the matrix.BUILDdocuments the PRs itimplements (inverted from the model).
simulator, led_driver) tag the PRs they verify; the on-hardware HITL e2e
harness emits per-phase tagged jUnit (
flash_boot,improv_provision,websocket_checks). C++/Rust/Go/TS suites trace at target granularity via eachPR's
verified_by, with a documented path to per-case tags.//requirements:check_annotationsfails on any source/test/module reference to an unknown PR id (94 references today, all resolve).
traceability-reportjob validates the model + annotations, runs thesuites, aggregates
bazel-testlogs/**/test.xml, and uploadstraceability-report.html.docs/requirements-driven-development.md.Verification
//tools/traceability:traceability_test,//requirements:requirements_valid_test,//pi/hitl/tests:hitl_test,//pi/led_driver:led_driver_test,//pi/server:{server,server_integration}_test,//pi/reconstruction:{reconstruct,vio,rust_parity}_test,//shared/simulator:sim_recon_roundtrip_test;//pi/hitl/harness:e2ebuilds.bazel-testlogs/**/test.xmlconfirmed to carry per-testcase<property name="requirement">tags.prek run --all-filesand whole-repobazel build --nobuild //...both pass.cases → 14/37 PRs VERIFIED, 0 FAILED; the full CI run adds the firmware/web/
solver
verified_bytargets.Closes-Linear: FUG-89 (https://linear.app/fughilli/issue/FUG-89/build-requirements-driven-development-workflow)