Issue #2743: Record backplane registry evidence - #2747
Merged
Conversation
stranske
added
agent:codex
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThe PR adds offline and CI validation for the backplane registry, records participant lifecycle and reference evidence, updates reporting, and adds tests for valid and invalid registry records. It also updates worker-attempt metadata and a contract-test fixture repo. ChangesBackplane registry integrity
Worker attempt metadata
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant Registry as backplane_participants.json
participant Validator as validate_backplane_registry.py
participant CI as Backplane Contract Integrity
participant Artifact as registry-status.json
CI->>Validator: run --json
Validator->>Registry: load and validate registry
Validator->>Artifact: write findings and participant rows
Possibly related issues
Suggested labels: 🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Contributor
🤖 Keepalive Loop StatusPR #2747 | Agent: Codex | Iteration 0/12 Current State
Last Codex Run
To retry immediately:
Or wait for the next successful Gate run to automatically retry. 🔍 Failure Classification| Error type | infrastructure |
|
Contributor
Keepalive Work Log (click to expand)
|
stranske
temporarily deployed
to
agent-high-privilege
July 10, 2026 10:10 — with 
GitHub Actions
Inactive
GitHub Actions
Inactive
Contributor
|
Runner dispatch state for codex on PR #2747. Do not edit. |
Contributor
|
Runner dispatch state for codex on PR #2747. Do not edit. |
stranske
force-pushed
the
codex/issue-2743-backplane-registry
branch
from
July 10, 2026 10:11
57cae0c to
4a9c51e
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 57cae0cb61
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Contributor
There was a problem hiding this comment.
Pull request overview
This PR replaces backplane registry placeholders with real cross-repo issue/PR references, records Pension-Data’s first verified “conformant” reference-run evidence, and introduces an offline validator + CI wiring to keep the registry/evidence contract honest over time.
Changes:
- Update
config/backplane_participants.jsonto usestranske/Workflows#2743as the parent issue, defer non-active participants explicitly, and mark Pension-Data as conformant with captured reference-run evidence + lifecycle history. - Add
scripts/validate_backplane_registry.pyplus focused tests to enforce “no TBDs”, real issue-ref formats, deferred expiries, and conformant evidence requirements offline. - Extend the backplane integrity workflow to run the new registry validator and publish a machine-readable registry status artifact.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_backplane_registry.py | New tests asserting “no TBD” registry hygiene and Pension-Data conformant evidence requirements. |
| scripts/validate_backplane_registry.py | New offline validator for registry lifecycle/evidence integrity (issue refs, URLs, sha256s, deferred expiries). |
| docs/reports/backplane_registry_status.md | Snapshot report of current participant lifecycle/reference state and Pension-Data evidence pointers. |
| config/backplane_participants.json | Replaces placeholders with real issue refs, defers inactive participants with expiry+reason, records Pension-Data conformant evidence and history. |
| .github/workflows/health-78-backplane-contract.yml | Wires the new registry validator into the integrity workflow and uploads its JSON output as an artifact. |
stranske-keepalive
Bot
added
agent:needs-attention
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/validate_backplane_registry.py`:
- Around line 105-152: The validate_registry function accepts conformant
participants without validating lifecycle_history. Add lifecycle-history
validation before the status-specific checks, requiring planned → emitting →
conformant order, valid evidence for each transition, and monotonically ordered
timestamps; emit findings under the participant’s lifecycle_history path and
reject conformant entries lacking valid history.
- Around line 105-120: Handle malformed registry structures without exceptions:
in validate_registry, verify the top-level registry is a mapping and each
participants entry is a mapping before calling .get, appending findings for
invalid entries and continuing validation; apply the same guards in the
report-generation logic around the participant entries so inputs such as null
produce validation/report results rather than crashes.
- Around line 90-92: The evidence URL validation loop currently uses permissive
matching and one pattern for incompatible URL types. Update the validation logic
in the evidence-checking function to use complete matching via fullmatch, with
distinct patterns for emit/conformance run-job URLs versus disposition
issue-comment URLs, and add negative tests covering incomplete, wrong-type, and
non-job URLs.
- Around line 60-71: Update _validate_deferred_issue to require expires_at to
parse as a timezone-aware ISO timestamp, then compare it with datetime.now(UTC)
and add a Finding when the deadline is expired. Add a regression test using a
fixed past expires_at value to verify stale deferred issues are rejected.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: fbd2cc9d-2e78-4f6d-b15e-2feeb8fbc4be
📒 Files selected for processing (5)
.github/workflows/health-78-backplane-contract.ymlconfig/backplane_participants.jsondocs/reports/backplane_registry_status.mdscripts/validate_backplane_registry.pytests/test_backplane_registry.py
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
.github/workflows/health-78-backplane-contract.yml (1)
101-104: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winRun the new registry tests in this integrity job.
This command only executes
tests/contracts/;tests/test_backplane_registry.pyis not run here despite triggering this workflow. The CLI check validates only the checked-in registry, not the intentional invalid copies used for regression tests.Proposed fix
- python -m pytest tests/contracts/ -q \ + python -m pytest tests/contracts/ tests/test_backplane_registry.py -q \ | tee artifacts/backplane-contract/pytest.txt🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/health-78-backplane-contract.yml around lines 101 - 104, Update the “Contract test suite” command in the workflow to run both tests/contracts/ and tests/test_backplane_registry.py, while preserving the existing quiet output and tee to artifacts/backplane-contract/pytest.txt.docs/reports/backplane_registry_status.md (1)
3-25: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick winGenerate and verify this report instead of maintaining a partial snapshot.
CI only writes
registry-status.json; nothing renders or checks this Markdown. It can drift, and it already omitsconformance_report_sha256recorded in the registry. Generate this document from the validator output and verify it is current in CI.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/reports/backplane_registry_status.md` around lines 3 - 25, The registry status Markdown is a manually maintained snapshot that can drift and omits validator fields such as conformance_report_sha256. Add generation of docs/reports/backplane_registry_status.md from the validator’s registry-status.json output, including all relevant fields and evidence, and update CI to run the generator and fail when the committed report differs from generated output.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/health-78-backplane-contract.yml:
- Around line 80-83: The “Registry lifecycle and evidence validation” workflow
step must preserve failures from validate_backplane_registry.py when piping
output through tee. Enable shell pipefail for this run step before the validator
command, ensuring the step fails if the validator exits nonzero while still
writing registry-status.json.
In `@scripts/validate_backplane_registry.py`:
- Around line 145-156: The conformant-entry validation currently permits
deferred or unrelated evidence issues. Update the validation logic around
_validate_reference_evidence and the conformant branch to require a real entry
issue, reject issue: null or deferred conformant entries, and ensure
reference_run_evidence.source_issue exactly matches entry["issue"]; add a
negative regression test covering this mismatch.
- Around line 151-160: The validator currently ignores stale evidence and
incorrectly couples reference evidence state to lifecycle status. Update the
validation logic around `_validate_reference_evidence` to read
`stale_after_hours`, parse `generated_at`, classify evidence as `stale` once it
exceeds the configured threshold while preserving `conformant` lifecycle status,
and retain validation for invalid state combinations. Add tests covering the
exact freshness cutoff boundary and stale evidence reporting.
---
Outside diff comments:
In @.github/workflows/health-78-backplane-contract.yml:
- Around line 101-104: Update the “Contract test suite” command in the workflow
to run both tests/contracts/ and tests/test_backplane_registry.py, while
preserving the existing quiet output and tee to
artifacts/backplane-contract/pytest.txt.
In `@docs/reports/backplane_registry_status.md`:
- Around line 3-25: The registry status Markdown is a manually maintained
snapshot that can drift and omits validator fields such as
conformance_report_sha256. Add generation of
docs/reports/backplane_registry_status.md from the validator’s
registry-status.json output, including all relevant fields and evidence, and
update CI to run the generator and fail when the committed report differs from
generated output.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 261f286a-571e-486b-8e46-653d7af788ec
📒 Files selected for processing (6)
.github/workflows/health-78-backplane-contract.ymlconfig/backplane_participants.jsondocs/reports/backplane_registry_status.mdlangsmith-fleet-worker-attempt.jsonscripts/validate_backplane_registry.pytests/test_backplane_registry.py
stranske
had a problem deploying
to
agent-standard
July 10, 2026 11:03 — with
GitHub Actions
Failure
GitHub Actions
Failure
agents-workflows-bot
Bot
added
agent:needs-attention
stranske
added
agent:retry
stranske
had a problem deploying
to
agent-high-privilege
July 10, 2026 11:25 — with
GitHub Actions
Error
GitHub Actions
Error
Owner
Author
|
Closer review-fix pushed in Addressed the latest CodeRabbit registry validation review set:
Validation passed locally on head
I removed stale automation blocker labels and requested a fresh keepalive/check pass. |
stranske-keepalive
Bot
removed
the
agent:retry
stranske
temporarily deployed
to
agent-high-privilege
July 10, 2026 11:26 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
stranske
had a problem deploying
to
agent-standard
July 10, 2026 11:26 — with
GitHub Actions
Failure
GitHub Actions
Failure
agents-workflows-bot
Bot
added
agent:needs-attention
10 tasks
Contributor
Provider Comparison ReportProvider Summary
📋 Full Provider Details (click to expand)openai
anthropic
Agreement
Disagreement
Unique Insights
🔍 LangSmith Traces |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #2743
Automated Status Summary
Scope
config/backplane_participants.jsonstill contains aTBDparent issue andTBDparticipant issue links. Its lifecycle contract isplanned -> emitting -> conformant, whiledocs/contracts/research-backplane-contract.md:95-109requires central registry truth. Leaving placeholder links and declaring status from schema presence would repeat the built-but-not-flowing failure. Pension-Data should advance only after its participant-owned emitter has produced a real validated reference run.Context for Agent
Related Issues/PRs
Tasks
config/backplane_participants.json::parent_issueto this issue's URL.TBDissue with a real issue link or an explicit deferred/retired decision plus expiry; do not fabricate implementation status.scripts/validate_backplane_registry.pyandtests/test_backplane_registry.pyto validate real issue links/deferred expiries, evidence-backed transitions, and reference-run requirements offline.planned -> emittingwhen the emitter exists, thenemitting -> conformantonly when the real reference run passes the reusable workflow; preserve transition timestamps/history.Acceptance criteria
python scripts/validate_backplane_registry.py --strictreports zeroTBDlinks and validates every real issue URL/deferred expiry.conformantwhen the participant CI URL or reference artifact hash is missing/invalid.tests/test_backplane_registry.py::pension_conformant_entry, delete onlyreference_run_sha256; runpython -m pytest -q tests/test_backplane_registry.py::test_conformant_requires_reference_evidence; observeAssertionError: conformant participant lacks reference evidence; revert and show the exact test passes.Summary by CodeRabbit