Skip to content

fix(e2e): expect coupled MCP credential evidence - #8028

Merged
senthilr-nv merged 2 commits into
mainfrom
codex/fix-pr-e2e-coupled-mcp-evidence
Jul 31, 2026
Merged

fix(e2e): expect coupled MCP credential evidence#8028
senthilr-nv merged 2 commits into
mainfrom
codex/fix-pr-e2e-coupled-mcp-evidence

Conversation

@senthilr-nv

@senthilr-nv senthilr-nv commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Summary

Make the trusted PR E2E coordinator account for the credential-window job that the workflow couples to every MCP bridge selection. This prevents successful evidence from being rejected as an unexpected job.

Related Issue

Prerequisite for #8024.

Changes

  • Expand MCP bridge selections with the coupled credential-window job before dispatch.
  • Expect the coupled job's signed evidence during coordinator validation.
  • Add a regression test for the exact selection and shard contract.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: This changes only the internal trusted PR E2E coordinator and does not change supported product behavior.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: The change preserves fail-closed SHA, plan, correlation, job, and shard validation and only adds the workflow-defined coupled job.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

  • Documentation writer subagent reviewed the completed changes
  • Result: no-docs-needed
  • Evidence: tools/e2e/pr-e2e-gate.mts, test/pr-e2e-gate-signal-shards.test.ts
  • Agent: Codex Desktop

DGX Station Hardware Evidence

  • Tested on DGX Station
  • Tested commit:
  • Station profile/scenario:
  • Result:
  • Supporting evidence:

Verification

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run validate:pr passed after refreshing origin/main when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: 40 PR-gate tests and 14 MCP workflow-boundary tests passed.
  • Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result:
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Senthil Ravichandran senthilr@nvidia.com

Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
@senthilr-nv
senthilr-nv enabled auto-merge (squash) July 31, 2026 23:07
@coderabbitai

coderabbitai Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The PR gate now expands mcp-bridge selections to include credential-generation evidence. Shard calculation and workflow dispatch use the expanded selections. Tests verify deduplication and expected shard sets.

Changes

PR gate selection expansion

Layer / File(s) Summary
Selection expansion helper
tools/e2e/pr-e2e-gate.mts
Adds expandPrGateJobSelections, which appends openshell-credential-generation-window when mcp-bridge is selected and the dependent job is absent.
Expanded shard and dispatch flow
tools/e2e/pr-e2e-gate.mts, test/pr-e2e-gate-signal-shards.test.ts
Uses expanded selections for evidence shard calculation, shard discovery, and PR gate dispatch. Tests verify deduplication and expected shard sets.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

Suggested reviewers: cv, apurvvkumaria, prekshivyas

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the MCP credential evidence change, which is the main purpose of the pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/fix-pr-e2e-coupled-mcp-evidence

Comment @coderabbitai help to get the list of available commands.

@senthilr-nv senthilr-nv self-assigned this Jul 31, 2026
@github-code-quality

github-code-quality Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit a8e8406 in the codex/fix-pr-e2e-cou... branch remains at 96%, unchanged from commit b2e9d1b in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit a8e8406 in the codex/fix-pr-e2e-cou... branch remains at 81%, unchanged from commit b2e9d1b in the main branch.

Show a code coverage summary of the most impacted files.
File main b2e9d1b codex/fix-pr-e2e-cou... a8e8406 +/-
src/lib/inferen...ompatibility.ts 95% 94% -1%
src/lib/sandbox...rce-identity.ts 88% 88% 0%

Updated July 31, 2026 23:24 UTC

@senthilr-nv
senthilr-nv requested review from apurvvkumaria and cv July 31, 2026 23:08
@senthilr-nv senthilr-nv added bug-fix PR fixes a bug or regression area: e2e End-to-end tests, nightly failures, or validation infrastructure area: ci CI workflows, checks, release automation, or GitHub Actions security labels Jul 31, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
test/pr-e2e-gate-signal-shards.test.ts (1)

54-64: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Cover the no-duplicate selection case.

The current test covers only the append path. It does not cover a selection that already contains openshell-credential-generation-window. Add an assertion that both jobs remain unique and that expectedSignalShards returns both entries without throwing.

As per path instructions, assert the observable selection and shard behavior at the public boundary.

Suggested regression case
+  it("does not duplicate credential-window evidence when it is already selected", () => {
+    const selected = ["mcp-bridge", "openshell-credential-generation-window"];
+    expect(expandPrGateJobSelections(selected)).toEqual(selected);
+    expect(expectedSignalShards(selected)).toEqual({
+      "mcp-bridge": ["openclaw", "hermes", "deepagents"],
+      "openshell-credential-generation-window": ["default"],
+    });
+  });
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/pr-e2e-gate-signal-shards.test.ts` around lines 54 - 64, Extend the test
around expandPrGateJobSelections and expectedSignalShards to cover an input that
already includes openshell-credential-generation-window. Assert the expanded
selection contains each job only once, and verify expectedSignalShards returns
the mcp-bridge and credential-window shard entries without throwing.

Source: Path instructions

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@test/pr-e2e-gate-signal-shards.test.ts`:
- Around line 54-64: Extend the test around expandPrGateJobSelections and
expectedSignalShards to cover an input that already includes
openshell-credential-generation-window. Assert the expanded selection contains
each job only once, and verify expectedSignalShards returns the mcp-bridge and
credential-window shard entries without throwing.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: cccd98ef-c7c6-422b-9587-e84f98ebf1a8

📥 Commits

Reviewing files that changed from the base of the PR and between b2e9d1b and cd5351f.

📒 Files selected for processing (2)
  • test/pr-e2e-gate-signal-shards.test.ts
  • tools/e2e/pr-e2e-gate.mts

Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
@github-actions

github-actions Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: Review the warnings below.
Findings: 0 blockers · 1 warning · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 1 warning · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Model comparison: normalized findings differ; normalized E2E selections differ; Nemotron reported the same number of blockers, 1 fewer warning, the same number of suggestions.
2 additional E2E selections from the second opinion

Advisory only. The primary lane did not select these E2E jobs or targets.

  • mcp-bridge: The completed second-opinion lane identified E2E coverage that the primary lane omitted.
  • openshell-credential-generation-window: The completed second-opinion lane identified E2E coverage that the primary lane omitted.

Second-opinion E2E selections are advisory. They do not change the primary assessment or E2E / PR Gate.

E2E guidance

Advisory only. E2E / PR Gate selects and runs jobs independently.

Recommended E2E: cloud-inference, cloud-onboard, security-posture

1 warning · 0 suggestions

Warnings

Warnings do not block.

PRA-1 Warning — Cover coupled selection through dispatch state

  • Location: tools/e2e/pr-e2e-gate.mts:2562
  • Category: tests
  • Problem: The new coupling is tested only as a pure selection and shard function. No mocked gate-start test verifies that an MCP bridge risk plan dispatches and persists the added credential-window job.
  • Impact: A later change can preserve the helper test while bypassing expansion in the dispatch-to-state path, allowing final evidence classification to omit the credential-window result.
  • Recommendation: Add one mocked start/dispatch test for an MCP bridge plan. Assert the workflow jobs input and serialized expectedJobs include openshell-credential-generation-window, then verify classification fails when that signal is absent.
  • Verification: Inspect the gate-start test state and dispatch request after a plan selects mcp-bridge; both must include openshell-credential-generation-window before evidence classification.
  • Test coverage: A mocked gate-start regression test that selects mcp-bridge, inspects dispatch and persisted state for openshell-credential-generation-window, and rejects evidence without its default-shard signal.
  • Evidence: tools/e2e/pr-e2e-gate.mts:2562-2602 expands jobs before dispatch and persists the result in expectedJobs and expectedShards. test/pr-e2e-gate-signal-shards.test.ts:55-70 tests pure expansion and shard derivation only. test/pr-e2e-gate.test.ts:384-614 contains mocked dispatch coverage but no MCP bridge coupled-selection case.

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

@senthilr-nv
senthilr-nv merged commit ffc1ae6 into main Jul 31, 2026
93 of 97 checks passed
@senthilr-nv
senthilr-nv deleted the codex/fix-pr-e2e-coupled-mcp-evidence branch July 31, 2026 23:33
@senthilr-nv senthilr-nv mentioned this pull request Aug 1, 2026
23 tasks
senthilr-nv added a commit that referenced this pull request Aug 1, 2026
<!-- markdownlint-disable MD041 -->
## Summary

Adds the canonical dated changelog entry for `v0.0.100` so the
maintainer release plan can verify the pre-tag documentation
prerequisite. The entry summarizes the user-facing changes merged since
`v0.0.99` and links to the relevant guides.

## Changes

- Add `docs/changelog/2026-07-31.mdx` with the exact `## v0.0.100`
heading.
- Cover restored OpenClaw pairing, transactional replacement, Deep
Agents Code, onboarding recovery, lifecycle cleanup, Hermes builds, host
provenance, documentation, and trusted E2E evidence.
- Distinguish active Docker and Kubernetes runtime-bundle enforcement
from the still-inactive managed shared-state transaction foundation.

## Source Coverage

The release entry maps the doc-impacting merged PRs in the
`v0.0.99..main` release range to `docs/changelog/2026-07-31.mdx`: #8021,
#8024, #7973, #8028, #7947, #7788, #7884, #8023, #7969, #8020, #7989,
#8000, #7907, #7942, #7567, #8013, #7955, #8017, #8014, #8015, #7629,
#7644, #7821, #7971, and #7991.

PR #7974 was reviewed after the final rebase and excluded because it
changes internal maintainer-skill attribution policy and tests only; it
does not change a user-facing product or documentation surface.

## Type of Change

- [ ] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [x] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates

- [ ] Tests added or updated for changed behavior
- [x] Existing tests cover changed behavior — justification: the
changelog contract test validates the dated entry, version heading, SPDX
form, and route constraints.
- [ ] Tests not applicable — justification:
- [x] Docs updated for user-facing behavior changes
- [ ] Docs not applicable — justification:
- [ ] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification:
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Documentation Writer Review

- [x] Documentation writer subagent reviewed the completed changes
- Result: `docs-updated`
- Evidence: `docs/changelog/2026-07-31.mdx`; exact-head review passed
for `6093f44f`; writing rules and documentation style reviewed; `npx
vitest run test/changelog-docs.test.ts` passed 6/6; `npm run docs`
passed with zero Fern errors and two generic Fern upgrade notices.
- Agent: Codex Desktop
<!-- docs-review-head-sha: 6093f44 -->
<!-- docs-review-agents-blob-sha: 3dd7c24 -->

## DGX Station Hardware Evidence

- [ ] Tested on DGX Station
- Tested commit: Not applicable; no DGX Station host script changed.
- Station profile/scenario: Not applicable.
- Result: Not applicable.
- Supporting evidence: Not applicable.

## Verification

- [x] PR description includes a `Signed-off-by:` line and every commit
appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run validate:pr` passed after refreshing `origin/main` when hooks
were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — command/result or justification: `npx
vitest run test/changelog-docs.test.ts` passed 6/6 at `6093f44f`.
- [ ] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes — command/result: Not applicable to a dated
prose-only release entry.
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only) —
validation passed with zero errors; Fern emitted two generic upgrade
notices.
- [x] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)
— the changelog entry has the required parser-safe MDX SPDX header;
dated changelog entries intentionally do not use page frontmatter.

---

Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
  * Added release notes for v0.0.100.
* Documented improvements to restore pairing, sandbox replacement,
onboarding recovery, lifecycle cleanup, runtime handling, build support,
host readiness, and end-to-end validation.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: ci CI workflows, checks, release automation, or GitHub Actions area: e2e End-to-end tests, nightly failures, or validation infrastructure bug-fix PR fixes a bug or regression security

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants