test(e2e): cover Hermes managed policy changes

[jyaunches]

Summary

Hermes managed-policy changes currently receive only the generic sandbox E2E floor, and the Bedrock Runtime compatible Anthropic job runs only its OpenClaw shard. This change selects every live E2E job required by #8008 and enables the existing Hermes-aware Bedrock test through the trusted Hermes runner setup.

Changes

• Select the seven Hermes managed-policy live E2E jobs when the policy generator, schema, consumers, or validation probes change.
• Run the Bedrock Runtime compatible Anthropic job for OpenClaw and Hermes.
• Provision the existing trusted Hermes swap before checkout for the Hermes Bedrock shard.
• Pin the expanded job set, shard set, workflow matrix, and trusted-runner condition in boundary tests.

Type of Change

• Code change (feature, bug fix, or refactor)
• Code change with doc updates
• Doc only (prose changes, no code sample modifications)
• Doc only (includes code sample changes)

Quality Gates

• Tests added or updated for changed behavior
• Existing tests cover changed behavior — justification:
• Tests not applicable — justification:
• Docs updated for user-facing behavior changes
• Docs not applicable — justification: This changes maintainer-owned PR risk planning and trusted live E2E execution only. It does not change a supported user-facing API, CLI, configuration, workflow, default, or error.
• Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
• Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Codex Desktop reviewed the completed diff through commit e2a01b309 against the nine-category security checklist. No findings were identified; trusted dispatch conditions remain exact, fork-controlled code does not run before the trusted swap step, no new secrets are exposed, and gitleaks passed.
• Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

• Documentation writer subagent reviewed the completed changes
• Result: no-docs-needed
• Evidence: The change only updates maintainer-owned PR risk planning, workflow boundaries, trusted live E2E selection, and their regression coverage. It does not change a supported user-facing API, CLI, configuration, workflow, default, or error.
• Agent: Codex Desktop

DGX Station Hardware Evidence

• Tested on DGX Station
• Tested commit:
• Station profile/scenario:
• Result:
• Supporting evidence:

Verification

• PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
• Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run validate:pr passed after refreshing origin/main when hooks were skipped or unavailable
• Targeted behavior tests pass for the current change set, or tests are marked not applicable above — 83 focused risk-plan and compatibility tests passed; the affected e2e-support lane passed. The earlier complete PR E2E controller and risk-plan suite passed 302 tests.
• Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result:
• Quality Gates section completed with required justifications or waivers
• No secrets, API keys, or credentials committed
• npm run docs builds without warnings (doc changes only)
• Doc pages follow the style guide (doc changes only)
• New doc pages include SPDX header and frontmatter (new pages only)

---

Signed-off-by: Julie Yaunches jyaunches@nvidia.com

Summary by CodeRabbit

• New Features

  • Bedrock Runtime-compatible Anthropic end-to-end testing now supports both OpenClaw and Hermes agents.
  • Hermes test runs include trusted environment validation before execution.

• Bug Fixes

  • Improved change detection so relevant Hermes policy updates select all required end-to-end jobs.
  • Updated risk and workflow checks to cover the expanded agent test matrix.
  • Refined runtime compatibility validation for the expanded testing coverage.

[coderabbitai]

📝 Walkthrough

Walkthrough

The Bedrock-compatible Anthropic E2E job now runs OpenClaw and Hermes agents. Eligible Hermes runs provision the trusted swap setup. Risk-plan selection now includes Hermes policy jobs and related runtime files.

Changes

Hermes Bedrock E2E integration

Layer / File(s)	Summary
Hermes policy risk selection tools/advisors/risk-plan.mts, test/pr-risk-plan.test.ts, test/e2e/support/e2e-cross-runtime-compatibility.test.ts	Risk-plan version 11 adds Hermes runtime mappings and selects matching managed-startup and Hermes policy jobs. Tests cover all seven required E2E jobs and unrelated runtime files.
Bedrock Hermes E2E workflow tools/e2e/workflow-boundary.mts, tools/e2e/trusted-hermes-swap-workflow-boundary.mts, test/e2e/support/trusted-hermes-swap-workflow-boundary.test.ts, test/pr-e2e-gate-shards.test.ts, .github/workflows/e2e.yaml	The Bedrock-compatible Anthropic matrix includes OpenClaw and Hermes. Eligible Hermes runs provision the trusted swap setup. Boundary checks, shard expectations, and runtime probe documentation cover both agents.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant GitHubActions
  participant TrustedHermesSwapSetup
  participant HermesRuntimeProbe
  GitHubActions->>TrustedHermesSwapSetup: provision trusted swap for Hermes
  TrustedHermesSwapSetup-->>GitHubActions: provide validated setup
  GitHubActions->>HermesRuntimeProbe: run Bedrock-compatible Anthropic probes

Loading

Suggested labels: integration: hermes, area: e2e, area: ci

Suggested reviewers: jason-ma-nv

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the main change: adding E2E coverage for Hermes managed-policy changes.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/pr-risk-plan.test.ts`:
- Around line 120-155: Extend the risk-plan tests around the existing Hermes
matcher case to use table-driven coverage for each exact path:
hermes-wrapper.py, image-build-probes.py, patch-profile-policy-defaults.py, and
start.sh, while preserving the existing config-prefix and explicit-file
coverage. Add an unrelated agents/hermes runtime file case that must not select
the focused jobs, and assert both detection and false-positive behavior through
the existing plan and riskPlanRequiredJobIds helpers.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 7050dc56-ca59-4469-8046-5eef09a9a37c

📥 Commits

Reviewing files that changed from the base of the PR and between 98a0e0c and 8514595.

📒 Files selected for processing (7)

• .github/workflows/e2e.yaml
• test/e2e/support/trusted-hermes-swap-workflow-boundary.test.ts
• test/pr-e2e-gate-shards.test.ts
• test/pr-risk-plan.test.ts
• tools/advisors/risk-plan.mts
• tools/e2e/trusted-hermes-swap-workflow-boundary.mts
• tools/e2e/workflow-boundary.mts

[github-actions]

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions

Model lanes

• GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
• Nemotron 3 Ultra (second opinion): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
• Model comparison: normalized findings match; normalized terminology decisions differ; normalized E2E selections differ; severity counts match.

1 additional E2E selection from the second opinion

Advisory only. The primary lane did not select these E2E jobs or targets.

• bootstrap-install-smoke: The completed second-opinion lane identified E2E coverage that the primary lane omitted.

Second-opinion terminology and E2E selections are advisory. They do not change the primary assessment or E2E / PR Gate.

2 semantic terminology decisions

Terminology decisions are advisory. They affect the assessment only when a separate finding identifies concrete semantic impact.

• established — agent runtime at .github/workflows/e2e.yaml:5075: Keep `agent runtime probes`.
• established — managed policy at test/pr-risk-plan.test.ts:141: Keep `managed-policy` for this established file-group distinction.

E2E guidance

Advisory only. E2E / PR Gate selects and runs jobs independently.

Recommended E2E: cloud-inference, cloud-onboard, security-posture

1 optional E2E recommendation

• bedrock-runtime-compatible-anthropic

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.