ci(images): bootstrap inert protected multiarch lane - #8226
Conversation
Reconstruct the net #8077 terminal-outcome slice on current main. Signed-off-by: Aaron Erickson <aerickson@nvidia.com> (cherry picked from commit ce6f313e9f7705aa9374c0184710ca995dafdfa8)
Reconstruct the net #8078 shared-state authority slice on current main. Signed-off-by: Aaron Erickson <aerickson@nvidia.com> (cherry picked from commit 31236f767aa79c9e110be55bc1bf56b5396b227a)
Reconstruct the net #8080 journal-compatibility slice on current main. Signed-off-by: Aaron Erickson <aerickson@nvidia.com> (cherry picked from commit c52370db1119ec1b8f3365a0ce4c22beebdd28e4)
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com> (cherry picked from commit c3fea51)
Signed-off-by: Aaron Erickson <aerickson@nvidia.com> (cherry picked from commit 2060e9c)
Signed-off-by: Aaron Erickson <aerickson@nvidia.com> (cherry picked from commit 5976d9d)
Signed-off-by: Aaron Erickson <aerickson@nvidia.com> (cherry picked from commit c9a92ae)
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThe PR adds activation-aware release E2E preflight, managed-image contract publication, protected multiarch image builds with strict validation, workflow boundary checks, evidence tests, and risk-plan coverage. ChangesManaged image release qualification
Estimated code review effort: 5 (Critical) | ~120 minutes Sequence Diagram(s)sequenceDiagram
participant Controller
participant ManagedImageMultiarchStartup
participant Buildx
participant LocalRegistry
participant DirectE2e
participant EvidenceTest
Controller->>ManagedImageMultiarchStartup: dispatch with SHA, platform, cohort, run identity
ManagedImageMultiarchStartup->>Buildx: resolve bases and extract platform digests
ManagedImageMultiarchStartup->>LocalRegistry: start isolated registry
ManagedImageMultiarchStartup->>LocalRegistry: build and push three managed agents
ManagedImageMultiarchStartup->>DirectE2e: execute contract for each agent
DirectE2e->>EvidenceTest: validate and emit evidence
ManagedImageMultiarchStartup->>LocalRegistry: cleanup and verify ownership
ManagedImageMultiarchStartup->>Controller: upload shard artifacts and report result
Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Comment |
Code Coverage OverviewLanguages: TypeScript TypeScript / code-coverage/pluginThe overall coverage in commit 6a18088 in the TypeScript / code-coverage/cliThe overall coverage in commit 6a18088 in the Show a code coverage summary of the most impacted files.
Updated |
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
PR Review Advisor — No blocking findings reportedAdvisor assessment: No blocking advisor findings reported Model lanes
5 terminology differences from the second opinionAdvisory only. These are normalized differences from the primary terminology receipt.
3 additional E2E selections from the second opinionAdvisory only. The primary lane did not select these E2E jobs or targets.
Second-opinion terminology and E2E selections are advisory. They do not change the primary assessment or E2E / PR Gate. 3 semantic terminology decisionsTerminology decisions are advisory. They affect the assessment only when a separate finding identifies concrete semantic impact.
E2E guidanceAdvisory only. E2E / PR Gate selects and runs jobs independently. Recommended E2E: This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge. |
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (4)
test/e2e/live/managed-image-multiarch-startup-helpers.ts (1)
12-13: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winShare the SHA and cohort patterns with the contract module.
SHA_PATTERNandCOHORT_PATTERNhere duplicate the private patterns inscripts/checks/protected-managed-image-contract.ts(lines 64-66). The helper validates dispatch identity and the contract module validates evidence identity, so both must agree. If one copy changes, the two validators disagree and the mismatch appears only as a confusing runtime failure. Export the patterns from the contract module and import them here.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@test/e2e/live/managed-image-multiarch-startup-helpers.ts` around lines 12 - 13, Export SHA_PATTERN and COHORT_PATTERN from protected-managed-image-contract.ts, then import and reuse those symbols in the managed-image startup helpers instead of defining duplicate local patterns. Keep both validators aligned through the shared contract-module definitions.test/pr-risk-plan.test.ts (1)
316-330: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winScope the dormancy assertion to this family and cover the other new protected paths.
The negative case proves dormancy for one runtime file only. This PR adds several other protected paths, including
scripts/checks/build-protected-managed-images.sh,scripts/checks/protected-managed-image-contract.ts, andtest/e2e/live/managed-image-multiarch-startup.test.ts. The last path is matched by thee2e-control-planerule, sotoEqual([])cannot express dormancy for it.Assert instead that no family with id
managed-image-multiarchis selected for each pre-activation path. That assertion states the real claim and does not fail when an unrelated family starts matchingscripts/checks/.As per path instructions: "Migration tests must prove the superseded path is unreachable or removed, not merely prove that the new path also works."♻️ Proposed change to a family-scoped dormancy assertion
- const preActivationRuntime = plan("scripts/checks/run-managed-image-direct-e2e.ts"); + const preActivationPaths = [ + "scripts/checks/run-managed-image-direct-e2e.ts", + "scripts/checks/build-protected-managed-images.sh", + "scripts/checks/protected-managed-image-contract.ts", + "test/e2e/live/managed-image-multiarch-startup.test.ts", + ]; @@ expect(riskPlanRequiredJobIds(result)).toEqual(["managed-image-multiarch-startup"]); - expect(preActivationRuntime.families).toEqual([]); + for (const file of preActivationPaths) { + expect(plan(file).families.map((family) => family.id)).not.toContain( + "managed-image-multiarch", + ); + } });🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@test/pr-risk-plan.test.ts` around lines 316 - 330, Update the dormancy test around the activation marker to cover each protected pre-activation path, including the managed-image build script, contract file, and startup E2E test. For every path, assert that no selected family has id "managed-image-multiarch" rather than requiring families to be empty, preserving unrelated matches such as "e2e-control-plane".Source: Path instructions
tools/advisors/risk-plan.mts (1)
52-53: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winThe protected job id and activation path are re-declared in two places instead of imported.
scripts/checks/protected-managed-image-contract.tsexportsPROTECTED_MANAGED_IMAGE_MULTIARCH_JOB_IDandPROTECTED_MANAGED_IMAGE_ACTIVATION_PATHas the canonical identity for this lane. Both files below copy those values as local string literals. A rename in the canonical contract leaves the advisor matching a stale path and the boundary validator asserting a stale job id, with no failing check to expose the drift.
tools/advisors/risk-plan.mts#L52-L53: replace theMANAGED_IMAGE_MULTIARCH_ACTIVATIONliteral with the importedPROTECTED_MANAGED_IMAGE_ACTIVATION_PATH, and setrequiredJobson line 393 fromPROTECTED_MANAGED_IMAGE_MULTIARCH_JOB_ID.tools/e2e/managed-image-multiarch-workflow-boundary.mts#L15-L21: replace theJOB_IDandACTIVATION_PATHliterals with the same two imported constants, and buildSELECTORfrom the imported job id.As per path instructions: "Derive inventories and limits from a canonical source where possible; flag duplicated lists that can silently drift."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tools/advisors/risk-plan.mts` around lines 52 - 53, Replace duplicated string literals with the canonical constants exported from scripts/checks/protected-managed-image-contract.ts to prevent silent drift. In tools/advisors/risk-plan.mts at lines 52-53, replace the MANAGED_IMAGE_MULTIARCH_ACTIVATION literal with the imported PROTECTED_MANAGED_IMAGE_ACTIVATION_PATH constant, and update the requiredJobs assignment on line 393 to use the imported PROTECTED_MANAGED_IMAGE_MULTIARCH_JOB_ID. In tools/e2e/managed-image-multiarch-workflow-boundary.mts at lines 15-21, replace the local JOB_ID and ACTIVATION_PATH string literals with the same two imported constants, and derive the SELECTOR from the imported PROTECTED_MANAGED_IMAGE_MULTIARCH_JOB_ID.Source: Path instructions
test/e2e/mock-parity.json (1)
5-13: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd focused fast parity tests for the managed-image boundary and startup helper.
test/e2e/support/e2e-workflow.test.tsruns ine2e-supportand calls the validator, but it is not listed here and does not test managed-image mutations. No fast test coversmanaged-image-multiarch-startup-helpers.ts. Add focused tests and list them in this entry.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@test/e2e/mock-parity.json` around lines 5 - 13, Add focused fast parity tests for managed-image mutation boundaries and the helpers in managed-image-multiarch-startup-helpers.ts, including the validator flow exercised by test/e2e/support/e2e-workflow.test.ts. Add the new test paths to the fast array for the live managed-image-multiarch-startup.test.ts entry in test/e2e/mock-parity.json.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
@.agents/skills/nemoclaw-maintainer-cut-release-tag/scripts/release-e2e-evidence.mts:
- Around line 293-303: Update candidatePathExists to distinguish a confirmed
missing path from failures inspecting the candidate commit: return false only
for Git’s expected absent-path result, and propagate errors indicating a missing
candidate object, executable failure, or repository problem. Add a regression
test covering an inspection failure and preserve the preflight behavior at the
caller around candidatePathExists.
In @.github/workflows/base-image.yaml:
- Around line 551-571: The mutable first_tag can be retagged between imagetools
create and imagetools inspect, causing digest to reference a different valid
index than this run's images. At the anchor site
(.github/workflows/base-image.yaml lines 551-571), after the imagetools inspect
of "$reference" succeeds, extract the linux/amd64 and linux/arm64 child
descriptors from the raw index data and reject with an error if they do not
equal platform_digests[linux/amd64] and platform_digests[linux/arm64]
respectively, then pass this validation check before calling
export-managed-base-image-contract.sh. Apply the identical child-descriptor
equality validation check at the sibling sites
(.github/workflows/base-image.yaml lines 695-715 and 841-861) before their
respective export script invocations, ensuring all three contract exports only
proceed when the manifest's platform children match this run's built digests.
In @.github/workflows/e2e.yaml:
- Around line 1727-1741: Add NEMOCLAW_E2E_EXPECTED_SHA to the job-level env
block with the value from inputs.checkout_sha, alongside the existing protected
managed-image identity variables such as
NEMOCLAW_PROTECTED_MANAGED_IMAGE_BASE_SHA and
NEMOCLAW_PROTECTED_MANAGED_IMAGE_WORKFLOW_SHA, so
protectedManagedImageDispatchEnvironment() receives the required value.
In `@tools/advisors/risk-plan.mts`:
- Line 397: Update the invariant text near the amd64 and arm64 shard requirement
to remove the duplicated “base” term, preserving one occurrence and the
remaining evidence-field wording unchanged.
---
Nitpick comments:
In `@test/e2e/live/managed-image-multiarch-startup-helpers.ts`:
- Around line 12-13: Export SHA_PATTERN and COHORT_PATTERN from
protected-managed-image-contract.ts, then import and reuse those symbols in the
managed-image startup helpers instead of defining duplicate local patterns. Keep
both validators aligned through the shared contract-module definitions.
In `@test/e2e/mock-parity.json`:
- Around line 5-13: Add focused fast parity tests for managed-image mutation
boundaries and the helpers in managed-image-multiarch-startup-helpers.ts,
including the validator flow exercised by test/e2e/support/e2e-workflow.test.ts.
Add the new test paths to the fast array for the live
managed-image-multiarch-startup.test.ts entry in test/e2e/mock-parity.json.
In `@test/pr-risk-plan.test.ts`:
- Around line 316-330: Update the dormancy test around the activation marker to
cover each protected pre-activation path, including the managed-image build
script, contract file, and startup E2E test. For every path, assert that no
selected family has id "managed-image-multiarch" rather than requiring families
to be empty, preserving unrelated matches such as "e2e-control-plane".
In `@tools/advisors/risk-plan.mts`:
- Around line 52-53: Replace duplicated string literals with the canonical
constants exported from scripts/checks/protected-managed-image-contract.ts to
prevent silent drift. In tools/advisors/risk-plan.mts at lines 52-53, replace
the MANAGED_IMAGE_MULTIARCH_ACTIVATION literal with the imported
PROTECTED_MANAGED_IMAGE_ACTIVATION_PATH constant, and update the requiredJobs
assignment on line 393 to use the imported
PROTECTED_MANAGED_IMAGE_MULTIARCH_JOB_ID. In
tools/e2e/managed-image-multiarch-workflow-boundary.mts at lines 15-21, replace
the local JOB_ID and ACTIVATION_PATH string literals with the same two imported
constants, and derive the SELECTOR from the imported
PROTECTED_MANAGED_IMAGE_MULTIARCH_JOB_ID.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 2f21654e-117d-485b-95cb-5a53ef9c4ec3
📒 Files selected for processing (29)
.agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md.agents/skills/nemoclaw-maintainer-cut-release-tag/scripts/release-e2e-evidence.mts.agents/skills/nemoclaw-maintainer-policies/references/release-train.md.github/workflows/base-image.yaml.github/workflows/e2e.yamlscripts/checks/build-protected-managed-images.shscripts/checks/protected-managed-image-contract.tsscripts/checks/run-managed-image-direct-e2e.tsscripts/export-managed-base-image-contract.shtest/dcode-base-image-workflow.test.tstest/e2e/live/managed-image-multiarch-startup-helpers.tstest/e2e/live/managed-image-multiarch-startup.test.tstest/e2e/mock-parity.jsontest/e2e/support/base-image-publication.test.tstest/e2e/support/e2e-cross-runtime-compatibility.test.tstest/helpers/vitest-watch-triggers.tstest/maintainer-skills-policy.test.tstest/managed-base-image-contract.test.tstest/managed-image-publication-workflow.test.tstest/pr-risk-plan.test.tstest/protected-managed-image-contract.test.tstest/release-e2e-evidence.test.tstest/vitest-watch-triggers.test.tstools/advisors/risk-plan.mtstools/e2e/base-image-publication.mtstools/e2e/managed-image-multiarch-workflow-boundary.mtstools/e2e/prepare-e2e-workflow-boundary.mtstools/e2e/upload-e2e-artifacts-workflow-boundary.mtstools/e2e/workflow-boundary.mts
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
test/managed-image-publication-workflow.test.ts (1)
125-133: 📐 Maintainability & Code Quality | 🟠 Major | 🏗️ Heavy liftTest the workflow package resolver through its executable boundary.
isStrictChildPathis a test-local copy of the path-confinement algorithm. The later assertions inspect workflow text fragments. These checks can pass if the workflow stops applying the resolver or changes its effective behavior.Extract the workflow resolver into a repository entrypoint that the workflow invokes. Test that entrypoint with the symlink fixture. Keep workflow-boundary tests focused on the workflow invocation and inputs.
As per path instructions, “Prefer observable outcomes through the public boundary over source-text, private-shape, or mock-call assertions” and “Flag copied production algorithms.”
Also applies to: 266-282, 297-325
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@test/managed-image-publication-workflow.test.ts` around lines 125 - 133, Replace the test-local isStrictChildPath algorithm and workflow text-fragment assertions with an executable repository entrypoint for the workflow resolver. Update the workflow to invoke that entrypoint, then test the entrypoint using the existing symlink fixture and keep workflow-boundary tests limited to invocation inputs and observable outcomes.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@test/managed-image-publication-workflow.test.ts`:
- Around line 125-133: Replace the test-local isStrictChildPath algorithm and
workflow text-fragment assertions with an executable repository entrypoint for
the workflow resolver. Update the workflow to invoke that entrypoint, then test
the entrypoint using the existing symlink fixture and keep workflow-boundary
tests limited to invocation inputs and observable outcomes.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 684738d9-7ca4-4877-8a50-3667be359fdd
📒 Files selected for processing (1)
test/managed-image-publication-workflow.test.ts
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
|
Addressed the prior advisor PRA-1 warning in 99b09ae. The checked-in subprocess test now runs both risk-plan and managed-image workflow-boundary consumers under the exact node --import tsx loader, proves the protected job ID selects correctly, and proves the activation path survives the normalized CJS/ESM import shape. The focused test and repository growth guards pass locally. |
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
## Summary - activates the already-trusted exact-head all-agent managed-image startup lane for native amd64 and arm64 - adds an exact-image production managed-bootstrap harness for OpenClaw, Hermes, and Deep Agents Code - adds a dormant protected amd64 runtime lane covering real NVIDIA GPU access, host-local Ollama, NVIDIA NIM, vLLM, transactional bootstrap rollback, and exact owned cleanup for every agent - keeps the new runtime lane explicit-only until a follow-on activation marker can qualify it from trusted main ## Stack - based on merged #8226 commit `08435d31e45ce91391d884a94f5ae59caf5701d5`; signed merge `beeb0d990f38b4db5809a7ad8a87aaaf39215947` preserves B3-C patch ID `784174a359c3f95d1898eef01075e5bd6c995afd`, and exact repair head `59fbc1af816b759c102f391ec289cc0f2023a231` refreshes the two intended workflow/risk-plan compatibility hashes - tracks #7744 - does not advertise or activate user-visible buildless support ## Validation - `npm run checks:repository` - `npm run typecheck:cli` - `npm run source-shape:check` - `npm run test-size:check` - `npm run test-conditionals:scan -- --top 25` - `npm run test:e2e-phases:check` - 9 protected runtime contract tests - 4 protected runtime workflow-boundary tests - 95 risk-plan and runtime contract tests - 50 E2E workflow, upload, preparation, mock-parity, and PR-gate tests ## Safety boundary The trusted workflow and its fail-closed validator land before activation. The NVIDIA credential is scoped only to host-side NGC/NIM setup; managed sandboxes receive generated local route tokens, and the E2E artifact sink redacts the workflow secret. Signed-off-by: Aaron Erickson <aerickson@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added protected managed-image GPU qualification for local inference, startup, rollback, cleanup, and container health. - Added multi-architecture activation for Linux AMD64 and ARM64 environments. - Added support for Ollama, NVIDIA NIM, and vLLM inference runtimes. - Added isolated runtime validation and qualification artifacts. - **Security** - Strengthened safeguards for trusted sources, credentials, immutable images, and cleanup. - **Tests** - Expanded end-to-end, contract, workflow, risk-plan, and mock-parity coverage. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Aaron Erickson <aerickson@nvidia.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Summary
Hardens managed-image publication evidence and adds the trusted protected multi-architecture qualification lane while keeping it fail-closed and inert. This is the consolidated B3-B batch stacked on #8225; it does not add the activation marker, select the lane for candidate changes, or advertise buildless support.
Related Issue
Refs #7744
Depends on #8225.
Changes
linux/amd64andlinux/arm64protected build, platform-digest, cohort, and evidence contracts.ci/protected-managed-image-multiarch-activation-v1.jsonmarker, so this candidate cannot select or execute its own newly introduced trusted lane.backup/podman-stack/pr8065-source-2860766e,backup/podman-stack/pr8065-public-head-eea54e92,backup/podman-stack/pr8068-source-2060e9cc,backup/podman-stack/pr8075-source-5976d9d0, andbackup/podman-stack/pr3-14b2a-source-c9a92ae7.The shared publication and protected-lane contracts are required by all three managed-image consumers. Per-agent workflow copies or candidate-authored dispatch would allow schema drift or weaken the trusted-main boundary. The managed base-contract, publication workflow, protected contract, risk-plan, direct-input, workflow-boundary, release-evidence, and publication-support tests protect the shared seam.
Type of Change
Quality Gates
Documentation Writer Review
docs-updated.agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.mdand.agents/skills/nemoclaw-maintainer-policies/references/release-train.md. Both now define thatRELEASE_E2E_ACTIVATION_PATHadmits an explicit-only job and its matrix executions into the release denominator only when the exact relative path exists at the candidate SHA. They consistently direct maintainers not to dispatch or require evidence for a dormant lane.DGX Station Hardware Evidence
Verification
Signed-off-by:line and every commit appears asVerifiedin GitHubpre-commit,commit-msg, andpre-pushhooks passed, ornpm run validate:prpassed after refreshingorigin/mainwhen hooks were skipped or unavailableb166aebdd.npm testfor broad runtime/test-harness changes;npm run checkfor repo-wide validation/coverage changes — command/result: Authoritative sharded CI is pending for the exact head.npm run docsbuilds without warnings (doc changes only)Signed-off-by: Aaron Erickson aerickson@nvidia.com
Summary by CodeRabbit
New Features
Bug Fixes
Tests