Skip to content

fix(e2e): pin old gateway base fallback#4047

Merged
jyaunches merged 1 commit into
mainfrom
fix/openshell-gateway-upgrade-base-pin
May 22, 2026
Merged

fix(e2e): pin old gateway base fallback#4047
jyaunches merged 1 commit into
mainfrom
fix/openshell-gateway-upgrade-base-pin

Conversation

@jyaunches
Copy link
Copy Markdown
Contributor

@jyaunches jyaunches commented May 22, 2026

Summary

  • force the old gateway-upgrade Docker wrapper to add the historical BASE_IMAGE build arg when the old installer does not pass one
  • keep the old install fixture pinned to the intended sandbox base digest instead of falling through to the mutable current base
  • preserve wrapper diagnostics by logging the added BASE_IMAGE fallback

Why

The openshell-gateway-upgrade E2E prepares a v0.0.36 install before exercising the current upgrade path. PR #4041 pinned the old OpenClaw version, but if the old installer does not pass BASE_IMAGE=ghcr.io/nvidia/nemoclaw/sandbox-base:latest, the wrapper only logged that no override was seen and left the old Dockerfile default in place. That allowed the fixture to consume a current base containing OpenClaw 2026.5.18, causing the old rcf_patch.py Patch 4 to fail before the upgrade path ran.

Test plan

  • bash -n test/e2e/test-openshell-gateway-upgrade.sh
  • git diff --check
  • npm run typecheck:cli
  • npm run test -- test/onboard-openshell-version.test.ts

Fixes the nightly openshell-gateway-upgrade-e2e preparation failure after PR #4041.

Summary by CodeRabbit

  • Tests
    • Enhanced Docker wrapper generator logic for upgrade regression testing to properly handle base image configuration when no override is specified, ensuring accurate testing scenarios during gateway upgrades.

Review Change Stack

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 22, 2026

📝 Walkthrough

Walkthrough

This PR modifies the create_old_docker_wrapper function in the E2E gateway upgrade test script to automatically inject a BASE_IMAGE build-arg when no override is already present, ensuring the "old install" Docker build path uses the pinned old sandbox base image reference during regression testing.

Changes

Docker base image override for gateway upgrade testing

Layer / File(s) Summary
Base image build-arg injection in old Docker wrapper
test/e2e/test-openshell-gateway-upgrade.sh
When rewrote_base is unset, the Docker wrapper now injects a BASE_IMAGE=${base_ref} build-arg and logs the addition, replacing the prior behavior that left the Dockerfile's default BASE_IMAGE unchanged.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~5 minutes

Possibly related PRs

  • NVIDIA/NemoClaw#4041: Directly refines the same create_old_docker_wrapper() logic by explicitly injecting a pinned BASE_IMAGE build-arg when no override is detected, representing a continuation of the wrapper's build-arg rewriting behavior.

Suggested labels

fix, E2E, Docker, OpenShell, CI/CD

Suggested reviewers

  • cv

Poem

🐰 A wrapper revised with care divine,
BASE_IMAGE args now tightly aligned!
Old sandbox images stand tall and true,
Gateway upgrades tested through and through. ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'fix(e2e): pin old gateway base fallback' directly relates to the main change: updating Docker wrapper logic to pin the old sandbox base image. It is concise, specific, and accurately summarizes the primary intent of the changeset.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/openshell-gateway-upgrade-base-pin

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Copy Markdown
Contributor

E2E Advisor Recommendation

Required E2E: None
Optional E2E: openshell-gateway-upgrade-e2e

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

  • None. No merge-blocking E2E is required because this PR changes only an existing E2E test script and does not modify installer, onboarding, sandbox lifecycle, credentials, policy, inference routing, deployment, or user-facing runtime code. Running the touched E2E job is useful as an optional confidence check for CI harness correctness.

Optional E2E

  • openshell-gateway-upgrade-e2e (high): Optional validation that the modified gateway-upgrade E2E harness still executes successfully and correctly provisions the old sandbox base image before testing the OpenShell gateway upgrade path.

New E2E recommendations

  • None.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/e2e/test-openshell-gateway-upgrade.sh`:
- Around line 176-177: The fallback BASE_IMAGE arg is being appended even when
an explicit non-`latest` BASE_IMAGE was provided because `rewrote_base` is only
set when matching the `:latest` pattern; update the logic that decides to append
(the code around `args+=("--build-arg" "BASE_IMAGE=${base_ref}")` and the
`rewrote_base` flag) to detect any explicit override: either set
`rewrote_base=true` when an explicit BASE_IMAGE build-arg was passed in (or when
`base_ref` was set from user input), or instead check `args` for an existing
`--build-arg` "BASE_IMAGE=" entry before adding the fallback; ensure the log
write to `"$log_file"` remains unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 41b18465-c8a2-4b9c-800f-c8090c242164

📥 Commits

Reviewing files that changed from the base of the PR and between 971c526 and c452b76.

📒 Files selected for processing (1)
  • test/e2e/test-openshell-gateway-upgrade.sh

Comment on lines +176 to +177
args+=("--build-arg" "BASE_IMAGE=${base_ref}")
printf 'add build-arg BASE_IMAGE=%s\n' "$base_ref" >>"$log_file"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Fallback currently overrides explicit non-latest BASE_IMAGE args.

This add-path is correct only if rewrote_base truly means “a BASE_IMAGE override already exists.” Right now it is set only for the :latest pattern, so an explicit custom BASE_IMAGE=... still gets a second arg appended and overridden by this fallback.

Suggested fix
 while [ "$#" -gt 0 ]; do
   case "$1" in
     --build-arg)
+      if [ "$#" -ge 2 ] && [ "${2#BASE_IMAGE=}" != "$2" ]; then
+        rewrote_base=1
+      fi
       if [ "$#" -ge 2 ] && [ "${2#OPENCLAW_VERSION=}" != "$2" ]; then
         args+=("--build-arg" "OPENCLAW_VERSION=${old_openclaw}")
         rewrote_openclaw=1
         printf 'rewrite build-arg %s -> OPENCLAW_VERSION=%s\n' "$2" "$old_openclaw" >>"$log_file"
         shift 2
         continue
       fi
       if [ "$#" -ge 2 ] && [ "$2" = "BASE_IMAGE=ghcr.io/nvidia/nemoclaw/sandbox-base:latest" ]; then
         args+=("--build-arg" "BASE_IMAGE=${base_ref}")
         rewrote_base=1
         printf 'rewrite build-arg %s -> BASE_IMAGE=%s\n' "$2" "$base_ref" >>"$log_file"
         shift 2
         continue
       fi
       ;;
+    --build-arg=BASE_IMAGE=*)
+      rewrote_base=1
+      ;;
     --build-arg=OPENCLAW_VERSION=*)
       args+=("--build-arg=OPENCLAW_VERSION=${old_openclaw}")
       rewrote_openclaw=1
       printf 'rewrite build-arg %s -> OPENCLAW_VERSION=%s\n' "$1" "$old_openclaw" >>"$log_file"
       shift
       continue
       ;;
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/e2e/test-openshell-gateway-upgrade.sh` around lines 176 - 177, The
fallback BASE_IMAGE arg is being appended even when an explicit non-`latest`
BASE_IMAGE was provided because `rewrote_base` is only set when matching the
`:latest` pattern; update the logic that decides to append (the code around
`args+=("--build-arg" "BASE_IMAGE=${base_ref}")` and the `rewrote_base` flag) to
detect any explicit override: either set `rewrote_base=true` when an explicit
BASE_IMAGE build-arg was passed in (or when `base_ref` was set from user input),
or instead check `args` for an existing `--build-arg` "BASE_IMAGE=" entry before
adding the fallback; ensure the log write to `"$log_file"` remains unchanged.

@jyaunches jyaunches merged commit 6431f33 into main May 22, 2026
30 checks passed
@github-actions
Copy link
Copy Markdown
Contributor

PR Review Advisor

Recommendation: blocked
Confidence: high
Analyzed HEAD: c452b76b6b91526701ba7593986897c66ee61cd0
Findings: 2 blocker(s), 2 warning(s), 0 suggestion(s)

This is an automated advisory review. A human maintainer must make the final merge decision.

Limitations: Review is based on trusted metadata and the provided diff; no commands, package-manager operations, or E2E scripts were executed.; The full file was not re-read beyond the provided diff, so line references are based on trusted review metadata and diff hunk locations.; No linked issues were present; acceptance coverage uses PR body clauses and E2E advisor comment clauses as available acceptance evidence.; The optional openshell-gateway-upgrade-e2e result for this exact head SHA was not provided.

Workflow run

Full advisor summary

PR Review Advisor

Base: origin/main
Head: HEAD
Analyzed SHA: c452b76b6b91526701ba7593986897c66ee61cd0
Recommendation: blocked
Confidence: high

Small E2E harness fix is plausible, but merge is blocked by mergeStateStatus=BLOCKED, one unresolved review thread, and a likely correctness bug where the new fallback can override explicit non-latest BASE_IMAGE build args.

Gate status

  • CI: pass — 5 required status context(s) completed with no failures for c452b76: checks, commit-lint, dco-check, check-hash, changes. Non-required contexts pending: 0; failed: 0.
  • Mergeability: fail — GraphQL mergeStateStatus=BLOCKED for PR fix(e2e): pin old gateway base fallback #4047 at head c452b76.
  • Review threads: fail — 1 unresolved review thread exists on test/e2e/test-openshell-gateway-upgrade.sh around line 177.
  • Risky code tested: pass — Trusted path heuristics detected no risky runtime code areas; change is limited to an existing E2E test script. Required CI passed, and E2E advisor required no merge-blocking E2E.

🔴 Blockers

  • PR is not mergeable under current repository gates: The latest trusted PR metadata reports mergeStateStatus=BLOCKED even though required CI contexts passed.
    • Recommendation: Resolve the repository merge block before merging; verify the final head SHA still has passing required checks.
    • Evidence: gateStatus.mergeability evidence: mergeStateStatus=BLOCKED for head c452b76.
  • Unresolved review thread on BASE_IMAGE fallback behavior (test/e2e/test-openshell-gateway-upgrade.sh:177): There is one unresolved review thread. The thread raises a correctness concern that the fallback appends BASE_IMAGE whenever rewrote_base remains 0, but rewrote_base is only set when the old latest image literal is rewritten.
    • Recommendation: Resolve the review thread by either fixing the detection of any explicit BASE_IMAGE build arg or documenting why the wrapper intentionally overrides all non-latest explicit BASE_IMAGE values in this E2E fixture.
    • Evidence: GraphQL reviewThreads shows isResolved=false; review comment at line 177 says fallback can override explicit non-latest BASE_IMAGE args.

🟡 Warnings

  • Fallback can append a second BASE_IMAGE and override explicit non-latest values (test/e2e/test-openshell-gateway-upgrade.sh:177): The new fallback adds --build-arg BASE_IMAGE=${base_ref} when rewrote_base is 0. In the visible wrapper logic, rewrote_base is only set for BASE_IMAGE=ghcr.io/nvidia/nemoclaw/sandbox-base:latest, not for other explicit BASE_IMAGE build args. Docker build args are typically last-one-wins, so an explicit non-latest BASE_IMAGE passed by the old installer could be overridden unexpectedly by the fallback.
    • Recommendation: Set rewrote_base when any BASE_IMAGE build arg is present, or explicitly check the accumulated args for a BASE_IMAGE= entry before appending the fallback. Add/extend a shell-level regression case for an explicit non-latest BASE_IMAGE build arg.
    • Evidence: Diff adds args+=("--build-arg" "BASE_IMAGE=${base_ref}") under if [ "$rewrote_base" = "0" ]; then, while earlier cases only set rewrote_base for the exact latest image literal.
  • Active overlapping PRs touch the same E2E upgrade script (test/e2e/test-openshell-gateway-upgrade.sh): Trusted drift evidence shows this file exists and has recent related history. Open PR overlap metadata reports PR chore: bump OpenShell pin to 0.0.44 #3830 and PR chore: upgrade agent runtime dependencies #3925 also touch test/e2e/test-openshell-gateway-upgrade.sh, so this change may conflict with active OpenShell/runtime dependency work.

🔵 Suggestions

  • None.

Acceptance coverage

  • partial — force the old gateway-upgrade Docker wrapper to add the historical BASE_IMAGE build arg when the old installer does not pass one: Diff adds args+=("--build-arg" "BASE_IMAGE=${base_ref}") when rewrote_base=0, satisfying the no-BASE_IMAGE fallback path. Coverage is partial because the current detection may also append the fallback when an explicit non-latest BASE_IMAGE was passed.
  • partial — keep the old install fixture pinned to the intended sandbox base digest instead of falling through to the mutable current base: The fallback passes BASE_IMAGE=${base_ref}, sourced from NEMOCLAW_OLD_SANDBOX_BASE_IMAGE_REF, which should pin the old fixture when no override is present. However, explicit non-latest BASE_IMAGE handling remains ambiguous due to the unresolved review thread.
  • met — preserve wrapper diagnostics by logging the added BASE_IMAGE fallback: Diff adds printf 'add build-arg BASE_IMAGE=%s\n' "$base_ref" >>"$log_file" immediately after appending the fallback build arg.
  • unknown — The openshell-gateway-upgrade E2E prepares a v0.0.36 install before exercising the current upgrade path.: The provided diff does not include the surrounding version setup or actual E2E execution result for openshell-gateway-upgrade-e2e; this statement is PR-body context only.
  • met — if the old installer does not pass BASE_IMAGE=ghcr.io/nvidia/nemoclaw/sandbox-base:latest, the wrapper only logged that no override was seen and left the old Dockerfile default in place.: Removed line logged 'no BASE_IMAGE override seen; old Dockerfile default remains unchanged' and did not append a BASE_IMAGE argument.
  • unknown — That allowed the fixture to consume a current base containing OpenClaw 2026.5.18, causing the old rcf_patch.py Patch 4 to fail before the upgrade path ran.: The failure mode is plausible from the changed fallback behavior, but no nightly log or E2E run output was provided in the trusted context to independently verify the exact OpenClaw 2026.5.18/Patch 4 failure.
  • unknownbash -n test/e2e/test-openshell-gateway-upgrade.sh: PR body lists this in the test plan, but trusted gate data does not include command-level evidence for this exact command.
  • unknowngit diff --check: PR body lists this in the test plan, but trusted gate data does not include command-level evidence for this exact command.
  • unknownnpm run typecheck:cli: PR body lists this in the test plan, but trusted gate data does not include command-level evidence for this exact command.
  • unknownnpm run test -- test/onboard-openshell-version.test.ts: PR body lists this in the test plan, but trusted gate data does not include command-level evidence for this exact command.
  • partial — Fixes the nightly openshell-gateway-upgrade-e2e preparation failure after PR fix(e2e): pin old gateway upgrade install #4041.: The diff targets the described preparation path by adding the missing fallback BASE_IMAGE build arg. E2E advisor marked openshell-gateway-upgrade-e2e optional, and no passed openshell-gateway-upgrade-e2e run for this head SHA was provided.
  • met — Required E2E: None: Trusted E2E advisor comment says no merge-blocking E2E is required because the PR changes only an existing E2E test script and not installer, onboarding, sandbox lifecycle, credentials, policy, routing, deployment, or runtime code.
  • partial — Optional E2E: openshell-gateway-upgrade-e2e: The E2E advisor recommends openshell-gateway-upgrade-e2e as optional confidence validation. The trusted context does not show that optional job passed for this head SHA.

Security review

  • pass — 1. Secrets and Credentials: No hardcoded secrets, tokens, passwords, private keys, or credential files are added. The wrapper references existing environment variables for version/base-image configuration and logs image references, not credentials.
  • warning — 2. Input Validation and Data Sanitization: The shell wrapper processes Docker CLI arguments and now appends a BASE_IMAGE fallback. Argument values are array-quoted, which avoids shell-string injection, but the current rewrote_base detection only recognizes the latest image literal and may mishandle explicit non-latest BASE_IMAGE input.
  • pass — 3. Authentication and Authorization: Not applicable to auth flows; this change modifies an E2E Docker wrapper script and adds no endpoints, permissions, token validation, or authorization logic.
  • pass — 4. Dependencies and Third-Party Libraries: No dependencies, package manifests, registries, or third-party library versions are changed.
  • pass — 5. Error Handling and Logging: The change preserves diagnostic logging and adds a log line for the fallback BASE_IMAGE. The logged value is an image reference from the configured old sandbox base, not a secret. The wrapper remains set -euo pipefail.
  • pass — 6. Cryptography and Data Protection: Not applicable — no cryptographic operations, key handling, hashing, encryption, or data-protection behavior is changed.
  • warning — 7. Configuration and Security Headers: The change affects Docker build configuration in an E2E harness by forcing a BASE_IMAGE fallback. This is security-relevant for test fidelity because sandbox base image pinning matters, but the diff does not expose ports, weaken headers, or alter production container defaults. Warning is due to the possible override of explicit BASE_IMAGE configuration.
  • warning — 8. Security Testing: Required CI passed and E2E advisor required no blocking E2E, but no direct evidence was provided for a regression test covering explicit non-latest BASE_IMAGE build args or the optional openshell-gateway-upgrade-e2e job.
  • warning — 9. Holistic Security Posture: The intent improves test determinism by avoiding a mutable current sandbox base in an old-install fixture. However, the unresolved review thread indicates the wrapper may unintentionally override caller-provided BASE_IMAGE values, which could mask fixture configuration problems.

Test / E2E status

  • Test depth: unit_sufficient — Trusted context classifies the change as limited to tests/E2E harness code that cannot affect runtime behavior directly. Required CI passed. Because the touched file is itself an E2E test harness and an unresolved edge case exists, targeted shell-level argument-rewrite tests or the optional E2E would improve confidence but are not required by the E2E advisor.
  • E2E Advisor: ok

✅ What looks good

  • Patch is very small: one existing E2E shell script changed with 2 insertions and 1 deletion.
  • The changed file still exists on the target codebase, and recent history confirms this is active gateway-upgrade E2E code.
  • Required CI contexts passed for the specified head SHA.
  • The implementation uses Bash arrays for Docker arguments, avoiding shell-string concatenation for the new build arg.
  • The new fallback logs the BASE_IMAGE value it injects, preserving diagnostics for future E2E failures.

Review completeness

  • Review is based on trusted metadata and the provided diff; no commands, package-manager operations, or E2E scripts were executed.
  • The full file was not re-read beyond the provided diff, so line references are based on trusted review metadata and diff hunk locations.
  • No linked issues were present; acceptance coverage uses PR body clauses and E2E advisor comment clauses as available acceptance evidence.
  • The optional openshell-gateway-upgrade-e2e result for this exact head SHA was not provided.
  • Human maintainer review required: yes

cv pushed a commit that referenced this pull request May 22, 2026
## Summary
- address the unresolved CodeRabbit comment from PR #4047
- mark any explicit `BASE_IMAGE` Docker build arg as present, not just
the mutable `:latest` literal
- keep the #4047 fallback path for old installers that pass no
`BASE_IMAGE` at all

## Why
PR #4047 fixed the nightly gateway-upgrade fixture by appending a pinned
`BASE_IMAGE` when the old installer omitted one. CodeRabbit correctly
noted that `rewrote_base` was only set for the
`ghcr.io/nvidia/nemoclaw/sandbox-base:latest` literal, so a
caller-provided non-latest `BASE_IMAGE` could receive a second fallback
arg and be overridden by Docker last-arg-wins behavior.

## Test plan
- `bash -n test/e2e/test-openshell-gateway-upgrade.sh`
- `git diff --check`
- shell wrapper smoke: verified no-BASE adds pinned fallback, latest
rewrites to pinned fallback, explicit custom BASE_IMAGE does not add
pinned fallback
- `npm run build:cli`
- `npm run typecheck:cli`
- `npm run test -- test/onboard-openshell-version.test.ts`

Follow-up to PR #4047.

Resolves CodeRabbit thread:
#4047 (comment)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Tests**
* Improved Docker wrapper in test suite to correctly handle
pre-specified build arguments, preventing duplicate argument injection
and ensuring reliable test execution.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/NVIDIA/NemoClaw/pull/4048?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants