Skip to content

fix(start): drop slow-mode polling on late allowlisted scope upgrades#5387

Merged
jyaunches merged 55 commits into
mainfrom
fix/scope-upgrade-late-approval-race
Jun 30, 2026
Merged

fix(start): drop slow-mode polling on late allowlisted scope upgrades#5387
jyaunches merged 55 commits into
mainfrom
fix/scope-upgrade-late-approval-race

Conversation

@laitingsheng

@laitingsheng laitingsheng commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes the multi-sandbox parallel routing race where two sandboxes onboarded back-to-back each hit scope upgrade pending approval + pairing required on the gateway and both fall back to embedded mode. The in-sandbox auto-pair watcher kept polling at 30s slow-mode cadence after browser-pairing convergence; later openclaw tui / openclaw agent allowlisted scope upgrades waited up to one full SLOW_INTERVAL before approval, longer than the OpenClaw client's pending-approval tolerance. Lowers the slow-poll default to 5s and arms a bounded fast-reentry counter on the rising edge of each fresh allowlisted approval attempt. Phase 7 E2E covers the differing-provider concurrent-routing case end to end.

Related Issue

Fixes #5343

Changes

  • scripts/nemoclaw-start.sh:
    • Lower SLOW_INTERVAL default 30s → 5s.
    • Add sleep_for_next_poll(default, productive=True) and bounded FAST_REENTRY_* state — rising-edge bump per fresh allowlisted requestId, single-marker log, sticky requests do not re-arm. Error-path sleeps pass productive=False so gateway/JSON errors don't drain the bounded window.
    • Tighten gateway-watchdog integer guards: case [1-9]* glob (which accepted 12x/30abc) → =~ ^[1-9][0-9]*$ regex.
  • test/nemoclaw-start.test.ts: extract setupLateCliFixture; cover two concurrent late openclaw-cli upgrades + sticky-failure no-rearm assertion.
  • test/e2e/test-issue-4462-scope-upgrade-approval.sh:
    • Phase 6 strict-fails on missing slow-mode convergence; fast-reentry marker check is info-only (race-tolerant; unit test owns the strict invariant).
    • Phase 7 onboards a second sandbox via host-side Ollama (pinned v0.13.5 tarball, SHA256-verified, archive-layout-validated before privileged extract). Reads provider+model from host registry and effective baseUrl from /sandbox/.openclaw/openclaw.json inside each sandbox; fails closed when missing/non-inference.local. Two concurrent openclaw agent turns pinned to :18789 and :18790 with strict no-fallback / no-scope-upgrade / no-pairing-marker assertions. Default model qwen3:0.6b substitutes the GPU-only qwen3.5:9b; the result summary surfaces which lane ran.
    • All sandbox-command output appended to $APPROVAL_LOG / $AGENT_LOG passes through redact_text_for_log_or_marker; ${var:0:N} truncated excerpts in fail/info messages now use redacted_excerpt so secrets never reach uploaded artefacts. device_state_json() pipes through the device-state redactor.
    • Factor registry parsing into read_host_registry_provider_model + read_sandbox_openclaw_route helpers; extract_openclaw_upstream is now a thin merger.
    • Extract ollama_pinned_install_sha256_ok so a unit test can exercise both branches.
  • test/e2e/lib/redact-device-state.py (new): single source of truth for device-state secret redaction.
  • test/e2e/lib/redact-text.py (new): token-shape redactor for diagnostic excerpts. sk-… branch is left-bounded to avoid false positives on hyphenated text.
  • test/redact-device-state.test.ts (new): device-state redactor + sandbox-exec wrapper failure paths. E2E_DIR passed via env (not interpolated into the harness string) for CodeQL alert fix: improve gateway lifecycle recovery #953; failure-path assertions use stdout.trim().toBe(...).
  • test/redact-text.test.ts (new): focused vitest for redact-text.py plus assertions that the E2E script appends redacted output to every secret-bearing log.
  • test/ollama-pinned-install.test.ts (new): runtime negative test — version override without SHA256 fails closed.
  • test/e2e-script-workflow.test.ts: Phase 7 guardrails (Ollama sha256, CPU-lane substitute disclosure, provider-not-model identification for sandbox B).
  • src/lib/actions/inference-set.test.ts: track _nemoclaw_upstream annotation written by main to keep the no-op assertion truthful after rebase.
  • .github/workflows/nightly-e2e.yaml: lower approval-job NEMOCLAW_AUTO_PAIR_SLOW_INTERVAL_SECS 600 → 5 so CI exercises the new code path.
  • ci/test-file-size-budget.json: pin test/e2e-script-workflow.test.ts legacy budget after maintainer rebase grew the file.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: in-sandbox watcher cadence, Phase 7 secret redaction, and gateway-watchdog input validation are not user-facing knobs surfaced in docs/.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: @prekshivyas approved (fix(start): drop slow-mode polling on late allowlisted scope upgrades #5387 (review)) covering fast-reentry semantics, sticky-failure handling, and the 30→5 migration callout; later secret-redaction and effective-route hardening mirror the same boundary.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Verification

  • PR description includes the DCO sign-off declaration and every commit appears as Verified in GitHub
  • Git hooks passed during commit and push, or npx prek run --from-ref main --to-ref HEAD passes
  • Targeted tests pass for changed behavior
  • Full npm test passes (broad runtime changes only)
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Tinson Lai tinsonl@nvidia.com

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
@coderabbitai

coderabbitai Bot commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The PR renames nightly scope-upgrade jobs to CLI-scope variants, adds bounded fast-reentry polling in the auto-pair watcher, redacts captured device-state JSON through new helpers, and updates the CLI scope-upgrade E2E script and tests for the new flow.

Changes

CLI scope-upgrade approval flow and redaction support

Layer / File(s) Summary
Nightly workflow wiring
.github/workflows/nightly-e2e.yaml, test/e2e-script-workflow.test.ts
The nightly workflow dispatch list, reusable jobs, downstream needs references, and script contract tests move from the issue-4462 scope-upgrade job IDs to the CLI-scope job IDs and associated script checks.
Approval script setup
test/e2e/test-issue-4462-scope-upgrade-approval.sh
The approval script switches to CLI-scope environment names, redacts captured device-state JSON before summarization, and updates the approval subprocess and gateway-characterization plumbing to the new variable names.
Fast-reentry watcher polling
scripts/nemoclaw-start.sh, test/nemoclaw-start.test.ts
The auto-pair watcher starts unbuffered, tracks fast-reentry state, and routes poll and backoff sleeps through sleep_for_next_poll(); the watcher tests cover concurrent late-CLI approval behavior and the non-rearming failure case.
Device-state redaction pipeline
test/e2e/lib/redact-device-state.py, test/e2e/lib/redact-text.py, test/redact-device-state.test.ts
The Python redactors recursively scrub secret-shaped JSON and token-like text, and the Vitest suite covers nested redaction, invalid JSON, wrapper outcomes, text redaction, and the script wiring checks around both redaction paths.
Legacy-repro phases
test/e2e/test-issue-4462-scope-upgrade-approval.sh, test/redact-device-state.test.ts
The approval script renames the session IDs and success text, updates legacy-characterization wording, and adds Phase 6 slow-mode and Phase 7 two-sandbox gateway validation plus the matching test assertions.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related issues

Suggested labels

area: ci, area: e2e, refactor

Suggested reviewers

  • cv
  • sandl99
  • prekshivyas

Poem

A rabbit hopped through logs at night,
and tucked the secrets out of sight.
With fast-reentry, hop-hop, hop,
the watcher danced and wouldn’t stop.
🐇✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 3.85% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and accurately summarizes the slow-polling/fast-reentry fix for late allowlisted scope upgrades.
Linked Issues check ✅ Passed The PR adds the bounded fast-reentry fix and the multi-sandbox Phase 7 gateway-routing test required by #5343.
Out of Scope Changes check ✅ Passed The changes stay focused on the scope-upgrade race, gateway-routing coverage, and related redaction/test plumbing.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/scope-upgrade-late-approval-race

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in the branch is 96%. Coverage data for the branch is not yet available.

Show a code coverage summary of the most covered files.
File 17836ff +/-
nemoclaw/src/se...cret-scanner.ts 100%
nemoclaw/src/commands/slash.ts 100%
nemoclaw/src/li...bprocess-env.ts 100%
nemoclaw/src/bl...eprint/state.ts 98%
nemoclaw/src/onboard/config.ts 98%
nemoclaw/src/bl...int/snapshot.ts 97%
nemoclaw/src/bl...print/runner.ts 95%
nemoclaw/src/co...ration-state.ts 94%
nemoclaw/src/bl...ate-networks.ts 94%
nemoclaw/src/index.ts 94%

TypeScript / code-coverage/cli

The overall coverage in the branch is 68%. Coverage data for the branch is not yet available.

Show a code coverage summary of the most covered files.
File 17836ff +/-
src/lib/actions...all/run-plan.ts 80%
src/lib/state/o...oard-session.ts 79%
src/lib/actions...dbox/rebuild.ts 75%
src/lib/sandbox/config.ts 75%
src/lib/state/sandbox.ts 72%
src/lib/onboard/preflight.ts 69%
src/lib/actions...licy-channel.ts 60%
src/lib/onboard...er-gpu-patch.ts 59%
src/lib/policy/index.ts 53%
src/lib/onboard.ts 20%

Updated June 30, 2026 16:14 UTC
Code Coverage is in Public Preview. Learn more and provide us with your feedback.

@github-actions

github-actions Bot commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

E2E Advisor Recommendation

Required E2E: issue-4462-scope-upgrade-approval, device-auth-health, gateway-guard-recovery, sandbox-operations, cloud-onboard
Optional E2E: full-e2e, gateway-health-honest-e2e

Dispatch hint: issue-4462-scope-upgrade-approval,device-auth-health,gateway-guard-recovery,sandbox-operations,cloud-onboard

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

  • issue-4462-scope-upgrade-approval (high): Required because the changed auto-pair watcher is responsible for approving late OpenClaw CLI/agent scope upgrades without expanding to operator.admin. This live E2E is the closest existing coverage for pending scope-upgrade approval and final assistant traffic staying on the gateway path.
  • device-auth-health (medium): Required because the auto-pair changes directly affect device-auth pairing/approval behavior and dashboard/gateway health interpretation with device auth enabled.
  • gateway-guard-recovery (medium): Required because the PR changes gateway watchdog validation in the sandbox entrypoint, and a regression could leave a gateway process alive but unavailable instead of handing recovery back to the respawn loop.
  • sandbox-operations (high): Required because scripts/nemoclaw-start.sh is central to sandbox startup and long-running lifecycle behavior; sandbox operations E2E validates real create/start/stop/recover flows around the entrypoint.
  • cloud-onboard (high): Required because the changed startup entrypoint and auto-pair behavior can affect full hosted onboarding after the sandbox is created and the OpenClaw UI/agent pair with the gateway.

Optional E2E

  • full-e2e (high): Optional broad confidence check for the full OpenClaw assistant user flow. The required issue-4462 job is more targeted for this PR, but full-e2e would add coverage that normal agent-mediated use still works after the entrypoint changes.
  • gateway-health-honest-e2e (medium): Optional adjacent gateway-health regression check. The changed code is the in-sandbox serving watchdog rather than the host-side gateway health-honesty path, so this is useful but not the most direct merge blocker.

New E2E recommendations

  • gateway-serving-watchdog (high): Existing live jobs are adjacent, but there does not appear to be a dedicated E2E that forces an in-container OpenClaw gateway to keep its PID while dropping the HTTP listener, then proves the watchdog kills it and the respawn loop restores a working dashboard/agent path.
    • Suggested test: Add a live E2E that induces an alive-but-deaf gateway inside a real OpenClaw sandbox and asserts watchdog-triggered respawn plus successful post-recovery health and assistant command.
  • auto-pair-late-scope-upgrade-cadence (medium): The unit tests cover the bounded fast-reentry counter deterministically, and issue-4462 covers a real scope-upgrade path, but there is no clearly dedicated live E2E for multiple late allowlisted upgrades arriving after slow-mode convergence under the default 5s cadence.
    • Suggested test: Extend the scope-upgrade live coverage or add a new live E2E that pairs the browser first, waits for slow mode, then launches one or more late OpenClaw CLI/agent/TUI clients and asserts they are approved before falling back to embedded mode.

Dispatch hint

  • Workflow: .github/workflows/e2e.yaml
  • jobs input: issue-4462-scope-upgrade-approval,device-auth-health,gateway-guard-recovery,sandbox-operations,cloud-onboard

@github-actions

github-actions Bot commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Vitest E2E Scenario Recommendation

Required Vitest E2E scenarios: issue-4462-scope-upgrade-approval-vitest
Optional Vitest E2E scenarios: concurrent-gateway-ports-vitest, gateway-health-honest-vitest

Dispatch required Vitest E2E scenarios:

  • gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field jobs=issue-4462-scope-upgrade-approval-vitest

Workflow run

Full Vitest E2E advisor summary

Vitest E2E Scenario Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required Vitest E2E scenarios

  • issue-4462-scope-upgrade-approval-vitest: scripts/nemoclaw-start.sh changes the in-sandbox auto-pair watcher, scope-upgrade approval cadence, and approve retry/recovery behavior. The wired free-standing Vitest job exercises the real OpenClaw scope-upgrade approval path and verifies the final agent turn stays on the gateway path without admin-scope leakage.
    • Dispatch: gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field jobs=issue-4462-scope-upgrade-approval-vitest

Optional Vitest E2E scenarios

  • concurrent-gateway-ports-vitest: Adjacent coverage for the same startup/auto-pair changes under multiple coexisting sandboxes and gateway/dashboard ports; useful because the PR comments and legacy test changes emphasize two-sandbox late CLI scope-upgrade behavior, but the primary Vitest gate is the scope-upgrade job.
    • Dispatch: gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field jobs=concurrent-gateway-ports-vitest
  • gateway-health-honest-vitest: Adjacent gateway health coverage for startup/gateway failure reporting. The changed watchdog validation is primarily covered by non-scenario unit tests, but this live job can provide extra confidence that gateway health failures are still surfaced honestly.
    • Dispatch: gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field jobs=gateway-health-honest-vitest

Relevant changed files

  • scripts/nemoclaw-start.sh

@github-actions

github-actions Bot commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — Blocked

Merge posture: Do not merge until addressed
Primary next action: Fix PRA-4: Active tests still do not cover #5343's two-sandbox provider/model routing contract; then add or justify PRA-T1.
Open items: 1 required · 7 warnings · 0 suggestions · 8 test follow-ups
Since last review: 0 prior items resolved · 8 still apply · 0 new items found

Action checklist

  • PRA-4 Fix: Active tests still do not cover [All Platforms][Inference] Multi-sandbox parallel routing test fails: both sandboxes hit gateway scope-upgrade block and fall back to embedded mode #5343's two-sandbox provider/model routing contract in test/e2e/live/issue-4462-scope-upgrade-approval.test.ts:16
  • PRA-1 Resolve or justify: Source-of-truth review needed: Auto-pair slow-mode cadence and fast-reentry workaround
  • PRA-2 Resolve or justify: Source-of-truth review needed: E2E diagnostic redaction helpers
  • PRA-3 Resolve or justify: Source-of-truth review needed: Host registry reader for provider/model assertions
  • PRA-5 Resolve or justify: `NEMOCLAW_AUTO_PAIR_FAST_REENTRY_POLLS` still uses a seconds parser for an integer count in scripts/nemoclaw-start.sh:2301
  • PRA-6 Resolve or justify: Fast-reentry can remain at fast cadence through sustained list or JSON errors in scripts/nemoclaw-start.sh:2352
  • PRA-7 Resolve or justify: New E2E redaction and registry helpers are not wired into the active E2E surface in test/e2e/lib/redact-text.py:35
  • PRA-8 Resolve or justify: Device-state redactor misses common camelCase credential fields in test/e2e/lib/redact-device-state.py:19
  • PRA-T1 Add or justify test follow-up: Runtime validation
  • PRA-T2 Add or justify test follow-up: Runtime validation
  • PRA-T3 Add or justify test follow-up: Runtime validation
  • PRA-T4 Add or justify test follow-up: Runtime validation
  • PRA-T5 Add or justify test follow-up: Runtime validation
  • PRA-T6 Add or justify test follow-up: Acceptance clause
  • PRA-T7 Add or justify test follow-up: Acceptance clause
  • PRA-T8 Add or justify test follow-up: Acceptance clause

Findings index

ID Severity Category Location Required action
PRA-1 Resolve/justify architecture Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
PRA-2 Resolve/justify architecture Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
PRA-3 Resolve/justify architecture Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
PRA-4 Required acceptance test/e2e/live/issue-4462-scope-upgrade-approval.test.ts:16 Add or update an active #5343 acceptance lane that creates or targets two sandboxes concurrently, asserts sandbox-A routes through `inference.local` to `nvidia/nemotron-3-super-120b-a12b`, asserts sandbox-B routes through `inference.local` to `qwen3.5:9b` or a documented substitute with explicit model evidence, checks distinct gateway URLs or dashboard ports `:18789` and `:18790`, and fails on `scope upgrade pending approval`, `pairing required`, or `EMBEDDED FALLBACK`. If this PR intentionally fixes only watcher cadence, narrow the linked-issue claim so #5343 remains partial.
PRA-5 Resolve/justify correctness scripts/nemoclaw-start.sh:2301 Add a dedicated positive-integer parser for count knobs. Reject non-integers, non-finite values, empty values, negatives, and trailing junk back to the default. Explicitly document whether `0` disables fast-reentry or is invalid; do not reuse `_env_seconds` for poll counts.
PRA-6 Resolve/justify correctness scripts/nemoclaw-start.sh:2352 Bound unproductive fast-reentry sleeps too. For example, decrement on every fast-reentry sleep, track a separate wall-clock fast-reentry deadline, or cap consecutive error-path fast sleeps before returning to `SLOW_INTERVAL`. Keep enough retries to catch cascading upgrades, but make the bound enforceable under persistent errors.
PRA-7 Resolve/justify architecture test/e2e/lib/redact-text.py:35 Either wire the helpers into the active E2E implementation that emits the relevant diagnostics and validates #5343 routes, or remove them from this PR until that lane lands. If kept, update comments to reference existing files and add direct tests for `read-host-registry.py` success, unreadable/malformed registry, missing sandbox, and missing-argument behavior.
PRA-8 Resolve/justify security test/e2e/lib/redact-device-state.py:19 Extend the field-name matcher to cover camelCase and mixed-case credential names, and keep value-shape redaction as defense in depth. Add tests where camelCase fields contain arbitrary non-token-shaped secret values so the test cannot pass solely via value-shape matching.

🚨 Required before merge

Address these before merging unless a maintainer explicitly overrides the advisor with rationale.

PRA-4 Required — Active tests still do not cover #5343's two-sandbox provider/model routing contract

Review findings by urgency: 1 required fix, 7 items to resolve/justify, 0 in-scope improvements

⚠️ Resolve or justify before merge

Investigate these in the current review; either fix them, explain why they are not applicable, or document the accepted risk.

PRA-1 Resolve/justify — Source-of-truth review needed: Auto-pair slow-mode cadence and fast-reentry workaround

  • Location: not file-specific
  • Category: architecture
  • Problem: The advisor marked localized patch analysis as needs_followup.
  • Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
  • Recommended action: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Missing regression test: Unit-style tests cover late request IDs in one watcher process, but active two-sandbox [All Platforms][Inference] Multi-sandbox parallel routing test fails: both sandboxes hit gateway scope-upgrade block and fall back to embedded mode #5343 runtime coverage and sustained error-path bounding coverage are missing.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Evidence: The new source-of-truth comment is present near `SLOW_INTERVAL`, but the claimed [All Platforms][Inference] Multi-sandbox parallel routing test fails: both sandboxes hit gateway scope-upgrade block and fall back to embedded mode #5343 end-to-end coverage is not present in changed active files, and `productive=False` sleeps do not drain the fast-reentry counter.

PRA-2 Resolve/justify — Source-of-truth review needed: E2E diagnostic redaction helpers

  • Location: not file-specific
  • Category: architecture
  • Problem: The advisor marked localized patch analysis as needs_followup.
  • Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
  • Recommended action: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Missing regression test: Unit tests cover regex substitution, but no active E2E integration test proves failure output is redacted before artifact/log emission.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Evidence: Repository search found the helpers referenced only by their tests and self-docs, while the active OpenClaw CLI scope-upgrade approval deadlocks and forces openclaw agent into embedded fallback #4462 file still has raw failure `cat` paths.

PRA-3 Resolve/justify — Source-of-truth review needed: Host registry reader for provider/model assertions

  • Location: not file-specific
  • Category: architecture
  • Problem: The advisor marked localized patch analysis as needs_followup.
  • Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
  • Recommended action: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Missing regression test: No direct tests cover the documented success, malformed registry, unreadable registry, missing sandbox, or missing-argument exit behavior.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Evidence: `test/e2e/lib/read-host-registry.py` documents exit codes, but `sandbox_name = sys.argv[1]` crashes on missing arguments and grep found no active caller.

PRA-5 Resolve/justify — `NEMOCLAW_AUTO_PAIR_FAST_REENTRY_POLLS` still uses a seconds parser for an integer count

  • Location: scripts/nemoclaw-start.sh:2301
  • Category: correctness
  • Problem: `FAST_REENTRY_POLLS = int(_env_seconds('NEMOCLAW_AUTO_PAIR_FAST_REENTRY_POLLS', 5))` parses a poll count as a floating-point duration and then coerces it with `int(...)`. Fractional or non-finite values can truncate, disable, or crash the auto-pair watcher instead of falling back to a safe default.
  • Impact: A sandbox environment override such as `NEMOCLAW_AUTO_PAIR_FAST_REENTRY_POLLS=inf` can raise during watcher startup, and fractional values can silently alter the intended bounded retry count. Because the watcher mediates gateway scope-upgrade approvals, bad parsing can turn an availability fix into a startup-time denial of service or silently remove the intended fast-reentry behavior.
  • Recommended action: Add a dedicated positive-integer parser for count knobs. Reject non-integers, non-finite values, empty values, negatives, and trailing junk back to the default. Explicitly document whether `0` disables fast-reentry or is invalid; do not reuse `_env_seconds` for poll counts.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect `scripts/nemoclaw-start.sh` around the `FAST_REENTRY_POLLS` assignment and confirm it no longer calls `_env_seconds` or `int(float_value)` for this count.
  • Missing regression test: Add `auto-pair fast-reentry count env rejects inf nan fractions and trailing junk without crashing` covering `inf`, `nan`, `0.5`, `12x`, `0`, and a valid integer.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect `scripts/nemoclaw-start.sh` around the `FAST_REENTRY_POLLS` assignment and confirm it no longer calls `_env_seconds` or `int(float_value)` for this count.
  • Evidence: The diff leaves `FAST_REENTRY_POLLS = int(_env_seconds('NEMOCLAW_AUTO_PAIR_FAST_REENTRY_POLLS', 5))`, while the new watchdog integer guard demonstrates the safer regex approach for integer environment knobs.

PRA-6 Resolve/justify — Fast-reentry can remain at fast cadence through sustained list or JSON errors

  • Location: scripts/nemoclaw-start.sh:2352
  • Category: correctness
  • Problem: `sleep_for_next_poll(..., productive=False)` deliberately avoids decrementing `FAST_REENTRY_REMAINING` on `devices list` failures and JSON parse failures. After a fast-reentry bump, a persistent gateway/list/JSON error stream can therefore keep the watcher sleeping at `FAST_REENTRY_INTERVAL` until the overall deadline rather than for a bounded number of polls.
  • Impact: The PR describes the fast-reentry override as bounded, but sustained unproductive errors can keep every affected sandbox polling at the fast interval for hours. That can increase gateway pressure exactly when the gateway is unhealthy, undermining the load-safety rationale for slow mode and reviving a denial-of-service style failure mode.
  • Recommended action: Bound unproductive fast-reentry sleeps too. For example, decrement on every fast-reentry sleep, track a separate wall-clock fast-reentry deadline, or cap consecutive error-path fast sleeps before returning to `SLOW_INTERVAL`. Keep enough retries to catch cascading upgrades, but make the bound enforceable under persistent errors.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Read `sleep_for_next_poll` and the `rc != 0 or not out` / `json.loads` exception branches in `scripts/nemoclaw-start.sh`; confirm that the fast-reentry counter is consumed or otherwise bounded when those branches repeat.
  • Missing regression test: Add `auto-pair fast-reentry returns to slow cadence during sustained devices list failures or invalid JSON` with a fake `openclaw devices list --json` that bumps once and then returns errors/invalid JSON for more than `FAST_REENTRY_POLLS` iterations.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Read `sleep_for_next_poll` and the `rc != 0 or not out` / `json.loads` exception branches in `scripts/nemoclaw-start.sh`; confirm that the fast-reentry counter is consumed or otherwise bounded when those branches repeat.
  • Evidence: `sleep_for_next_poll` only decrements `FAST_REENTRY_REMAINING` when `productive` is true, and both list-failure branches call it with `productive=False`.

PRA-7 Resolve/justify — New E2E redaction and registry helpers are not wired into the active E2E surface

  • Location: test/e2e/lib/redact-text.py:35
  • Category: architecture
  • Problem: The PR adds `read-host-registry.py`, `redact-device-state.py`, and `redact-text.py`, but repository search only finds the new helpers referenced by their unit tests and self-documentation. The `redact-text.py` doc points to `test/e2e/test-issue-4462-scope-upgrade-approval.sh`, which is not present, while the active test is `test/e2e/live/issue-4462-scope-upgrade-approval.test.ts` and still has raw failure log `cat` paths.
  • Impact: The helper files create a false sense that [All Platforms][Inference] Multi-sandbox parallel routing test fails: both sandboxes hit gateway scope-upgrade block and fall back to embedded mode #5343/OpenClaw CLI scope-upgrade approval deadlocks and forces openclaw agent into embedded fallback #4462 diagnostics are redacted and provider/model assertions are centralized, while the active E2E path does not use them. If a failure emits gateway or agent logs, secrets can still reach uploaded artifacts, and `read-host-registry.py`'s documented fail-closed contract is untested and unused.
  • Recommended action: Either wire the helpers into the active E2E implementation that emits the relevant diagnostics and validates [All Platforms][Inference] Multi-sandbox parallel routing test fails: both sandboxes hit gateway scope-upgrade block and fall back to embedded mode #5343 routes, or remove them from this PR until that lane lands. If kept, update comments to reference existing files and add direct tests for `read-host-registry.py` success, unreadable/malformed registry, missing sandbox, and missing-argument behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Run a read-only search for `read-host-registry.py`, `redact-device-state.py`, `redact-text.py`, `redact_text_for_log_or_marker`, and `test-issue-4462-scope-upgrade-approval.sh`; confirm whether anything besides unit tests and helper docs call the helpers.
  • Missing regression test: Add `active E2E diagnostics pass gateway agent and device-state failure output through redactors before artifact emission` and, if `read-host-registry.py` remains, `read-host-registry emits provider/model and documented exit codes for malformed registry missing sandbox and missing argument`.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Run a read-only search for `read-host-registry.py`, `redact-device-state.py`, `redact-text.py`, `redact_text_for_log_or_marker`, and `test-issue-4462-scope-upgrade-approval.sh`; confirm whether anything besides unit tests and helper docs call the helpers.
  • Evidence: Search found `test/e2e/lib/redact-text.py` self-doc references to `redact_text_for_log_or_marker` and the absent `test/e2e/test-issue-4462-scope-upgrade-approval.sh`; it found `redact-text.py` and `redact-device-state.py` only in their new unit tests. `test/e2e/live/issue-4462-scope-upgrade-approval.test.ts` still contains raw `cat /tmp/issue4462-*-agent.log >&2` failure output.

PRA-8 Resolve/justify — Device-state redactor misses common camelCase credential fields

  • Location: test/e2e/lib/redact-device-state.py:19
  • Category: security
  • Problem: `SECRET_FIELD_RE` matches delimited names such as `api_key`, `apikey`, `access_key`, and `headers`, but not common camelCase or mixed-case JSON keys such as `apiKey`, `accessToken`, `refreshToken`, `clientSecret`, `githubToken`, `hfToken`, or `awsSecretAccessKey` when their values are not already token-shaped.
  • Impact: Device-state diagnostics can contain arbitrary credential values under camelCase keys. Those values would pass through the new redactor and could be written to E2E logs or artifacts once the helper is wired into active diagnostics.
  • Recommended action: Extend the field-name matcher to cover camelCase and mixed-case credential names, and keep value-shape redaction as defense in depth. Add tests where camelCase fields contain arbitrary non-token-shaped secret values so the test cannot pass solely via value-shape matching.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect `SECRET_FIELD_RE` in `test/e2e/lib/redact-device-state.py` and confirm it matches camelCase keys directly, not only token-shaped values under those keys.
  • Missing regression test: Add `device-state redactor redacts camelCase and mixed-case secret field names with arbitrary non-token values` covering `apiKey`, `accessToken`, `refreshToken`, `clientSecret`, `githubToken`, `hfToken`, and `awsSecretAccessKey` with values such as `not-a-known-token-shape`.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect `SECRET_FIELD_RE` in `test/e2e/lib/redact-device-state.py` and confirm it matches camelCase keys directly, not only token-shaped values under those keys.
  • Evidence: `test/redact-device-state.test.ts` includes camelCase fields like `githubToken` and `hfToken`, but the sample values are known token shapes. The regex in `test/e2e/lib/redact-device-state.py:19` does not include camelCase boundaries for those field names.

💡 In-scope improvements

These are lower-risk, not throwaway. Prefer fixing them in this PR when they are local to changed code; defer only with rationale or a linked follow-up.

  • None.
Simplification opportunities: 1 possible cut, net -211 lines possible

These are safe simplification checks only. Do not remove validation, security controls, data-loss prevention, or required tests.

  • PRA-7 yagni (test/e2e/lib/redact-text.py:35): Remove the unused `test/e2e/lib/read-host-registry.py`, `test/e2e/lib/redact-device-state.py`, and `test/e2e/lib/redact-text.py` helper layer if the active [All Platforms][Inference] Multi-sandbox parallel routing test fails: both sandboxes hit gateway scope-upgrade block and fall back to embedded mode #5343 E2E lane is not included in this PR.
    • Replacement: Keep the focused unit tests only when the active E2E caller is added; otherwise land the helpers together with the caller that proves they are needed.
    • Net: -211 lines
    • Safety boundary: Do not remove or weaken any redaction that is actually wired into active artifact/log emission; if helpers are wired in this PR, keep fail-closed redaction and add the missing integration tests instead of deleting them.
Test follow-ups to resolve or justify

If these cover changed behavior, prefer adding them in this PR; otherwise state why existing coverage is enough or link the follow-up.

Since last review details

Current findings, using the urgency labels above:

PRA-1 Resolve/justify — Source-of-truth review needed: Auto-pair slow-mode cadence and fast-reentry workaround

  • Location: not file-specific
  • Category: architecture
  • Problem: The advisor marked localized patch analysis as needs_followup.
  • Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
  • Recommended action: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Missing regression test: Unit-style tests cover late request IDs in one watcher process, but active two-sandbox [All Platforms][Inference] Multi-sandbox parallel routing test fails: both sandboxes hit gateway scope-upgrade block and fall back to embedded mode #5343 runtime coverage and sustained error-path bounding coverage are missing.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Evidence: The new source-of-truth comment is present near `SLOW_INTERVAL`, but the claimed [All Platforms][Inference] Multi-sandbox parallel routing test fails: both sandboxes hit gateway scope-upgrade block and fall back to embedded mode #5343 end-to-end coverage is not present in changed active files, and `productive=False` sleeps do not drain the fast-reentry counter.

PRA-2 Resolve/justify — Source-of-truth review needed: E2E diagnostic redaction helpers

  • Location: not file-specific
  • Category: architecture
  • Problem: The advisor marked localized patch analysis as needs_followup.
  • Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
  • Recommended action: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Missing regression test: Unit tests cover regex substitution, but no active E2E integration test proves failure output is redacted before artifact/log emission.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Evidence: Repository search found the helpers referenced only by their tests and self-docs, while the active OpenClaw CLI scope-upgrade approval deadlocks and forces openclaw agent into embedded fallback #4462 file still has raw failure `cat` paths.

PRA-3 Resolve/justify — Source-of-truth review needed: Host registry reader for provider/model assertions

  • Location: not file-specific
  • Category: architecture
  • Problem: The advisor marked localized patch analysis as needs_followup.
  • Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
  • Recommended action: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Missing regression test: No direct tests cover the documented success, malformed registry, unreadable registry, missing sandbox, or missing-argument exit behavior.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Evidence: `test/e2e/lib/read-host-registry.py` documents exit codes, but `sandbox_name = sys.argv[1]` crashes on missing arguments and grep found no active caller.

PRA-4 Required — Active tests still do not cover #5343's two-sandbox provider/model routing contract

PRA-5 Resolve/justify — `NEMOCLAW_AUTO_PAIR_FAST_REENTRY_POLLS` still uses a seconds parser for an integer count

  • Location: scripts/nemoclaw-start.sh:2301
  • Category: correctness
  • Problem: `FAST_REENTRY_POLLS = int(_env_seconds('NEMOCLAW_AUTO_PAIR_FAST_REENTRY_POLLS', 5))` parses a poll count as a floating-point duration and then coerces it with `int(...)`. Fractional or non-finite values can truncate, disable, or crash the auto-pair watcher instead of falling back to a safe default.
  • Impact: A sandbox environment override such as `NEMOCLAW_AUTO_PAIR_FAST_REENTRY_POLLS=inf` can raise during watcher startup, and fractional values can silently alter the intended bounded retry count. Because the watcher mediates gateway scope-upgrade approvals, bad parsing can turn an availability fix into a startup-time denial of service or silently remove the intended fast-reentry behavior.
  • Recommended action: Add a dedicated positive-integer parser for count knobs. Reject non-integers, non-finite values, empty values, negatives, and trailing junk back to the default. Explicitly document whether `0` disables fast-reentry or is invalid; do not reuse `_env_seconds` for poll counts.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect `scripts/nemoclaw-start.sh` around the `FAST_REENTRY_POLLS` assignment and confirm it no longer calls `_env_seconds` or `int(float_value)` for this count.
  • Missing regression test: Add `auto-pair fast-reentry count env rejects inf nan fractions and trailing junk without crashing` covering `inf`, `nan`, `0.5`, `12x`, `0`, and a valid integer.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect `scripts/nemoclaw-start.sh` around the `FAST_REENTRY_POLLS` assignment and confirm it no longer calls `_env_seconds` or `int(float_value)` for this count.
  • Evidence: The diff leaves `FAST_REENTRY_POLLS = int(_env_seconds('NEMOCLAW_AUTO_PAIR_FAST_REENTRY_POLLS', 5))`, while the new watchdog integer guard demonstrates the safer regex approach for integer environment knobs.

PRA-6 Resolve/justify — Fast-reentry can remain at fast cadence through sustained list or JSON errors

  • Location: scripts/nemoclaw-start.sh:2352
  • Category: correctness
  • Problem: `sleep_for_next_poll(..., productive=False)` deliberately avoids decrementing `FAST_REENTRY_REMAINING` on `devices list` failures and JSON parse failures. After a fast-reentry bump, a persistent gateway/list/JSON error stream can therefore keep the watcher sleeping at `FAST_REENTRY_INTERVAL` until the overall deadline rather than for a bounded number of polls.
  • Impact: The PR describes the fast-reentry override as bounded, but sustained unproductive errors can keep every affected sandbox polling at the fast interval for hours. That can increase gateway pressure exactly when the gateway is unhealthy, undermining the load-safety rationale for slow mode and reviving a denial-of-service style failure mode.
  • Recommended action: Bound unproductive fast-reentry sleeps too. For example, decrement on every fast-reentry sleep, track a separate wall-clock fast-reentry deadline, or cap consecutive error-path fast sleeps before returning to `SLOW_INTERVAL`. Keep enough retries to catch cascading upgrades, but make the bound enforceable under persistent errors.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Read `sleep_for_next_poll` and the `rc != 0 or not out` / `json.loads` exception branches in `scripts/nemoclaw-start.sh`; confirm that the fast-reentry counter is consumed or otherwise bounded when those branches repeat.
  • Missing regression test: Add `auto-pair fast-reentry returns to slow cadence during sustained devices list failures or invalid JSON` with a fake `openclaw devices list --json` that bumps once and then returns errors/invalid JSON for more than `FAST_REENTRY_POLLS` iterations.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Read `sleep_for_next_poll` and the `rc != 0 or not out` / `json.loads` exception branches in `scripts/nemoclaw-start.sh`; confirm that the fast-reentry counter is consumed or otherwise bounded when those branches repeat.
  • Evidence: `sleep_for_next_poll` only decrements `FAST_REENTRY_REMAINING` when `productive` is true, and both list-failure branches call it with `productive=False`.

PRA-7 Resolve/justify — New E2E redaction and registry helpers are not wired into the active E2E surface

  • Location: test/e2e/lib/redact-text.py:35
  • Category: architecture
  • Problem: The PR adds `read-host-registry.py`, `redact-device-state.py`, and `redact-text.py`, but repository search only finds the new helpers referenced by their unit tests and self-documentation. The `redact-text.py` doc points to `test/e2e/test-issue-4462-scope-upgrade-approval.sh`, which is not present, while the active test is `test/e2e/live/issue-4462-scope-upgrade-approval.test.ts` and still has raw failure log `cat` paths.
  • Impact: The helper files create a false sense that [All Platforms][Inference] Multi-sandbox parallel routing test fails: both sandboxes hit gateway scope-upgrade block and fall back to embedded mode #5343/OpenClaw CLI scope-upgrade approval deadlocks and forces openclaw agent into embedded fallback #4462 diagnostics are redacted and provider/model assertions are centralized, while the active E2E path does not use them. If a failure emits gateway or agent logs, secrets can still reach uploaded artifacts, and `read-host-registry.py`'s documented fail-closed contract is untested and unused.
  • Recommended action: Either wire the helpers into the active E2E implementation that emits the relevant diagnostics and validates [All Platforms][Inference] Multi-sandbox parallel routing test fails: both sandboxes hit gateway scope-upgrade block and fall back to embedded mode #5343 routes, or remove them from this PR until that lane lands. If kept, update comments to reference existing files and add direct tests for `read-host-registry.py` success, unreadable/malformed registry, missing sandbox, and missing-argument behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Run a read-only search for `read-host-registry.py`, `redact-device-state.py`, `redact-text.py`, `redact_text_for_log_or_marker`, and `test-issue-4462-scope-upgrade-approval.sh`; confirm whether anything besides unit tests and helper docs call the helpers.
  • Missing regression test: Add `active E2E diagnostics pass gateway agent and device-state failure output through redactors before artifact emission` and, if `read-host-registry.py` remains, `read-host-registry emits provider/model and documented exit codes for malformed registry missing sandbox and missing argument`.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Run a read-only search for `read-host-registry.py`, `redact-device-state.py`, `redact-text.py`, `redact_text_for_log_or_marker`, and `test-issue-4462-scope-upgrade-approval.sh`; confirm whether anything besides unit tests and helper docs call the helpers.
  • Evidence: Search found `test/e2e/lib/redact-text.py` self-doc references to `redact_text_for_log_or_marker` and the absent `test/e2e/test-issue-4462-scope-upgrade-approval.sh`; it found `redact-text.py` and `redact-device-state.py` only in their new unit tests. `test/e2e/live/issue-4462-scope-upgrade-approval.test.ts` still contains raw `cat /tmp/issue4462-*-agent.log >&2` failure output.

PRA-8 Resolve/justify — Device-state redactor misses common camelCase credential fields

  • Location: test/e2e/lib/redact-device-state.py:19
  • Category: security
  • Problem: `SECRET_FIELD_RE` matches delimited names such as `api_key`, `apikey`, `access_key`, and `headers`, but not common camelCase or mixed-case JSON keys such as `apiKey`, `accessToken`, `refreshToken`, `clientSecret`, `githubToken`, `hfToken`, or `awsSecretAccessKey` when their values are not already token-shaped.
  • Impact: Device-state diagnostics can contain arbitrary credential values under camelCase keys. Those values would pass through the new redactor and could be written to E2E logs or artifacts once the helper is wired into active diagnostics.
  • Recommended action: Extend the field-name matcher to cover camelCase and mixed-case credential names, and keep value-shape redaction as defense in depth. Add tests where camelCase fields contain arbitrary non-token-shaped secret values so the test cannot pass solely via value-shape matching.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect `SECRET_FIELD_RE` in `test/e2e/lib/redact-device-state.py` and confirm it matches camelCase keys directly, not only token-shaped values under those keys.
  • Missing regression test: Add `device-state redactor redacts camelCase and mixed-case secret field names with arbitrary non-token values` covering `apiKey`, `accessToken`, `refreshToken`, `clientSecret`, `githubToken`, `hfToken`, and `awsSecretAccessKey` with values such as `not-a-known-token-shape`.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect `SECRET_FIELD_RE` in `test/e2e/lib/redact-device-state.py` and confirm it matches camelCase keys directly, not only token-shaped values under those keys.
  • Evidence: `test/redact-device-state.test.ts` includes camelCase fields like `githubToken` and `hfToken`, but the sample values are known token shapes. The regex in `test/e2e/lib/redact-device-state.py:19` does not include camelCase boundaries for those field names.

Workflow run details

This is an automated, non-binding review; it still expects maintainers and agents to respond to each required or warning item. Treat suggestions as current-PR improvements when they touch changed code; defer only with maintainer rationale or a linked follow-up. A human maintainer must make the final merge decision.

@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ❌ Some jobs failed

Run: 27463690336
Target ref: 04069f70a3743c2892e409295d79c3e029d1f638
Workflow ref: main
Requested jobs: issue-4462-scope-upgrade-approval-e2e,issue-4462-gateway-pinned-approval-characterization-e2e
Summary: 0 passed, 2 failed, 0 cancelled, 0 skipped

Job Result
issue-4462-gateway-pinned-approval-characterization-e2e ❌ failure
issue-4462-scope-upgrade-approval-e2e ❌ failure

Failed jobs: issue-4462-gateway-pinned-approval-characterization-e2e, issue-4462-scope-upgrade-approval-e2e. Check run artifacts for logs.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/nemoclaw-start.sh`:
- Around line 2050-2061: The fast-reentry counter is being reinitialized every
loop when attempted_approval is true, allowing a single failing pending request
to keep fast mode indefinitely; change the logic in the watcher loop so
FAST_REENTRY_REMAINING is only reset once per approval lifecycle (e.g., on the
rising edge of attempted_approval or when FAST_REENTRY_REMAINING == 0) instead
of every loop iteration: add a small persistent flag like
prev_attempted_approval or check FAST_REENTRY_REMAINING to detect transitions
and only set FAST_REENTRY_REMAINING = FAST_REENTRY_POLLS and print the log when
the transition occurs (leave APPROVED and SLOW_MODE usage as-is).

In `@test/nemoclaw-start.test.ts`:
- Around line 1746-1859: The test added ("drops slow-mode polling back to fast
cadence when a late scope upgrade arrives") exceeds the file line budget;
extract the duplicated late-cli fixture and setup into a small reusable helper
(e.g. create a function like createLateCliFixture that returns { tmpDir,
fakeOpenclaw, stateFile, approveLog } and sets up the script and permissions)
and call that helper from this test (and the adjacent similar test) instead of
inlining the long bash string and env setup, or alternatively fold the
fast-reentry assertions into the existing late-upgrade test to avoid duplicating
the entire fixture; locate references to fakeOpenclaw, stateFile, approveLog,
and buildAutoPairScript() to update call sites to use the helper and remove the
duplicated lines from the test body.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 5b40365e-a235-4bb4-bb63-2de367836ecb

📥 Commits

Reviewing files that changed from the base of the PR and between 39fd60a and 04069f7.

📒 Files selected for processing (5)
  • .github/workflows/nightly-e2e.yaml
  • scripts/nemoclaw-start.sh
  • test/e2e-script-workflow.test.ts
  • test/e2e/test-cli-scope-upgrade-approval.sh
  • test/nemoclaw-start.test.ts

Comment thread scripts/nemoclaw-start.sh Outdated
Comment thread test/nemoclaw-start.test.ts Outdated
@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ❌ Some jobs failed

Run: 27463736935
Target ref: 1ff7eb0c5a32f8b430d88fc5685923744caa1dc8
Workflow ref: main
Requested jobs: issue-4462-scope-upgrade-approval-e2e,issue-4462-gateway-pinned-approval-characterization-e2e
Summary: 0 passed, 2 failed, 0 cancelled, 0 skipped

Job Result
issue-4462-gateway-pinned-approval-characterization-e2e ❌ failure
issue-4462-scope-upgrade-approval-e2e ❌ failure

Failed jobs: issue-4462-gateway-pinned-approval-characterization-e2e, issue-4462-scope-upgrade-approval-e2e. Check run artifacts for logs.

@laitingsheng laitingsheng added integration: openclaw OpenClaw integration behavior bug-fix PR fixes a bug or regression labels Jun 13, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/e2e/test-cli-scope-upgrade-approval.sh`:
- Around line 1082-1086: The grep check for '[auto-pair] fast-reentry bumped'
using auto_pair_log_snapshot should be made informational: in the else branch
replace the fail call with a non-fatal log/warn (or call a function like warn or
echo) so the script doesn't hard-fail when the watcher didn't log the marker;
keep the pass branch unchanged. Update the block that currently uses grep -F ...
and calls fail to instead emit a warning/message while allowing the test to
continue.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 5a348f44-52fd-43f1-a690-23bae8b5c178

📥 Commits

Reviewing files that changed from the base of the PR and between 04069f7 and 2b040db.

📒 Files selected for processing (4)
  • .github/workflows/nightly-e2e.yaml
  • scripts/nemoclaw-start.sh
  • test/e2e/test-cli-scope-upgrade-approval.sh
  • test/nemoclaw-start.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • test/nemoclaw-start.test.ts

Comment thread test/e2e/test-issue-4462-scope-upgrade-approval.sh Outdated
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ❌ Some jobs failed

Run: 27464785119
Target ref: fix/scope-upgrade-late-approval-race
Requested jobs: cli-scope-upgrade-approval-e2e,cli-scope-upgrade-legacy-repro-e2e
Summary: 0 passed, 2 failed, 0 cancelled, 0 skipped

Job Result
cli-scope-upgrade-approval-e2e ❌ failure
cli-scope-upgrade-legacy-repro-e2e ❌ failure

Failed jobs: cli-scope-upgrade-approval-e2e, cli-scope-upgrade-legacy-repro-e2e. Check run artifacts for logs.

@laitingsheng laitingsheng added v0.0.65 Release target and removed v0.0.65 Release target labels Jun 13, 2026
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ❌ Some jobs failed

Run: 27465290873
Target ref: fix/scope-upgrade-late-approval-race
Requested jobs: cli-scope-upgrade-approval-e2e,cli-scope-upgrade-legacy-repro-e2e
Summary: 0 passed, 2 failed, 0 cancelled, 0 skipped

Job Result
cli-scope-upgrade-approval-e2e ❌ failure
cli-scope-upgrade-legacy-repro-e2e ❌ failure

Failed jobs: cli-scope-upgrade-approval-e2e, cli-scope-upgrade-legacy-repro-e2e. Check run artifacts for logs.

@laitingsheng laitingsheng added v0.0.65 Release target and removed v0.0.65 Release target labels Jun 13, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/nemoclaw-start.test.ts`:
- Around line 2248-2250: The test's assertion currently only matches
"fast-reentry bumped polls=3 approved=0 mode=fast" and can miss alternate bump
lines; update the marker and expectation used in test variable markerRe (and the
subsequent expect using run.stdout.match(...).length) to match any "fast-reentry
bumped" line (ignore the particular field values) and assert the total number of
such occurrences is exactly 1 so any extra bump log (e.g., with approved=1) will
fail the test.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 97da2f1c-484a-4ce2-a416-670cedbb0123

📥 Commits

Reviewing files that changed from the base of the PR and between bba211b and cae6c20.

📒 Files selected for processing (1)
  • test/nemoclaw-start.test.ts

Comment thread test/nemoclaw-start.test.ts Outdated
…ng-edge assertion

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
@laitingsheng laitingsheng added v0.0.65 Release target and removed v0.0.65 Release target labels Jun 13, 2026
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ❌ Some jobs failed

Run: 28251337754
Target ref: fix/scope-upgrade-late-approval-race
Requested jobs: cli-scope-upgrade-approval-e2e,device-auth-health-e2e,cloud-onboard-e2e
Summary: 2 passed, 1 failed, 0 cancelled, 0 skipped

Job Result
cli-scope-upgrade-approval-e2e ❌ failure
cloud-onboard-e2e ✅ success
device-auth-health-e2e ✅ success

Failed jobs: cli-scope-upgrade-approval-e2e. Check run artifacts for logs.

@github-actions

github-actions Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor (Nemotron Ultra) — Changes requested

Merge posture: Do not merge yet
Primary next action: Fix PRA-1: SLOW_INTERVAL default changed 30s→5s without CHANGELOG entry; then add or justify PRA-T1.
Open items: 3 required · 6 warnings · 2 suggestions · 8 test follow-ups
Since last review: 1 prior item resolved · 6 still apply · 5 new items found

Action checklist

  • PRA-1 Fix: SLOW_INTERVAL default changed 30s→5s without CHANGELOG entry in scripts/nemoclaw-start.sh:2274
  • PRA-2 Fix: Ollama installer full sudo tar path lacks runtime E2E test in test/ollama-pinned-install.test.ts:1
  • PRA-3 Fix: Documentation references non-existent shell script test/e2e/test-issue-4462-scope-upgrade-approval.sh in test/e2e/lib/redact-text.py:37
  • PRA-4 Resolve or justify: redact-text.py lacks stdin read failure test in test/redact-text.test.ts:1
  • PRA-5 Resolve or justify: Auto-pair fast-reentry lacks stress test for many concurrent sandboxes in scripts/nemoclaw-start.sh:2300
  • PRA-6 Resolve or justify: Phase 7 differing-providers test coupled to scope-upgrade script and CPU-lane only in test/e2e/live/issue-4462-scope-upgrade-approval.test.ts:200
  • PRA-7 Resolve or justify: _env_seconds() float/whitespace parsing vs watchdog strict regex inconsistency in scripts/nemoclaw-start.sh:2237
  • PRA-8 Resolve or justify: read-host-registry.py lacks path traversal validation on sandbox_name in test/e2e/lib/read-host-registry.py:1
  • PRA-9 Resolve or justify: test/nemoclaw-start.test.ts monolith growth (~2500 lines) in test/nemoclaw-start.test.ts:1
  • PRA-T1 Add or justify test follow-up: Runtime validation
  • PRA-T2 Add or justify test follow-up: Runtime validation
  • PRA-T3 Add or justify test follow-up: Runtime validation
  • PRA-T4 Add or justify test follow-up: Runtime validation
  • PRA-T5 Add or justify test follow-up: Runtime validation
  • PRA-T6 Add or justify test follow-up: Ollama installer full sudo tar path lacks runtime E2E test
  • PRA-T7 Add or justify test follow-up: Documentation references non-existent shell script test/e2e/test-issue-4462-scope-upgrade-approval.sh
  • PRA-T8 Add or justify test follow-up: redact-text.py lacks stdin read failure test
  • PRA-10 In-scope improvement: Scope-upgrade shell script embedded as template string in test/e2e/live/issue-4462-scope-upgrade-approval.test.ts:1
  • PRA-11 In-scope improvement: inference-set.test.ts monolith growth (+5 lines) in src/lib/actions/inference-set.test.ts:217

Findings index

ID Severity Category Location Required action
PRA-1 Required docs scripts/nemoclaw-start.sh:2274 Add a CHANGELOG.md entry documenting the SLOW_INTERVAL default change, the 6x polling increase, and the NEMOCLAW_AUTO_PAIR_SLOW_INTERVAL_SECS=30 opt-out. Consider whether 5s is appropriate for all deployment environments or if a more conservative default with opt-in to faster polling is safer.
PRA-2 Required tests test/ollama-pinned-install.test.ts:1 Add an integration-style test that sources the shell install block, sets OLLAMA_PINNED_VERSION override with empty OLLAMA_PINNED_SHA256, and asserts the script exits before sudo tar with OLLAMA_PIN_REQUIRES_SHA256 message. Or explicitly justify that unit test coverage of the guard function is sufficient.
PRA-3 Required tests test/e2e/lib/redact-text.py:37 Either create the standalone shell script with the referenced functions, update the documentation to reference the E2E test's embedded scopeUpgradeScript() function, or add a follow-up task to extract the shell helpers.
PRA-4 Resolve/justify tests test/redact-text.test.ts:1 Add a test simulating stdin read failure (e.g., spawnSync with closed stdin or a mocked failure) or justify why this is infeasible/low-risk.
PRA-5 Resolve/justify architecture scripts/nemoclaw-start.sh:2300 Add a focused vitest that spawns multiple fake openclaw processes simulating many sibling sandboxes with concurrent late scope upgrades, verifying the fast-reentry counter bounds and garbage collection behave correctly under contention.
PRA-6 Resolve/justify tests test/e2e/live/issue-4462-scope-upgrade-approval.test.ts:200 Extract Phase 7 into a dedicated E2E script/job for independent execution and GPU-lane validation, or document why coupling is intentional.
PRA-7 Resolve/justify correctness scripts/nemoclaw-start.sh:2237 Document why float parsing is intentional for auto-pair env vars (sub-second intervals), or add a shared positive-int/positive-float validator with consistent behavior. If sub-second is not needed for auto-pair, align with regex validation.
PRA-8 Resolve/justify security test/e2e/lib/read-host-registry.py:1 Add validation that sandbox_name contains only safe characters (alphanumeric, hyphen, underscore) and does not contain path traversal sequences.
PRA-9 Resolve/justify tests test/nemoclaw-start.test.ts:1 Consider extracting auto-pair watcher tests to test/nemoclaw-start-auto-pair.test.ts to maintain test file size budget and focus.
PRA-10 Improvement scope test/e2e/live/issue-4462-scope-upgrade-approval.test.ts:1 Extract the scopeUpgradeScript() template to test/e2e/test-cli-scope-upgrade-approval.sh and update all references (workflow, unit tests, comments). Can be done in follow-up PR.
PRA-11 Improvement correctness src/lib/actions/inference-set.test.ts:217 Defer extraction to follow-up as previously agreed. No action needed in this PR.

🚨 Required before merge

Address these before merging unless a maintainer explicitly overrides the advisor with rationale.

PRA-1 Required — SLOW_INTERVAL default changed 30s→5s without CHANGELOG entry

  • Location: scripts/nemoclaw-start.sh:2274
  • Category: docs
  • Problem: The auto-pair watcher's slow-mode polling interval default was reduced from 30 seconds to 5 seconds (6x increase in steady-state polling frequency). The SOURCE_OF_TRUTH_REVIEW comment documents this well, but no CHANGELOG or release notes entry exists in the diff. Operators relying on the old 30s cadence for gateway load management will see 6x more `openclaw devices list --json` calls per sandbox without warning.
  • Impact: Operators on load-sensitive gateways or large multi-sandbox deployments experience unexpected 6x increase in connect-handler load. The env var NEMOCLAW_AUTO_PAIR_SLOW_INTERVAL_SECS=30 restores old behavior but requires proactive configuration.
  • Required action: Add a CHANGELOG.md entry documenting the SLOW_INTERVAL default change, the 6x polling increase, and the NEMOCLAW_AUTO_PAIR_SLOW_INTERVAL_SECS=30 opt-out. Consider whether 5s is appropriate for all deployment environments or if a more conservative default with opt-in to faster polling is safer.
  • Expected follow-up: Fix before merge or get explicit maintainer override.
  • Verification: Check CHANGELOG.md or release notes for mention of NEMOCLAW_AUTO_PAIR_SLOW_INTERVAL_SECS default change.
  • Missing regression test: Release notes documentation (non-code)
  • Done when: The required change is committed and verification passes: Check CHANGELOG.md or release notes for mention of NEMOCLAW_AUTO_PAIR_SLOW_INTERVAL_SECS default change.
  • Evidence: scripts/nemoclaw-start.sh:2274 sets SLOW_INTERVAL = _env_seconds('NEMOCLAW_AUTO_PAIR_SLOW_INTERVAL_SECS', 5) with SOURCE_OF_TRUTH_REVIEW comment documenting the change but no CHANGELOG entry in diff.

PRA-2 Required — Ollama installer full sudo tar path lacks runtime E2E test

  • Location: test/ollama-pinned-install.test.ts:1
  • Category: tests
  • Problem: The Ollama installer three-guard chain (SHA256 guard + computed SHA match + layout validator) is covered by unit tests, but the full install flow reaching 'sudo tar -xzf' is not exercised by any runtime test. A regression where the guard is bypassed before privileged extract would not be caught.
  • Impact: An attacker or misconfiguration overriding OLLAMA_PINNED_VERSION without providing a matching SHA256 could reach the privileged tar extraction step, potentially allowing a malicious tarball to execute arbitrary code via symlink/hardlink/FIFO escapes before layout validation runs.
  • Required action: Add an integration-style test that sources the shell install block, sets OLLAMA_PINNED_VERSION override with empty OLLAMA_PINNED_SHA256, and asserts the script exits before sudo tar with OLLAMA_PIN_REQUIRES_SHA256 message. Or explicitly justify that unit test coverage of the guard function is sufficient.
  • Expected follow-up: Fix before merge or get explicit maintainer override.
  • Verification: Search for test/ollama-pinned-install.test.ts — file does not exist in the repo.
  • Missing regression test: E2E test running Ollama install code path with version override + no SHA256, verifying guard triggers before sudo tar
  • Done when: The required change is committed and verification passes: Search for test/ollama-pinned-install.test.ts — file does not exist in the repo.
  • Evidence: Previous review PRA-8 evidence: test/ollama-pinned-install.test.ts only tests ollama_pinned_install_sha256_ok guard function and validate_ollama_tarball_layout validator in isolation; no test runs the full install block.

PRA-3 Required — Documentation references non-existent shell script test/e2e/test-issue-4462-scope-upgrade-approval.sh

  • Location: test/e2e/lib/redact-text.py:37
  • Category: tests
  • Problem: The redact-text.py docstring mentions `redact_text_for_log_or_marker` function in test/e2e/test-issue-4462-scope-upgrade-approval.sh. This file does not exist in the repository. While unit tests don't depend on it, the documentation implies a runtime dependency for fail-closed markers.
  • Impact: Confusing documentation; if callers expect this helper to exist, they may encounter missing function errors. The E2E test embeds the script as a template string but unit tests have no access.
  • Required action: Either create the standalone shell script with the referenced functions, update the documentation to reference the E2E test's embedded scopeUpgradeScript() function, or add a follow-up task to extract the shell helpers.
  • Expected follow-up: Fix before merge or get explicit maintainer override.
  • Verification: Check if test/e2e/test-issue-4462-scope-upgrade-approval.sh exists; grep for redact_text_for_log_or_marker in the codebase.
  • Missing regression test: N/A - documentation fix
  • Done when: The required change is committed and verification passes: Check if test/e2e/test-issue-4462-scope-upgrade-approval.sh exists; grep for redact_text_for_log_or_marker in the codebase.
  • Evidence: test/e2e/lib/redact-text.py:37 references test/e2e/test-issue-4462-scope-upgrade-approval.sh in backticks; file does not exist per ls and grep.
Review findings by urgency: 3 required fixes, 6 items to resolve/justify, 2 in-scope improvements

⚠️ Resolve or justify before merge

Investigate these in the current review; either fix them, explain why they are not applicable, or document the accepted risk.

PRA-4 Resolve/justify — redact-text.py lacks stdin read failure test

  • Location: test/redact-text.test.ts:1
  • Category: tests
  • Problem: redact-text.py reads from sys.stdin.read() which could fail on broken pipes, closed stdin, or I/O errors. No test simulates stdin read failure to verify graceful handling or explicit failure with non-zero exit code.
  • Impact: If stdin read fails (e.g., broken pipe from upstream), the script may crash silently or produce corrupted output without proper error signaling.
  • Recommended action: Add a test simulating stdin read failure (e.g., spawnSync with closed stdin or a mocked failure) or justify why this is infeasible/low-risk.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Search test/redact-text.test.ts for stdin read failure test — none exist currently.
  • Missing regression test: Test simulating stdin read failure verifying graceful handling or explicit non-zero exit
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Search test/redact-text.test.ts for stdin read failure test — none exist currently.
  • Evidence: test/redact-text.py main() uses sys.stdin.read() with no try/except for IOError; test/redact-text.test.ts has 9 tests covering token shapes but no I/O error cases.

PRA-5 Resolve/justify — Auto-pair fast-reentry lacks stress test for many concurrent sandboxes

  • Location: scripts/nemoclaw-start.sh:2300
  • Category: architecture
  • Problem: Auto-pair fast-reentry mechanism (FAST_REENTRY_POLLS, FAST_REENTRY_BUMPED_REQUEST_IDS) lacks a stress test for many concurrent sandboxes. The bounded counter is designed to prevent sticky failing requests from pinning the watcher in fast polling, but this has not been validated under contention.
  • Impact: Under high contention from many sibling sandboxes with concurrent late scope upgrades, the fast-reentry counter bounds and garbage collection may not behave correctly, potentially causing watchdog pile-up or missed approvals.
  • Recommended action: Add a focused vitest that spawns multiple fake openclaw processes simulating many sibling sandboxes with concurrent late scope upgrades, verifying the fast-reentry counter bounds and garbage collection behave correctly under contention.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Search test/nemoclaw-start.test.ts for concurrent sandbox stress test — none exist currently.
  • Missing regression test: Vitest spawning multiple fake openclaw processes with concurrent late scope upgrades, verifying counter bounds and GC
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Search test/nemoclaw-start.test.ts for concurrent sandbox stress test — none exist currently.
  • Evidence: FAST_REENTRY_BUMPED_REQUEST_IDS is a process-local set; intersection_update GC runs each poll but untested under concurrent load.

PRA-6 Resolve/justify — Phase 7 differing-providers test coupled to scope-upgrade script and CPU-lane only

  • Location: test/e2e/live/issue-4462-scope-upgrade-approval.test.ts:200
  • Category: tests
  • Problem: Phase 7 differing-providers test remains coupled to the embedded scopeUpgradeScript() template and CPU-lane only. The read-host-registry.py is a good step toward decoupling, but the test still embeds the full shell script as a base64-encoded string and lacks GPU-lane validation.
  • Impact: Test cannot be run independently; GPU-lane provider configurations are not validated; changes to the embedded script require E2E test updates rather than a shared library.
  • Recommended action: Extract Phase 7 into a dedicated E2E script/job for independent execution and GPU-lane validation, or document why coupling is intentional.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Check if test/e2e/live/issue-4462-scope-upgrade-approval.test.ts still embeds scopeUpgradeScript() as a template string and lacks GPU provider matrix.
  • Missing regression test: Independent Phase 7 E2E job with GPU-lane provider matrix
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Check if test/e2e/live/issue-4462-scope-upgrade-approval.test.ts still embeds scopeUpgradeScript() as a template string and lacks GPU provider matrix.
  • Evidence: test/e2e/live/issue-4462-scope-upgrade-approval.test.ts lines 1-200 define scopeUpgradeScript() as a template string; no GPU provider matrix in test config.

PRA-7 Resolve/justify — _env_seconds() float/whitespace parsing vs watchdog strict regex inconsistency

  • Location: scripts/nemoclaw-start.sh:2237
  • Category: correctness
  • Problem: _env_seconds() accepts float values (e.g., '0.5') and whitespace-padded strings, while gateway_watchdog_positive_int_ok() uses strict regex ^[1-9][0-9]*$ that rejects floats, zero, and whitespace. This inconsistency is now documented but the two validators serve different purposes (auto-pair vs watchdog).
  • Impact: Operators setting NEMOCLAW_AUTO_PAIR_SLOW_INTERVAL_SECS=0.5 get 0.5s polling, but NEMOCLAW_GATEWAY_WATCHDOG_INTERVAL_SECONDS=0.5 falls back to 30s with a warning. Inconsistent UX and potential confusion.
  • Recommended action: Document why float parsing is intentional for auto-pair env vars (sub-second intervals), or add a shared positive-int/positive-float validator with consistent behavior. If sub-second is not needed for auto-pair, align with regex validation.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Compare _env_seconds() implementation (line ~2237) with gateway_watchdog_positive_int_ok() (line ~3841).
  • Missing regression test: Test documenting the intentional difference or aligning the validators
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Compare _env_seconds() implementation (line ~2237) with gateway_watchdog_positive_int_ok() (line ~3841).
  • Evidence: _env_seconds() uses float() conversion; gateway_watchdog_positive_int_ok() uses [[ =~ ^[1-9][0-9]*$ ]].

PRA-8 Resolve/justify — read-host-registry.py lacks path traversal validation on sandbox_name

  • Location: test/e2e/lib/read-host-registry.py:1
  • Category: security
  • Problem: read-host-registry.py reads ~/.nemoclaw/sandboxes.json from the host filesystem. Path construction uses os.environ.get('HOME', '/tmp') which could be influenced by environment. No path traversal validation on sandbox_name argument.
  • Impact: If sandbox_name contains path traversal sequences (e.g., '../../etc/passwd'), the script could read arbitrary files when the registry path is constructed. The script is only called from controlled E2E tests but defense-in-depth suggests input validation.
  • Recommended action: Add validation that sandbox_name contains only safe characters (alphanumeric, hyphen, underscore) and does not contain path traversal sequences.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Check read-host-registry.py main() for sandbox_name validation — none exists currently.
  • Missing regression test: Test with sandbox_name='../../etc/passwd' verifying rejection or safe handling
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Check read-host-registry.py main() for sandbox_name validation — none exists currently.
  • Evidence: read-host-registry.py line 16-17: sandbox_name = sys.argv[1] used directly in registry_file path construction via os.path.join.

PRA-9 Resolve/justify — test/nemoclaw-start.test.ts monolith growth (~2500 lines)

  • Location: test/nemoclaw-start.test.ts:1
  • Category: tests
  • Problem: test/nemoclaw-start.test.ts grew by ~100 lines (now ~2500 lines) with new auto-pair fixtures for fast-reentry and concurrent late upgrades. This contributes to monolith growth; consider splitting auto-pair tests into a dedicated file.
  • Impact: Test file exceeds typical size budgets; harder to maintain and navigate; auto-pair tests are a cohesive subset that could be extracted.
  • Recommended action: Consider extracting auto-pair watcher tests to test/nemoclaw-start-auto-pair.test.ts to maintain test file size budget and focus.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Check test/nemoclaw-start.test.ts line count and ci/test-file-size-budget.json.
  • Missing regression test: N/A - test organization
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Check test/nemoclaw-start.test.ts line count and ci/test-file-size-budget.json.
  • Evidence: File has 20 named test blocks covering non-root fallback, gateway preload, token export, auto-pair whitelisting, slow-mode, late-CLI, retries, migration, etc.

💡 In-scope improvements

These are lower-risk, not throwaway. Prefer fixing them in this PR when they are local to changed code; defer only with rationale or a linked follow-up.

PRA-10 Improvement — Scope-upgrade shell script embedded as template string

  • Location: test/e2e/live/issue-4462-scope-upgrade-approval.test.ts:1
  • Category: scope
  • Problem: Scope-upgrade shell script embedded as template string (scopeUpgradeScript()) in the E2E test; should be extracted to a standalone file for maintainability and reuse.
  • Impact: Script is ~1000 lines embedded in TypeScript test; changes require TypeScript string escaping; not reusable by other tests or workflows.
  • Suggested action: Extract the scopeUpgradeScript() template to test/e2e/test-cli-scope-upgrade-approval.sh and update all references (workflow, unit tests, comments). Can be done in follow-up PR.
  • Expected follow-up: Prefer a current-PR fix when local to changed code; defer only with rationale or linked follow-up.
  • Verification: Check if test/e2e/test-cli-scope-upgrade-approval.sh exists.
  • Missing regression test: N/A - refactoring
  • Done when: The local improvement is applied, or the PR notes why it should be deferred.
  • Evidence: test/e2e/live/issue-4462-scope-upgrade-approval.test.ts lines 40-500+ define scopeUpgradeScript() as a multi-line template string.

PRA-11 Improvement — inference-set.test.ts monolith growth (+5 lines)

  • Location: src/lib/actions/inference-set.test.ts:217
  • Category: correctness
  • Problem: inference-set.test.ts monolith grew by 5 lines (+_nemoclaw_upstream test coverage). Previously agreed to defer extraction to follow-up.
  • Impact: File at 1354 lines; adding tests without extraction increases maintenance burden.
  • Suggested action: Defer extraction to follow-up as previously agreed. No action needed in this PR.
  • Expected follow-up: Prefer a current-PR fix when local to changed code; defer only with rationale or linked follow-up.
  • Verification: File size at 1354 lines; track for future extraction.
  • Missing regression test: N/A - known technical debt
  • Done when: The local improvement is applied, or the PR notes why it should be deferred.
  • Evidence: Commit 199b4d0 adds _nemoclaw_upstream test case; previous review PRA-7 agreed to defer.
Simplification opportunities: 1 possible cut, net -950 lines possible

These are safe simplification checks only. Do not remove validation, security controls, data-loss prevention, or required tests.

  • PRA-10 shrink (test/e2e/live/issue-4462-scope-upgrade-approval.test.ts:1): scopeUpgradeScript() template string (~1000 lines) from test/e2e/live/issue-4462-scope-upgrade-approval.test.ts
    • Replacement: Separate file test/e2e/test-cli-scope-upgrade-approval.sh sourced or exec'd by the E2E test
    • Net: -950 lines
    • Safety boundary: Must preserve the exact shell script behavior including all Python embedded helpers and trap logic
Test follow-ups to resolve or justify

If these cover changed behavior, prefer adding them in this PR; otherwise state why existing coverage is enough or link the follow-up.

  • PRA-T1 Runtime validation — Integration test spawning full nemoclaw-start.sh with fake OpenClaw gateway, verifying watchdog + auto-pair + fast-reentry interaction. Runtime/sandbox/infrastructure paths need behavioral runtime validation: scripts/nemoclaw-start.sh. Unit tests extract shell functions but don't exercise the full start script in a sandbox-like environment with auto-pair + watchdog integration.
  • PRA-T2 Runtime validation — Runtime test for Ollama installer: version override + no SHA256 → guard triggers before sudo tar. Runtime/sandbox/infrastructure paths need behavioral runtime validation: scripts/nemoclaw-start.sh. Unit tests extract shell functions but don't exercise the full start script in a sandbox-like environment with auto-pair + watchdog integration.
  • PRA-T3 Runtime validation — Stdin read failure test for redact-text.py. Runtime/sandbox/infrastructure paths need behavioral runtime validation: scripts/nemoclaw-start.sh. Unit tests extract shell functions but don't exercise the full start script in a sandbox-like environment with auto-pair + watchdog integration.
  • PRA-T4 Runtime validation — Concurrent sandbox stress test for auto-pair fast-reentry bounds/GC. Runtime/sandbox/infrastructure paths need behavioral runtime validation: scripts/nemoclaw-start.sh. Unit tests extract shell functions but don't exercise the full start script in a sandbox-like environment with auto-pair + watchdog integration.
  • PRA-T5 Runtime validation — Independent Phase 7 E2E job with GPU-lane provider matrix. Runtime/sandbox/infrastructure paths need behavioral runtime validation: scripts/nemoclaw-start.sh. Unit tests extract shell functions but don't exercise the full start script in a sandbox-like environment with auto-pair + watchdog integration.
  • PRA-T6 Ollama installer full sudo tar path lacks runtime E2E test — Add an integration-style test that sources the shell install block, sets OLLAMA_PINNED_VERSION override with empty OLLAMA_PINNED_SHA256, and asserts the script exits before sudo tar with OLLAMA_PIN_REQUIRES_SHA256 message. Or explicitly justify that unit test coverage of the guard function is sufficient.
  • PRA-T7 Documentation references non-existent shell script test/e2e/test-issue-4462-scope-upgrade-approval.sh — Either create the standalone shell script with the referenced functions, update the documentation to reference the E2E test's embedded scopeUpgradeScript() function, or add a follow-up task to extract the shell helpers.
  • PRA-T8 redact-text.py lacks stdin read failure test — Add a test simulating stdin read failure (e.g., spawnSync with closed stdin or a mocked failure) or justify why this is infeasible/low-risk.
Since last review details

Current findings, using the urgency labels above:

PRA-1 Required — SLOW_INTERVAL default changed 30s→5s without CHANGELOG entry

  • Location: scripts/nemoclaw-start.sh:2274
  • Category: docs
  • Problem: The auto-pair watcher's slow-mode polling interval default was reduced from 30 seconds to 5 seconds (6x increase in steady-state polling frequency). The SOURCE_OF_TRUTH_REVIEW comment documents this well, but no CHANGELOG or release notes entry exists in the diff. Operators relying on the old 30s cadence for gateway load management will see 6x more `openclaw devices list --json` calls per sandbox without warning.
  • Impact: Operators on load-sensitive gateways or large multi-sandbox deployments experience unexpected 6x increase in connect-handler load. The env var NEMOCLAW_AUTO_PAIR_SLOW_INTERVAL_SECS=30 restores old behavior but requires proactive configuration.
  • Required action: Add a CHANGELOG.md entry documenting the SLOW_INTERVAL default change, the 6x polling increase, and the NEMOCLAW_AUTO_PAIR_SLOW_INTERVAL_SECS=30 opt-out. Consider whether 5s is appropriate for all deployment environments or if a more conservative default with opt-in to faster polling is safer.
  • Expected follow-up: Fix before merge or get explicit maintainer override.
  • Verification: Check CHANGELOG.md or release notes for mention of NEMOCLAW_AUTO_PAIR_SLOW_INTERVAL_SECS default change.
  • Missing regression test: Release notes documentation (non-code)
  • Done when: The required change is committed and verification passes: Check CHANGELOG.md or release notes for mention of NEMOCLAW_AUTO_PAIR_SLOW_INTERVAL_SECS default change.
  • Evidence: scripts/nemoclaw-start.sh:2274 sets SLOW_INTERVAL = _env_seconds('NEMOCLAW_AUTO_PAIR_SLOW_INTERVAL_SECS', 5) with SOURCE_OF_TRUTH_REVIEW comment documenting the change but no CHANGELOG entry in diff.

PRA-2 Required — Ollama installer full sudo tar path lacks runtime E2E test

  • Location: test/ollama-pinned-install.test.ts:1
  • Category: tests
  • Problem: The Ollama installer three-guard chain (SHA256 guard + computed SHA match + layout validator) is covered by unit tests, but the full install flow reaching 'sudo tar -xzf' is not exercised by any runtime test. A regression where the guard is bypassed before privileged extract would not be caught.
  • Impact: An attacker or misconfiguration overriding OLLAMA_PINNED_VERSION without providing a matching SHA256 could reach the privileged tar extraction step, potentially allowing a malicious tarball to execute arbitrary code via symlink/hardlink/FIFO escapes before layout validation runs.
  • Required action: Add an integration-style test that sources the shell install block, sets OLLAMA_PINNED_VERSION override with empty OLLAMA_PINNED_SHA256, and asserts the script exits before sudo tar with OLLAMA_PIN_REQUIRES_SHA256 message. Or explicitly justify that unit test coverage of the guard function is sufficient.
  • Expected follow-up: Fix before merge or get explicit maintainer override.
  • Verification: Search for test/ollama-pinned-install.test.ts — file does not exist in the repo.
  • Missing regression test: E2E test running Ollama install code path with version override + no SHA256, verifying guard triggers before sudo tar
  • Done when: The required change is committed and verification passes: Search for test/ollama-pinned-install.test.ts — file does not exist in the repo.
  • Evidence: Previous review PRA-8 evidence: test/ollama-pinned-install.test.ts only tests ollama_pinned_install_sha256_ok guard function and validate_ollama_tarball_layout validator in isolation; no test runs the full install block.

PRA-3 Required — Documentation references non-existent shell script test/e2e/test-issue-4462-scope-upgrade-approval.sh

  • Location: test/e2e/lib/redact-text.py:37
  • Category: tests
  • Problem: The redact-text.py docstring mentions `redact_text_for_log_or_marker` function in test/e2e/test-issue-4462-scope-upgrade-approval.sh. This file does not exist in the repository. While unit tests don't depend on it, the documentation implies a runtime dependency for fail-closed markers.
  • Impact: Confusing documentation; if callers expect this helper to exist, they may encounter missing function errors. The E2E test embeds the script as a template string but unit tests have no access.
  • Required action: Either create the standalone shell script with the referenced functions, update the documentation to reference the E2E test's embedded scopeUpgradeScript() function, or add a follow-up task to extract the shell helpers.
  • Expected follow-up: Fix before merge or get explicit maintainer override.
  • Verification: Check if test/e2e/test-issue-4462-scope-upgrade-approval.sh exists; grep for redact_text_for_log_or_marker in the codebase.
  • Missing regression test: N/A - documentation fix
  • Done when: The required change is committed and verification passes: Check if test/e2e/test-issue-4462-scope-upgrade-approval.sh exists; grep for redact_text_for_log_or_marker in the codebase.
  • Evidence: test/e2e/lib/redact-text.py:37 references test/e2e/test-issue-4462-scope-upgrade-approval.sh in backticks; file does not exist per ls and grep.

PRA-4 Resolve/justify — redact-text.py lacks stdin read failure test

  • Location: test/redact-text.test.ts:1
  • Category: tests
  • Problem: redact-text.py reads from sys.stdin.read() which could fail on broken pipes, closed stdin, or I/O errors. No test simulates stdin read failure to verify graceful handling or explicit failure with non-zero exit code.
  • Impact: If stdin read fails (e.g., broken pipe from upstream), the script may crash silently or produce corrupted output without proper error signaling.
  • Recommended action: Add a test simulating stdin read failure (e.g., spawnSync with closed stdin or a mocked failure) or justify why this is infeasible/low-risk.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Search test/redact-text.test.ts for stdin read failure test — none exist currently.
  • Missing regression test: Test simulating stdin read failure verifying graceful handling or explicit non-zero exit
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Search test/redact-text.test.ts for stdin read failure test — none exist currently.
  • Evidence: test/redact-text.py main() uses sys.stdin.read() with no try/except for IOError; test/redact-text.test.ts has 9 tests covering token shapes but no I/O error cases.

PRA-5 Resolve/justify — Auto-pair fast-reentry lacks stress test for many concurrent sandboxes

  • Location: scripts/nemoclaw-start.sh:2300
  • Category: architecture
  • Problem: Auto-pair fast-reentry mechanism (FAST_REENTRY_POLLS, FAST_REENTRY_BUMPED_REQUEST_IDS) lacks a stress test for many concurrent sandboxes. The bounded counter is designed to prevent sticky failing requests from pinning the watcher in fast polling, but this has not been validated under contention.
  • Impact: Under high contention from many sibling sandboxes with concurrent late scope upgrades, the fast-reentry counter bounds and garbage collection may not behave correctly, potentially causing watchdog pile-up or missed approvals.
  • Recommended action: Add a focused vitest that spawns multiple fake openclaw processes simulating many sibling sandboxes with concurrent late scope upgrades, verifying the fast-reentry counter bounds and garbage collection behave correctly under contention.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Search test/nemoclaw-start.test.ts for concurrent sandbox stress test — none exist currently.
  • Missing regression test: Vitest spawning multiple fake openclaw processes with concurrent late scope upgrades, verifying counter bounds and GC
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Search test/nemoclaw-start.test.ts for concurrent sandbox stress test — none exist currently.
  • Evidence: FAST_REENTRY_BUMPED_REQUEST_IDS is a process-local set; intersection_update GC runs each poll but untested under concurrent load.

PRA-6 Resolve/justify — Phase 7 differing-providers test coupled to scope-upgrade script and CPU-lane only

  • Location: test/e2e/live/issue-4462-scope-upgrade-approval.test.ts:200
  • Category: tests
  • Problem: Phase 7 differing-providers test remains coupled to the embedded scopeUpgradeScript() template and CPU-lane only. The read-host-registry.py is a good step toward decoupling, but the test still embeds the full shell script as a base64-encoded string and lacks GPU-lane validation.
  • Impact: Test cannot be run independently; GPU-lane provider configurations are not validated; changes to the embedded script require E2E test updates rather than a shared library.
  • Recommended action: Extract Phase 7 into a dedicated E2E script/job for independent execution and GPU-lane validation, or document why coupling is intentional.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Check if test/e2e/live/issue-4462-scope-upgrade-approval.test.ts still embeds scopeUpgradeScript() as a template string and lacks GPU provider matrix.
  • Missing regression test: Independent Phase 7 E2E job with GPU-lane provider matrix
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Check if test/e2e/live/issue-4462-scope-upgrade-approval.test.ts still embeds scopeUpgradeScript() as a template string and lacks GPU provider matrix.
  • Evidence: test/e2e/live/issue-4462-scope-upgrade-approval.test.ts lines 1-200 define scopeUpgradeScript() as a template string; no GPU provider matrix in test config.

PRA-7 Resolve/justify — _env_seconds() float/whitespace parsing vs watchdog strict regex inconsistency

  • Location: scripts/nemoclaw-start.sh:2237
  • Category: correctness
  • Problem: _env_seconds() accepts float values (e.g., '0.5') and whitespace-padded strings, while gateway_watchdog_positive_int_ok() uses strict regex ^[1-9][0-9]*$ that rejects floats, zero, and whitespace. This inconsistency is now documented but the two validators serve different purposes (auto-pair vs watchdog).
  • Impact: Operators setting NEMOCLAW_AUTO_PAIR_SLOW_INTERVAL_SECS=0.5 get 0.5s polling, but NEMOCLAW_GATEWAY_WATCHDOG_INTERVAL_SECONDS=0.5 falls back to 30s with a warning. Inconsistent UX and potential confusion.
  • Recommended action: Document why float parsing is intentional for auto-pair env vars (sub-second intervals), or add a shared positive-int/positive-float validator with consistent behavior. If sub-second is not needed for auto-pair, align with regex validation.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Compare _env_seconds() implementation (line ~2237) with gateway_watchdog_positive_int_ok() (line ~3841).
  • Missing regression test: Test documenting the intentional difference or aligning the validators
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Compare _env_seconds() implementation (line ~2237) with gateway_watchdog_positive_int_ok() (line ~3841).
  • Evidence: _env_seconds() uses float() conversion; gateway_watchdog_positive_int_ok() uses [[ =~ ^[1-9][0-9]*$ ]].

PRA-8 Resolve/justify — read-host-registry.py lacks path traversal validation on sandbox_name

  • Location: test/e2e/lib/read-host-registry.py:1
  • Category: security
  • Problem: read-host-registry.py reads ~/.nemoclaw/sandboxes.json from the host filesystem. Path construction uses os.environ.get('HOME', '/tmp') which could be influenced by environment. No path traversal validation on sandbox_name argument.
  • Impact: If sandbox_name contains path traversal sequences (e.g., '../../etc/passwd'), the script could read arbitrary files when the registry path is constructed. The script is only called from controlled E2E tests but defense-in-depth suggests input validation.
  • Recommended action: Add validation that sandbox_name contains only safe characters (alphanumeric, hyphen, underscore) and does not contain path traversal sequences.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Check read-host-registry.py main() for sandbox_name validation — none exists currently.
  • Missing regression test: Test with sandbox_name='../../etc/passwd' verifying rejection or safe handling
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Check read-host-registry.py main() for sandbox_name validation — none exists currently.
  • Evidence: read-host-registry.py line 16-17: sandbox_name = sys.argv[1] used directly in registry_file path construction via os.path.join.

PRA-9 Resolve/justify — test/nemoclaw-start.test.ts monolith growth (~2500 lines)

  • Location: test/nemoclaw-start.test.ts:1
  • Category: tests
  • Problem: test/nemoclaw-start.test.ts grew by ~100 lines (now ~2500 lines) with new auto-pair fixtures for fast-reentry and concurrent late upgrades. This contributes to monolith growth; consider splitting auto-pair tests into a dedicated file.
  • Impact: Test file exceeds typical size budgets; harder to maintain and navigate; auto-pair tests are a cohesive subset that could be extracted.
  • Recommended action: Consider extracting auto-pair watcher tests to test/nemoclaw-start-auto-pair.test.ts to maintain test file size budget and focus.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Check test/nemoclaw-start.test.ts line count and ci/test-file-size-budget.json.
  • Missing regression test: N/A - test organization
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Check test/nemoclaw-start.test.ts line count and ci/test-file-size-budget.json.
  • Evidence: File has 20 named test blocks covering non-root fallback, gateway preload, token export, auto-pair whitelisting, slow-mode, late-CLI, retries, migration, etc.

PRA-10 Improvement — Scope-upgrade shell script embedded as template string

  • Location: test/e2e/live/issue-4462-scope-upgrade-approval.test.ts:1
  • Category: scope
  • Problem: Scope-upgrade shell script embedded as template string (scopeUpgradeScript()) in the E2E test; should be extracted to a standalone file for maintainability and reuse.
  • Impact: Script is ~1000 lines embedded in TypeScript test; changes require TypeScript string escaping; not reusable by other tests or workflows.
  • Suggested action: Extract the scopeUpgradeScript() template to test/e2e/test-cli-scope-upgrade-approval.sh and update all references (workflow, unit tests, comments). Can be done in follow-up PR.
  • Expected follow-up: Prefer a current-PR fix when local to changed code; defer only with rationale or linked follow-up.
  • Verification: Check if test/e2e/test-cli-scope-upgrade-approval.sh exists.
  • Missing regression test: N/A - refactoring
  • Done when: The local improvement is applied, or the PR notes why it should be deferred.
  • Evidence: test/e2e/live/issue-4462-scope-upgrade-approval.test.ts lines 40-500+ define scopeUpgradeScript() as a multi-line template string.

PRA-11 Improvement — inference-set.test.ts monolith growth (+5 lines)

  • Location: src/lib/actions/inference-set.test.ts:217
  • Category: correctness
  • Problem: inference-set.test.ts monolith grew by 5 lines (+_nemoclaw_upstream test coverage). Previously agreed to defer extraction to follow-up.
  • Impact: File at 1354 lines; adding tests without extraction increases maintenance burden.
  • Suggested action: Defer extraction to follow-up as previously agreed. No action needed in this PR.
  • Expected follow-up: Prefer a current-PR fix when local to changed code; defer only with rationale or linked follow-up.
  • Verification: File size at 1354 lines; track for future extraction.
  • Missing regression test: N/A - known technical debt
  • Done when: The local improvement is applied, or the PR notes why it should be deferred.
  • Evidence: Commit 199b4d0 adds _nemoclaw_upstream test case; previous review PRA-7 agreed to defer.

Workflow run details

This is an automated, non-binding review; it still expects maintainers and agents to respond to each required or warning item. Treat suggestions as current-PR improvements when they touch changed code; defer only with maintainer rationale or a linked follow-up. A human maintainer must make the final merge decision.

@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ❌ Some jobs failed

Run: 28261023709
Target ref: fix/scope-upgrade-late-approval-race
Requested jobs: cli-scope-upgrade-approval-e2e,device-auth-health-e2e,cloud-onboard-e2e
Summary: 2 passed, 1 failed, 0 cancelled, 0 skipped

Job Result
cli-scope-upgrade-approval-e2e ✅ success
cloud-onboard-e2e ❌ failure
device-auth-health-e2e ✅ success

Failed jobs: cloud-onboard-e2e. Check run artifacts for logs.

@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ❌ Some jobs failed

Run: 28261844224
Target ref: fix/scope-upgrade-late-approval-race
Requested jobs: cloud-onboard-e2e
Summary: 0 passed, 1 failed, 0 cancelled, 0 skipped

Job Result
cloud-onboard-e2e ❌ failure

Failed jobs: cloud-onboard-e2e. Check run artifacts for logs.

@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ❌ Some jobs failed

Run: 28262730130
Target ref: fix/scope-upgrade-late-approval-race
Requested jobs: cloud-onboard-e2e
Summary: 0 passed, 1 failed, 0 cancelled, 0 skipped

Job Result
cloud-onboard-e2e ❌ failure

Failed jobs: cloud-onboard-e2e. Check run artifacts for logs.

prekshivyas and others added 11 commits June 26, 2026 19:35
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
…race' into fix/scope-upgrade-late-approval-race

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
…rink workflow test

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
…host registry reader

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
…ns, mark host registry script executable

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
…Ollama tar extract

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
…ape extensibility

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
@cv cv added the v0.0.71 Release target label Jun 28, 2026
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
@github-actions

github-actions Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

E2E Target Recommendation

Required E2E targets: issue-4462-scope-upgrade-approval, double-onboard, ubuntu-repo-cloud-openclaw
Optional E2E targets: None

Dispatch required E2E targets:

  • gh workflow run e2e.yaml --ref <pr-head-ref> --field jobs=issue-4462-scope-upgrade-approval
  • gh workflow run e2e.yaml --ref <pr-head-ref> --field jobs=double-onboard
  • gh workflow run e2e.yaml --ref <pr-head-ref> --field targets=ubuntu-repo-cloud-openclaw

Workflow run

Full E2E target advisor summary

E2E Target Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E targets

  • issue-4462-scope-upgrade-approval: The PR changes the in-sandbox auto-pair watcher cadence, fast re-entry, and retry handling for late OpenClaw CLI scope-upgrade approvals. The wired issue-4462 free-standing live job is the smallest dispatch that exercises scope-upgrade approval behavior end to end.
    • Dispatch: gh workflow run e2e.yaml --ref <pr-head-ref> --field jobs=issue-4462-scope-upgrade-approval
  • double-onboard: The auto-pair change explicitly targets late allowlisted upgrades during back-to-back sibling sandbox onboarding, and the new host-registry E2E helper is tied to the differing-provider/double-onboard surface. The wired double-onboard job is the focused live coverage for that multi-sandbox path.
    • Dispatch: gh workflow run e2e.yaml --ref <pr-head-ref> --field jobs=double-onboard
  • ubuntu-repo-cloud-openclaw: scripts/nemoclaw-start.sh is on the normal OpenClaw sandbox boot path. This live-supported canonical Ubuntu/OpenClaw target verifies the changed gateway startup, watchdog setup, and auto-pair watcher do not regress the standard cloud OpenClaw onboarding path.
    • Dispatch: gh workflow run e2e.yaml --ref <pr-head-ref> --field targets=ubuntu-repo-cloud-openclaw

Optional E2E targets

  • None.

Relevant changed files

  • scripts/nemoclaw-start.sh
  • test/e2e/lib/read-host-registry.py
  • test/e2e/lib/redact-device-state.py
  • test/e2e/lib/redact-text.py

@github-actions

Copy link
Copy Markdown
Contributor

Vitest E2E Target Results — ✅ All requested jobs passed

Run: 28461725354
Workflow ref: fix/scope-upgrade-late-approval-race
Requested targets: (default — all supported)
Requested jobs: issue-4462-scope-upgrade-approval,sandbox-operations,device-auth-health,gateway-guard-recovery
Summary: 4 passed, 0 failed, 0 cancelled, 0 skipped

Job Result
device-auth-health ✅ success
gateway-guard-recovery ✅ success
issue-4462-scope-upgrade-approval ✅ success
sandbox-operations ✅ success

@jyaunches jyaunches merged commit dca3a49 into main Jun 30, 2026
119 checks passed
@jyaunches jyaunches deleted the fix/scope-upgrade-late-approval-race branch June 30, 2026 17:30
@jyaunches jyaunches mentioned this pull request Jun 30, 2026
21 tasks
jyaunches added a commit that referenced this pull request Jun 30, 2026
<!-- markdownlint-disable MD041 -->
## Summary
Refreshes the public documentation for NemoClaw v0.0.71 after scanning
commits since v0.0.70. Adds release notes and fills the remaining doc
gaps for Windows bootstrap diagnostics, OpenClaw agent auto-relock
warnings, auto-pair cadence tuning, and plugin-install recovery hints.

## Changes
- `docs/about/release-notes.mdx`: adds the v0.0.71 release-note section,
grouped by gateway recovery, OpenShell auth, policy provenance, day-two
maintenance, messaging/inference, and Windows setup.
- `docs/get-started/windows-preparation.mdx`: documents sanitized WSL
install output and reboot gating in the Windows bootstrap.
- `docs/reference/commands.mdx`: documents the host `agent` wrapper's
shields auto-relock warning and OpenClaw auto-pair watcher tuning
variables.
- `docs/reference/troubleshooting.mdx`: adds plugin-install network
failure recovery guidance and updates Windows WSL troubleshooting for
sanitized install logs and reboot-required handling.

Source summary:
- #6065 -> `docs/about/release-notes.mdx`: Notes explicit model override
preservation and gateway-log guard-chain recovery diagnostics.
- #5874 -> `docs/about/release-notes.mdx`: Summarizes host-mediated
`recover` and `gateway restart`, linking to lifecycle, command,
troubleshooting, and trusted-boundary docs already added by the source
PR.
- #5596 -> `docs/about/release-notes.mdx`: Summarizes OpenShell 0.0.71
gateway auth, loopback binding, and compatibility-container docs already
added by the source PR.
- #5797 and #5798 -> `docs/about/release-notes.mdx`: Summarizes
`policy-list` provenance, Restricted tier suppression, and Balanced tier
weather behavior already reflected in policy docs.
- #5784 -> `docs/about/release-notes.mdx`: Summarizes
`--destroy-user-data` and the safe `--yes` uninstall behavior already
documented in lifecycle and command docs.
- #6034 -> `docs/about/release-notes.mdx`: Summarizes custom Dockerfile
warm-build cache behavior already documented in the command reference.
- #5951 -> `docs/reference/commands.mdx`: Documents the stderr-only host
`agent` wrapper warning after recent shields auto-relock.
- #5387 -> `docs/reference/commands.mdx`: Documents OpenClaw auto-pair
watcher cadence and fast-reentry tuning variables.
- #5835 -> `docs/reference/troubleshooting.mdx`: Adds recovery guidance
for OpenClaw plugin-install network failures.
- #5995 and #5956 -> `docs/about/release-notes.mdx`: Summarizes
Microsoft Teams final-message delivery and runtime mention hints already
covered by messaging docs.
- #5716 -> `docs/about/release-notes.mdx`: Summarizes non-interactive
Ollama loopback safety already covered by local inference docs.
- #5505, #5527, and #5528 -> `docs/about/release-notes.mdx`: Summarizes
compatible local endpoint, model task-fit, and model capability audit
docs.
- #6009 -> `docs/get-started/windows-preparation.mdx`,
`docs/reference/troubleshooting.mdx`: Documents sanitized Windows
bootstrap WSL output and reboot-required gating.
- #6055 -> no additional source doc page change needed beyond the
already-merged quickstart update; release notes did not duplicate
routine quickstart cleanup.

No matching v0.0.71 GitHub announcement discussion was found in the
latest 20 discussions, so this refresh is based on the commit scan and
existing source PR docs.

## Type of Change

- [ ] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [ ] Doc only (prose changes, no code sample modifications)
- [x] Doc only (includes code sample changes)

## Quality Gates
<!-- Check all that apply. For any "covered by existing tests", "not
applicable", or waiver entry, add a brief justification on the same line
or in the Changes section. -->
- [ ] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [x] Tests not applicable — justification: docs-only refresh with no
runtime behavior changes.
- [x] Docs updated for user-facing behavior changes
- [ ] Docs not applicable — justification:
- [ ] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification:
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Verification
<!-- Check each item you ran and confirmed. Leave unchecked items you
skipped. Doc-only changes do not require npm test unless you ran it. -->
- [x] PR description includes the DCO sign-off declaration and every
commit appears as `Verified` in GitHub
- [x] Git hooks passed during commit and push, or `npx prek run
--from-ref main --to-ref HEAD` passes
- [ ] Targeted tests pass for changed behavior
- [ ] Full `npm test` passes (broad runtime changes only)
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only) — ran
`npm run docs`; Fern reported 0 errors and 2 existing warnings.
- [x] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)

---
<!-- DCO sign-off is required in this PR description, and every commit
must appear as Verified in GitHub. Run: git config user.name && git
config user.email -->
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
* Added a new release-notes entry covering gateway recovery,
authentication, network policy/provenance output, uninstall safety,
Windows bootstrap diagnostics, messaging defaults, and inference setup
guidance.
* Clarified Windows preparation steps around reboot behavior and
redacting troubleshooting transcripts.
* Expanded command reference details for OpenClaw wrapper behavior and
new auto-pair tuning options.
* Improved troubleshooting guidance for plugin installation issues, WSL
repair/reboot cases, and install timing problems.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
Co-authored-by: Prekshi Vyas <34834085+prekshivyas@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug-fix PR fixes a bug or regression integration: openclaw OpenClaw integration behavior NV QA Bugs found by the NVIDIA QA Team v0.0.71 Release target

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[All Platforms][Inference] Multi-sandbox parallel routing test fails: both sandboxes hit gateway scope-upgrade block and fall back to embedded mode

6 participants