fix(start): drop slow-mode polling on late allowlisted scope upgrades#5387
Conversation
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThe PR renames nightly scope-upgrade jobs to CLI-scope variants, adds bounded fast-reentry polling in the auto-pair watcher, redacts captured device-state JSON through new helpers, and updates the CLI scope-upgrade E2E script and tests for the new flow. ChangesCLI scope-upgrade approval flow and redaction support
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related issues
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Code Coverage OverviewLanguages: TypeScript TypeScript / code-coverage/pluginThe overall coverage in the branch is 96%. Coverage data for the branch is not yet available. Show a code coverage summary of the most covered files.
TypeScript / code-coverage/cliThe overall coverage in the branch is 68%. Coverage data for the branch is not yet available. Show a code coverage summary of the most covered files.
Updated |
E2E Advisor RecommendationRequired E2E: Dispatch hint: Full advisor summaryE2E Recommendation AdvisorBase: Required E2E
Optional E2E
New E2E recommendations
Dispatch hint
|
Vitest E2E Scenario RecommendationRequired Vitest E2E scenarios: Dispatch required Vitest E2E scenarios:
Full Vitest E2E advisor summaryVitest E2E Scenario AdvisorBase: Required Vitest E2E scenarios
Optional Vitest E2E scenarios
Relevant changed files
|
PR Review Advisor — BlockedMerge posture: Do not merge until addressed Action checklist
Findings index
🚨 Required before mergeAddress these before merging unless a maintainer explicitly overrides the advisor with rationale.
|
Selective E2E Results — ❌ Some jobs failedRun: 27463690336
|
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/nemoclaw-start.sh`:
- Around line 2050-2061: The fast-reentry counter is being reinitialized every
loop when attempted_approval is true, allowing a single failing pending request
to keep fast mode indefinitely; change the logic in the watcher loop so
FAST_REENTRY_REMAINING is only reset once per approval lifecycle (e.g., on the
rising edge of attempted_approval or when FAST_REENTRY_REMAINING == 0) instead
of every loop iteration: add a small persistent flag like
prev_attempted_approval or check FAST_REENTRY_REMAINING to detect transitions
and only set FAST_REENTRY_REMAINING = FAST_REENTRY_POLLS and print the log when
the transition occurs (leave APPROVED and SLOW_MODE usage as-is).
In `@test/nemoclaw-start.test.ts`:
- Around line 1746-1859: The test added ("drops slow-mode polling back to fast
cadence when a late scope upgrade arrives") exceeds the file line budget;
extract the duplicated late-cli fixture and setup into a small reusable helper
(e.g. create a function like createLateCliFixture that returns { tmpDir,
fakeOpenclaw, stateFile, approveLog } and sets up the script and permissions)
and call that helper from this test (and the adjacent similar test) instead of
inlining the long bash string and env setup, or alternatively fold the
fast-reentry assertions into the existing late-upgrade test to avoid duplicating
the entire fixture; locate references to fakeOpenclaw, stateFile, approveLog,
and buildAutoPairScript() to update call sites to use the helper and remove the
duplicated lines from the test body.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 5b40365e-a235-4bb4-bb63-2de367836ecb
📒 Files selected for processing (5)
.github/workflows/nightly-e2e.yamlscripts/nemoclaw-start.shtest/e2e-script-workflow.test.tstest/e2e/test-cli-scope-upgrade-approval.shtest/nemoclaw-start.test.ts
Selective E2E Results — ❌ Some jobs failedRun: 27463736935
|
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/e2e/test-cli-scope-upgrade-approval.sh`:
- Around line 1082-1086: The grep check for '[auto-pair] fast-reentry bumped'
using auto_pair_log_snapshot should be made informational: in the else branch
replace the fail call with a non-fatal log/warn (or call a function like warn or
echo) so the script doesn't hard-fail when the watcher didn't log the marker;
keep the pass branch unchanged. Update the block that currently uses grep -F ...
and calls fail to instead emit a warning/message while allowing the test to
continue.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 5a348f44-52fd-43f1-a690-23bae8b5c178
📒 Files selected for processing (4)
.github/workflows/nightly-e2e.yamlscripts/nemoclaw-start.shtest/e2e/test-cli-scope-upgrade-approval.shtest/nemoclaw-start.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
- test/nemoclaw-start.test.ts
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
Selective E2E Results — ❌ Some jobs failedRun: 27464785119
|
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
Selective E2E Results — ❌ Some jobs failedRun: 27465290873
|
…failures Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
…approve failures" This reverts commit bba211b.
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/nemoclaw-start.test.ts`:
- Around line 2248-2250: The test's assertion currently only matches
"fast-reentry bumped polls=3 approved=0 mode=fast" and can miss alternate bump
lines; update the marker and expectation used in test variable markerRe (and the
subsequent expect using run.stdout.match(...).length) to match any "fast-reentry
bumped" line (ignore the particular field values) and assert the total number of
such occurrences is exactly 1 so any extra bump log (e.g., with approved=1) will
fail the test.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 97da2f1c-484a-4ce2-a416-670cedbb0123
📒 Files selected for processing (1)
test/nemoclaw-start.test.ts
…ng-edge assertion Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
Selective E2E Results — ❌ Some jobs failedRun: 28251337754
|
PR Review Advisor (Nemotron Ultra) — Changes requestedMerge posture: Do not merge yet Action checklist
Findings index
🚨 Required before mergeAddress these before merging unless a maintainer explicitly overrides the advisor with rationale.
|
Selective E2E Results — ❌ Some jobs failedRun: 28261023709
|
Selective E2E Results — ❌ Some jobs failedRun: 28261844224
|
Selective E2E Results — ❌ Some jobs failedRun: 28262730130
|
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
…race' into fix/scope-upgrade-late-approval-race Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
…rink workflow test Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
…host registry reader Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
…ns, mark host registry script executable Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
…Ollama tar extract Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
…ape extensibility Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
E2E Target RecommendationRequired E2E targets: Dispatch required E2E targets:
Full E2E target advisor summaryE2E Target AdvisorBase: Required E2E targets
Optional E2E targets
Relevant changed files
|
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
Vitest E2E Target Results — ✅ All requested jobs passedRun: 28461725354
|
<!-- markdownlint-disable MD041 --> ## Summary Refreshes the public documentation for NemoClaw v0.0.71 after scanning commits since v0.0.70. Adds release notes and fills the remaining doc gaps for Windows bootstrap diagnostics, OpenClaw agent auto-relock warnings, auto-pair cadence tuning, and plugin-install recovery hints. ## Changes - `docs/about/release-notes.mdx`: adds the v0.0.71 release-note section, grouped by gateway recovery, OpenShell auth, policy provenance, day-two maintenance, messaging/inference, and Windows setup. - `docs/get-started/windows-preparation.mdx`: documents sanitized WSL install output and reboot gating in the Windows bootstrap. - `docs/reference/commands.mdx`: documents the host `agent` wrapper's shields auto-relock warning and OpenClaw auto-pair watcher tuning variables. - `docs/reference/troubleshooting.mdx`: adds plugin-install network failure recovery guidance and updates Windows WSL troubleshooting for sanitized install logs and reboot-required handling. Source summary: - #6065 -> `docs/about/release-notes.mdx`: Notes explicit model override preservation and gateway-log guard-chain recovery diagnostics. - #5874 -> `docs/about/release-notes.mdx`: Summarizes host-mediated `recover` and `gateway restart`, linking to lifecycle, command, troubleshooting, and trusted-boundary docs already added by the source PR. - #5596 -> `docs/about/release-notes.mdx`: Summarizes OpenShell 0.0.71 gateway auth, loopback binding, and compatibility-container docs already added by the source PR. - #5797 and #5798 -> `docs/about/release-notes.mdx`: Summarizes `policy-list` provenance, Restricted tier suppression, and Balanced tier weather behavior already reflected in policy docs. - #5784 -> `docs/about/release-notes.mdx`: Summarizes `--destroy-user-data` and the safe `--yes` uninstall behavior already documented in lifecycle and command docs. - #6034 -> `docs/about/release-notes.mdx`: Summarizes custom Dockerfile warm-build cache behavior already documented in the command reference. - #5951 -> `docs/reference/commands.mdx`: Documents the stderr-only host `agent` wrapper warning after recent shields auto-relock. - #5387 -> `docs/reference/commands.mdx`: Documents OpenClaw auto-pair watcher cadence and fast-reentry tuning variables. - #5835 -> `docs/reference/troubleshooting.mdx`: Adds recovery guidance for OpenClaw plugin-install network failures. - #5995 and #5956 -> `docs/about/release-notes.mdx`: Summarizes Microsoft Teams final-message delivery and runtime mention hints already covered by messaging docs. - #5716 -> `docs/about/release-notes.mdx`: Summarizes non-interactive Ollama loopback safety already covered by local inference docs. - #5505, #5527, and #5528 -> `docs/about/release-notes.mdx`: Summarizes compatible local endpoint, model task-fit, and model capability audit docs. - #6009 -> `docs/get-started/windows-preparation.mdx`, `docs/reference/troubleshooting.mdx`: Documents sanitized Windows bootstrap WSL output and reboot-required gating. - #6055 -> no additional source doc page change needed beyond the already-merged quickstart update; release notes did not duplicate routine quickstart cleanup. No matching v0.0.71 GitHub announcement discussion was found in the latest 20 discussions, so this refresh is based on the commit scan and existing source PR docs. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [x] Doc only (includes code sample changes) ## Quality Gates <!-- Check all that apply. For any "covered by existing tests", "not applicable", or waiver entry, add a brief justification on the same line or in the Changes section. --> - [ ] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [x] Tests not applicable — justification: docs-only refresh with no runtime behavior changes. - [x] Docs updated for user-facing behavior changes - [ ] Docs not applicable — justification: - [ ] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [ ] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification <!-- Check each item you ran and confirmed. Leave unchecked items you skipped. Doc-only changes do not require npm test unless you ran it. --> - [x] PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub - [x] Git hooks passed during commit and push, or `npx prek run --from-ref main --to-ref HEAD` passes - [ ] Targeted tests pass for changed behavior - [ ] Full `npm test` passes (broad runtime changes only) - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) — ran `npm run docs`; Fern reported 0 errors and 2 existing warnings. - [x] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- <!-- DCO sign-off is required in this PR description, and every commit must appear as Verified in GitHub. Run: git config user.name && git config user.email --> Signed-off-by: Julie Yaunches <jyaunches@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Added a new release-notes entry covering gateway recovery, authentication, network policy/provenance output, uninstall safety, Windows bootstrap diagnostics, messaging defaults, and inference setup guidance. * Clarified Windows preparation steps around reboot behavior and redacting troubleshooting transcripts. * Expanded command reference details for OpenClaw wrapper behavior and new auto-pair tuning options. * Improved troubleshooting guidance for plugin installation issues, WSL repair/reboot cases, and install timing problems. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Signed-off-by: Julie Yaunches <jyaunches@nvidia.com> Co-authored-by: Prekshi Vyas <34834085+prekshivyas@users.noreply.github.com>
Summary
Fixes the multi-sandbox parallel routing race where two sandboxes onboarded back-to-back each hit
scope upgrade pending approval+pairing requiredon the gateway and both fall back to embedded mode. The in-sandbox auto-pair watcher kept polling at 30s slow-mode cadence after browser-pairing convergence; lateropenclaw tui/openclaw agentallowlisted scope upgrades waited up to one fullSLOW_INTERVALbefore approval, longer than the OpenClaw client's pending-approval tolerance. Lowers the slow-poll default to 5s and arms a bounded fast-reentry counter on the rising edge of each fresh allowlisted approval attempt. Phase 7 E2E covers the differing-provider concurrent-routing case end to end.Related Issue
Fixes #5343
Changes
scripts/nemoclaw-start.sh:SLOW_INTERVALdefault 30s → 5s.sleep_for_next_poll(default, productive=True)and boundedFAST_REENTRY_*state — rising-edge bump per fresh allowlisted requestId, single-marker log, sticky requests do not re-arm. Error-path sleeps passproductive=Falseso gateway/JSON errors don't drain the bounded window.case [1-9]*glob (which accepted12x/30abc) →=~ ^[1-9][0-9]*$regex.test/nemoclaw-start.test.ts: extractsetupLateCliFixture; cover two concurrent lateopenclaw-cliupgrades + sticky-failure no-rearm assertion.test/e2e/test-issue-4462-scope-upgrade-approval.sh:v0.13.5tarball, SHA256-verified, archive-layout-validated before privileged extract). Reads provider+model from host registry and effectivebaseUrlfrom/sandbox/.openclaw/openclaw.jsoninside each sandbox; fails closed when missing/non-inference.local. Two concurrentopenclaw agentturns pinned to:18789and:18790with strict no-fallback / no-scope-upgrade / no-pairing-marker assertions. Default modelqwen3:0.6bsubstitutes the GPU-onlyqwen3.5:9b; the result summary surfaces which lane ran.$APPROVAL_LOG/$AGENT_LOGpasses throughredact_text_for_log_or_marker;${var:0:N}truncated excerpts infail/infomessages now useredacted_excerptso secrets never reach uploaded artefacts.device_state_json()pipes through the device-state redactor.read_host_registry_provider_model+read_sandbox_openclaw_routehelpers;extract_openclaw_upstreamis now a thin merger.ollama_pinned_install_sha256_okso a unit test can exercise both branches.test/e2e/lib/redact-device-state.py(new): single source of truth for device-state secret redaction.test/e2e/lib/redact-text.py(new): token-shape redactor for diagnostic excerpts.sk-…branch is left-bounded to avoid false positives on hyphenated text.test/redact-device-state.test.ts(new): device-state redactor + sandbox-exec wrapper failure paths.E2E_DIRpassed via env (not interpolated into the harness string) for CodeQL alert fix: improve gateway lifecycle recovery #953; failure-path assertions usestdout.trim().toBe(...).test/redact-text.test.ts(new): focused vitest forredact-text.pyplus assertions that the E2E script appends redacted output to every secret-bearing log.test/ollama-pinned-install.test.ts(new): runtime negative test — version override without SHA256 fails closed.test/e2e-script-workflow.test.ts: Phase 7 guardrails (Ollama sha256, CPU-lane substitute disclosure, provider-not-model identification for sandbox B).src/lib/actions/inference-set.test.ts: track_nemoclaw_upstreamannotation written by main to keep the no-op assertion truthful after rebase..github/workflows/nightly-e2e.yaml: lower approval-jobNEMOCLAW_AUTO_PAIR_SLOW_INTERVAL_SECS600 → 5 so CI exercises the new code path.ci/test-file-size-budget.json: pintest/e2e-script-workflow.test.tslegacy budget after maintainer rebase grew the file.Type of Change
Quality Gates
docs/.@prekshivyasapproved (fix(start): drop slow-mode polling on late allowlisted scope upgrades #5387 (review)) covering fast-reentry semantics, sticky-failure handling, and the 30→5 migration callout; later secret-redaction and effective-route hardening mirror the same boundary.Verification
Verifiedin GitHubnpx prek run --from-ref main --to-ref HEADpassesnpm testpasses (broad runtime changes only)npm run docsbuilds without warnings (doc changes only)Signed-off-by: Tinson Lai tinsonl@nvidia.com