feat(status): warn when the gateway route drifts from the sandbox's recorded route

[Hokonoken]

Summary

nemoclaw <name> status previously showed the live host-gateway inference route as though it were the sandbox's own route. When an out-of-band operation such as direct openshell inference set makes that route differ from the sandbox's recorded provider/model, this PR now emits an explicit warning and safe remediation guidance.

Related Issue

Refs #6315. This complements #6338's fail-closed containment for supported flows by surfacing residual out-of-band drift in status; it follows the connect-time divergence behavior from #3726.

Changes

• Compute routeDrift only when a complete, readable live route differs from a complete recorded route.
• Show the live and recorded routes, explain that connect realigns to the recorded route, and show the supported inference set --sandbox path for adopting the live route.
• Share one control-character-sanitized warning formatter between status and connect so the messages cannot drift.
• Cover divergent, aligned, unreadable, and missing-record cases plus rendered status output.
• Document the warning in the existing one-route-per-gateway guidance.

Original Author Credit

@Hokonoken authored the diagnosis, implementation, tests, live proof, and documentation. The maintainer synchronization preserves those commits, and the review-fix commit records Hokonoken as co-author.

Type of Change

• Code change (feature, bug fix, or refactor)
• Code change with doc updates
• Doc only (prose changes, no code sample modifications)
• Doc only (includes code sample changes)

Quality Gates

• Tests added or updated for changed behavior
• Existing tests cover changed behavior — justification:
• Tests not applicable — justification:
• Docs updated for user-facing behavior changes
• Docs not applicable — justification:
• Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
• Sensitive-path review completed — display-only route comparison; no route, registry, credential, or sandbox mutation is introduced; untrusted route values use the shared sanitizer
• Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Verification

• PR description includes DCO declarations and every commit is GitHub Verified
• Normal pre-commit, commit-msg, and pre-push hooks passed on the current head
• Focused config, connect-route, status-flow, and snapshot tests — 87/87 passed with one local worker
• npm run build:cli — passed
• Plugin build — passed
• npm run docs — 0 errors and 2 pre-existing Fern warnings
• Documentation-writer review confirmed the existing page update is accurate and sufficient
• Quality Gates section completed with required justifications
• No secrets, API keys, or credentials committed
• Doc pages follow the style guide
• New doc pages include SPDX header and frontmatter (new pages only)

---

Signed-off-by: Hokonoken 41166525+Hokonoken@users.noreply.github.com
Signed-off-by: Apurv Kumaria akumaria@nvidia.com

Summary by CodeRabbit

• New Features
  • Sandbox status now detects “route drift” between the live gateway inference route and the sandbox’s recorded route, showing a warning with commands to realign or adopt the live route.
• Bug Fixes
  • If the live route can’t be read, route-drift warnings are suppressed to avoid misleading output.
• Tests
  • Added/expanded coverage for route-drift metadata, warning/sanitization, and safe shell-escaping in emitted commands.
• Documentation
  • Updated “switch inference providers” docs with the new route-drift warning and resolution steps.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

Adds route drift tracking between live and recorded inference routes. The snapshot records drift, status output prints a warning with guidance when drift exists, connect reuses the same formatter, and tests plus docs cover the behavior.

Changes

Sandbox status route drift

Layer / File(s)	Summary
Shared drift formatter src/lib/inference/config.ts, src/lib/inference/config.test.ts, src/lib/actions/sandbox/connect.ts, src/lib/actions/sandbox/connect-route-lifecycle.test.ts	Adds a structured inference route-drift formatter, tests its sanitized output, and reuses it in the sandbox connect warning path.
Route drift in status snapshot src/lib/actions/sandbox/status-snapshot.ts, test/support/status-flow-test-harness.ts, src/lib/actions/sandbox/status-snapshot-route-drift.test.ts	Extends SandboxStatusSnapshot with routeDrift, computes it from live OpenShell inference versus the recorded sandbox route, and covers snapshot behavior in tests and harness support.
Route drift status rendering src/lib/actions/sandbox/status-text.ts, src/lib/actions/sandbox/status.ts, src/lib/actions/sandbox/status-flow.test.ts	Threads routeDrift through showSandboxStatus and prints a drift warning with command guidance when present.
Route drift docs docs/inference/switch-inference-providers.mdx	Documents the warning text shown when the live inference route differs from the recorded sandbox route.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Suggested labels: area: cli, area: sandbox, bug-fix

Suggested reviewers: jyaunches

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: adding a status warning when the live gateway route differs from the sandbox's recorded route.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

🧹 Nitpick comments (1)

src/lib/actions/sandbox/status-text.ts (1)

241-263: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Extract the shared route-drift warning text. src/lib/actions/sandbox/status-text.ts and src/lib/actions/sandbox/connect.ts duplicate the same sanitized warning line, so moving it into one helper would keep the status and connect messaging from drifting apart.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/actions/sandbox/status-text.ts` around lines 241 - 263, The
route-drift warning text is duplicated between the status text and connect flow,
which risks the messages drifting apart. Extract the shared sanitized warning
line into a single helper used by printInferenceRouteDrift in status-text.ts and
the corresponding logic in connect.ts, so both paths build the same warning from
the same source. Keep the helper focused on formatting the live provider/model
versus the recorded route, and reuse it wherever the gateway inference route
drift message is printed.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/lib/actions/sandbox/status-text.ts`:
- Around line 241-263: The route-drift warning text is duplicated between the
status text and connect flow, which risks the messages drifting apart. Extract
the shared sanitized warning line into a single helper used by
printInferenceRouteDrift in status-text.ts and the corresponding logic in
connect.ts, so both paths build the same warning from the same source. Keep the
helper focused on formatting the live provider/model versus the recorded route,
and reuse it wherever the gateway inference route drift message is printed.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: ea787716-2518-4608-8385-47f70088aac5

📥 Commits

Reviewing files that changed from the base of the PR and between be3e7cc and 69671e2.

📒 Files selected for processing (7)

• docs/inference/switch-inference-providers.mdx
• src/lib/actions/sandbox/status-flow.test.ts
• src/lib/actions/sandbox/status-snapshot-route-drift.test.ts
• src/lib/actions/sandbox/status-snapshot.ts
• src/lib/actions/sandbox/status-text.ts
• src/lib/actions/sandbox/status.ts
• test/support/status-flow-test-harness.ts

[Hokonoken]

Validated this end-to-end on a live sandbox (OpenShell 0.0.72, hermes --no-gpu, NVIDIA Endpoints provider) — driving the exact scenario the warning targets: a direct openshell inference set that moves the gateway route out from under a registered sandbox.

Drift case — after openshell inference set --provider nvidia-prod --model nvidia/llama-3.3-nemotron-super-49b-v1 --no-verify -g nemoclaw (sandbox's recorded route is nvidia-prod/nvidia/nemotron-3-super-120b-a12b):

$ nemoclaw repro6454 status
  Sandbox: repro6454
    Model:    nvidia/llama-3.3-nemotron-super-49b-v1
    Provider: nvidia-prod
    Warning: gateway inference route (nvidia-prod/nvidia/llama-3.3-nemotron-super-49b-v1) differs from the recorded route for this sandbox (nvidia-prod/nvidia/nemotron-3-super-120b-a12b).
    'nemoclaw repro6454 connect' realigns the gateway to nvidia-prod/nvidia/nemotron-3-super-120b-a12b; to adopt the live route instead:
      nemoclaw inference set --provider nvidia-prod --model nvidia/llama-3.3-nemotron-super-49b-v1 --sandbox repro6454

Aligned case — after restoring the route to the recorded one, the warning is correctly absent (no false positive):

$ nemoclaw repro6454 status
    Model:    nvidia/nemotron-3-super-120b-a12b
    Provider: nvidia-prod
    (no route-drift warning)

The drift line reads the live route and the registry entry and mutates nothing, and the nemoclaw <name> connect hint uses the canonical name-first order.

[apurvvkumaria]

Maintainer salvage is pushed at exact head 3d4bf8f85.

• merged current main and resolved the sole status-text conflict without dropping current inference-health behavior;
• addressed the CodeRabbit maintainability finding by sharing one sanitized route-drift warning formatter between status and connect;
• added a direct control-character regression test for that shared formatter;
• preserved @Hokonoken's two original Verified commits and recorded Hokonoken as co-author on the review-fix commit;
• refreshed the PR body and DCO declarations for the current ten-file diff.

Exact-head evidence:

• focused config, connect-route, status-flow, and snapshot tests: 87/87 passed with one worker;
• CLI and plugin builds: passed;
• repository pre-commit checks and all pre-push typechecks: passed;
• docs build: 0 errors, 2 pre-existing Fern warnings;
• independent documentation-writer review: the existing shared-route warning update is accurate and sufficient;
• both maintainer commits are signed and GitHub Verified;
• GitHub now reports the PR mergeable.

Please re-review the current head. I will monitor CI and automated findings; no approval or merge action was taken.

[cjagwani]

/ok to test 7636e5c

[coderabbitai]

🧹 Nitpick comments (1)

docs/inference/switch-inference-providers.mdx (1)

52-53: 📐 Maintainability & Code Quality | 🔵 Trivial

Name the remediation commands explicitly.

“The supported command” is too vague here. Spell out $$nemoclaw <name> connect for realignment and $$nemoclaw inference set --provider ... --model ... --sandbox <name> for adopting the live route so readers do not have to infer the two paths from earlier text.

Proposed wording

-If the live gateway route still ends up differing from a sandbox's recorded route, for example after a direct `openshell inference set`, `$$nemoclaw <name> status` prints a warning naming both routes and the supported command to realign or adopt the live route.
+If the live gateway route still differs from a sandbox's recorded route, for example after a direct `openshell inference set`, `$$nemoclaw <name> status` prints a warning naming both routes and pointing to `$$nemoclaw <name> connect` for realignment or `$$nemoclaw inference set --provider ... --model ... --sandbox <name>` for adopting the live route.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/inference/switch-inference-providers.mdx` around lines 52 - 53, The
status warning in the documentation is too vague about how to fix a route
mismatch. Update the wording around `$$nemoclaw <name> status` to explicitly
name both remediation commands: use `$$nemoclaw <name> connect` for realigning
the sandbox route, and `$$nemoclaw inference set --provider ... --model ...
--sandbox <name>` for adopting the live route. Keep the guidance tied to the
existing route-mismatch flow so readers can choose the correct command
immediately.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@docs/inference/switch-inference-providers.mdx`:
- Around line 52-53: The status warning in the documentation is too vague about
how to fix a route mismatch. Update the wording around `$$nemoclaw <name>
status` to explicitly name both remediation commands: use `$$nemoclaw <name>
connect` for realigning the sandbox route, and `$$nemoclaw inference set
--provider ... --model ... --sandbox <name>` for adopting the live route. Keep
the guidance tied to the existing route-mismatch flow so readers can choose the
correct command immediately.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 19eda4a0-84de-4c4f-9be1-83648ba101ac

📥 Commits

Reviewing files that changed from the base of the PR and between 3d4bf8f and 7636e5c.

📒 Files selected for processing (1)

• docs/inference/switch-inference-providers.mdx

[cjagwani]

/ok to test 5db5f67

[cjagwani]

/ok to test f8ee55e

[coderabbitai]

🧹 Nitpick comments (1)

src/lib/actions/sandbox/status-snapshot-route-drift.test.ts (1)

58-96: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Prefer outcome-based verification over mock-call assertions where possible.

Test 1 (Line 70) relies on expect(capture).toHaveBeenCalledWith([...]) to confirm the correct gateway name was queried. Since liveGatewayInference appears to make capture resolve a fixed value regardless of the arguments passed, an outcome-only assertion (routeDrift, currentProvider, currentModel) wouldn't catch a bug where the wrong gateway name is used but happens to still return the same mocked payload — the mock-call assertion is currently the only signal for that failure mode.

Consider making the liveGatewayInference/mock helper gateway-aware (e.g., return distinct data or reject for an unexpected gateway argument) so the existing outcome assertions alone can prove the correct gateway was queried, removing the need to assert on internal call shape. Test 2's expect(capture).not.toHaveBeenCalled() (Line 91) is comparatively lower-risk since snapshot.routeDrift already differentiates the fallback-bug case, but the same principle applies.

As per path instructions, tests under **/*.test.{ts,js,mts,mjs,cts,cjs} should "Prefer observable outcomes through the public boundary over source-text, private-shape, or mock-call assertions."

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/actions/sandbox/status-snapshot-route-drift.test.ts` around lines 58
- 96, The sandbox status snapshot tests rely too much on `capture` call
assertions instead of proving behavior through outcomes. Update the
`liveGatewayInference` test helper and related `snapshotDeps` setup in
`collectSandboxStatusSnapshot` tests so the mocked gateway response depends on
the queried gateway name, making an unexpected gateway either return different
data or fail. Then keep the assertions focused on `routeDrift`,
`currentProvider`, and `currentModel`, and remove the brittle
`expect(capture).toHaveBeenCalledWith(...)`/`not.toHaveBeenCalled()` checks
where the public results can verify the behavior.

Source: Path instructions

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/lib/actions/sandbox/status-snapshot-route-drift.test.ts`:
- Around line 58-96: The sandbox status snapshot tests rely too much on
`capture` call assertions instead of proving behavior through outcomes. Update
the `liveGatewayInference` test helper and related `snapshotDeps` setup in
`collectSandboxStatusSnapshot` tests so the mocked gateway response depends on
the queried gateway name, making an unexpected gateway either return different
data or fail. Then keep the assertions focused on `routeDrift`,
`currentProvider`, and `currentModel`, and remove the brittle
`expect(capture).toHaveBeenCalledWith(...)`/`not.toHaveBeenCalled()` checks
where the public results can verify the behavior.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 2a2bde8f-19a5-449a-beeb-9390692e4964

📥 Commits

Reviewing files that changed from the base of the PR and between 5db5f67 and f8ee55e.

📒 Files selected for processing (2)

• src/lib/actions/sandbox/status-snapshot-route-drift.test.ts
• src/lib/actions/sandbox/status-snapshot.ts

🚧 Files skipped from review as they are similar to previous changes (1)

• src/lib/actions/sandbox/status-snapshot.ts

[cjagwani]

/ok to test a2e05b5

[apurvvkumaria]

/ok to test 99a5410

[apurvvkumaria]

Exact head 99a541096115c637f15ebef22287c772902865bb is mergeable with the full current check set complete and no failures or unresolved review threads.

The latest signed, GitHub-Verified follow-up addresses the remaining CodeRabbit test-quality finding: the mocked gateway response now depends on the queried gateway name, and the assertions prove behavior through routeDrift, currentProvider, and currentModel outcomes rather than mock-call shape. The commit preserves @Hokonoken as co-author. Focused tests, source-shape/title gates, repository hooks, builds, documentation impact review, and pre-push type checks all passed.

This PR is ready for independent human review. No approval or merge action was taken.