Skip to content

fix(sandbox): preserve structured MCP destroy failures (#8103) - #8324

Closed
cv wants to merge 20 commits into
mainfrom
codex/8298-verified-refresh
Closed

fix(sandbox): preserve structured MCP destroy failures (#8103)#8324
cv wants to merge 20 commits into
mainfrom
codex/8298-verified-refresh

Conversation

@cv

@cv cv commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

Summary

destroy now returns the existing structured agent adapter error when adapter preparation or finalization fails. Previously, the command exposed an uncaught stack trace. This PR preserves Yanyun Liao's contributor commit and Apurv Kumaria's two remediation commits with cherry-pick provenance. It supersedes #8298; #8103 continues to track separate rebuild authority drift.

Related Issue

Refs #8103

Changes

  • Catch McpBridgeError during MCP preparation and return before OpenShell sandbox deletion.
  • Catch McpBridgeError after confirmed sandbox deletion, redact the finalization error, and retain the sandbox registry entry and gateway.
  • Complete registry and gateway cleanup after a later destroy rerun without running openshell sandbox delete again.
  • Rethrow other error types and cover both agent adapter failure stages.
  • Preserve contributor commit 4f0f7ccd unchanged.
  • Carry Apurv Kumaria's remediations as author-attributed cherry-picks from commits adf49fd8 and 5934ec1a.
  • Apply the maintainer commits to base commit 2b90d18ce.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: Existing sandbox lifecycle documentation describes failure before deletion, a later destroy rerun, registry cleanup after confirmed deletion, and agent adapter ownership checks. This change returns that documented structured error instead of an uncaught stack trace.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Final review is in progress for commit df6acd128 against base commit 2b90d18ce. The merge changed no PR-touched file.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: No waiver. installer-integration failed and the required E2E / PR Gate is still running for commit df6acd128.

Documentation Writer Review

  • Documentation writer subagent reviewed the completed changes
  • Result: no-docs-needed
  • Evidence: The change corrects structured agent adapter destroy failure handling, redaction, and preflight sequencing without changing the documented command or recovery workflow. Existing managed Model Context Protocol (MCP) lifecycle documentation covers preparation, retained recovery state, post-deletion cleanup, and destroy reruns.
  • Agent: Codex Desktop

DGX Station Hardware Evidence

  • Tested on DGX Station
  • Tested commit: Not applicable; scripts/prepare-dgx-station-host.sh is unchanged.
  • Station profile/scenario: Not applicable.
  • Result: Not applicable.
  • Supporting evidence: Not applicable.

Verification

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run validate:pr passed after refreshing origin/main when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: GitHub Actions is running for commit df6acd128.
  • Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result: GitHub Actions is running for commit df6acd128; installer-integration must be resolved.
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Sensitive-path Security Review

  • Secrets and credentials: PASS — no secret source, persistence, or logging path changed. Structured output uses existing full redaction.
  • Input validation and injection: PASS — no parser or command construction changed. Only the typed McpBridgeError boundary is handled.
  • Authentication and authorization: PASS — managed MCP ownership checks are unchanged. Preparation still fails closed before deletion.
  • Dependencies: PASS — no dependency or artifact changed.
  • Error handling and logging: PASS — typed bridge failures return structured nonzero results. Unrelated errors still propagate.
  • Cryptography and data protection: PASS — no cryptographic or storage behavior changed.
  • Configuration and security controls: PASS — no option, policy, permission, or network default changed.
  • Security testing: PASS — the diff covers preparation failure, delete-before-finalize ordering, redacted preparation and finalization errors, retained registry and gateway state, and one openshell sandbox delete call across the initial destroy and its rerun. GitHub Actions must validate commit df6acd128.
  • System security: PASS — a preparation failure prevents sandbox deletion. A finalization failure preserves registry state and authority for a later destroy rerun. Confirmed absence prevents that rerun from deleting a same-name replacement.

Signed-off-by: Carlos Villela cvillela@nvidia.com

yanyunl1991 and others added 6 commits August 5, 2026 12:37
…ze in destroy path (#8103)

`executeSandboxDestroy` called `prepareMcpDestroy` and `finalizeMcpDestroy`
with no `McpBridgeError` guard. When a managed MCP server is present and the
gateway becomes unreachable, `inspectExactMcpDestroyProvider` throws
`McpBridgeError("Could not inspect OpenShell provider…")` which propagated
uncaught, crashing `destroy --yes` with a stack trace instead of a clean
exit-1 message. The same uncaught escape existed for `finalizeMcpDestroy`'s
internal re-throw after post-delete cleanup fails.

Both call sites now catch `McpBridgeError` and return `{ ok: false, … }`,
letting the existing failure-path rendering in `destroy.ts` surface the error
cleanly and exit with the error's own `exitCode`.

Signed-off-by: yanyunl1991 <yanyunl@nvidia.com>
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
@cv cv added area: sandbox OpenShell sandbox lifecycle, runtime, config, or recovery bug-fix PR fixes a bug or regression v0.0.103 Release target labels Aug 5, 2026
@cv cv self-assigned this Aug 5, 2026
@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The destroy flow now structures typed MCP bridge failures, skips deletion for already-absent sandboxes, and propagates deletion status to cleanup logic. Tests cover redaction, exit codes, state preservation, and successful reruns.

Changes

Sandbox destruction error handling

Layer / File(s) Summary
Destroy execution results and error handling
src/lib/actions/sandbox/destroy-execution.ts
executeSandboxDestroy returns redacted failure results for McpBridgeError, skips deletion when the sandbox is absent, and reports deleteSucceededOrAlreadyGone.
Propagate deletion status to cleanup
src/lib/actions/sandbox/destroy.ts
destroySandboxUnlocked uses the execution result for host-service cleanup and confirmed-destroy decisions.
Simulate and validate destroy outcomes
test/helpers/destroy-flow-test-harness.ts, test/helpers/destroy-flow-test-assertions.ts, src/lib/actions/sandbox/destroy-flow.test.ts
The harness simulates typed MCP failures and mutable sandbox presence. Tests verify deletion order, redaction, exit codes, preserved state, and successful reruns.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant DestroyFlow
  participant MCPBridge
  participant SandboxState
  DestroyFlow->>MCPBridge: Prepare MCP bridge
  MCPBridge-->>DestroyFlow: McpBridgeError or prepared state
  DestroyFlow->>SandboxState: Delete sandbox when present
  DestroyFlow->>MCPBridge: Finalize MCP bridge
  MCPBridge-->>DestroyFlow: Success or McpBridgeError
  DestroyFlow-->>SandboxState: Apply cleanup from structured result
Loading

Possibly related PRs

Suggested reviewers: apurvvkumaria, ericksoa, yimoj

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: preserving structured MCP failures during sandbox destruction.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/8298-verified-refresh

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit df6acd1 in the codex/8298-verified-... branch remains at 96%, unchanged from commit 2b90d18 in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit df6acd1 in the codex/8298-verified-... branch remains at 81%, unchanged from commit 2b90d18 in the main branch.

Show a code coverage summary of the most impacted files.
File main 2b90d18 codex/8298-verified-... df6acd1 +/-
src/lib/inferen...er-discovery.ts 88% 87% -1%
src/lib/actions...dbox/destroy.ts 89% 89% 0%
src/lib/sandbox...rce-identity.ts 87% 87% 0%
src/lib/credentials/store.ts 55% 56% +1%
src/lib/actions...oy-execution.ts 89% 92% +3%
src/lib/domain/.../connect-env.ts 89% 97% +8%

Updated August 05, 2026 11:45 UTC

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/helpers/destroy-flow-test-assertions.ts`:
- Around line 232-237: Update expectMcpFinalizeBridgeErrorReturnsFailure to
assert that the sandbox deletion operation for “alpha” was called and that
finalizeMcpBridgesAfterSandboxDeleteSpy occurred after that deletion. Preserve
the existing assertions that finalization ran and registry cleanup did not
occur.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 2cb4ecb6-0768-4ccd-8209-69227eddcf34

📥 Commits

Reviewing files that changed from the base of the PR and between a82887e and ca7e49c.

📒 Files selected for processing (4)
  • src/lib/actions/sandbox/destroy-execution.ts
  • src/lib/actions/sandbox/destroy-flow.test.ts
  • test/helpers/destroy-flow-test-assertions.ts
  • test/helpers/destroy-flow-test-harness.ts

Comment thread test/helpers/destroy-flow-test-assertions.ts Outdated
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — Blocking findings reported

Advisor assessment: Blockers require maintainer review
Next action: Review the blockers below.
Findings: 1 blocker · 0 warnings · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · high confidence · 1 blocker · 0 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Completed · high confidence · 0 blockers · 0 warnings · 1 suggestion
  • Model comparison: normalized findings differ; normalized terminology decisions differ; normalized E2E selections differ; Nemotron reported 1 fewer blocker, the same number of warnings, 1 more suggestion.
3 additional E2E selections from the second opinion

Advisory only. The primary lane did not select these E2E jobs or targets.

  • ubuntu-repo-docker-post-reboot-recovery: The completed second-opinion lane identified E2E coverage that the primary lane omitted.
  • sandbox-operations: The completed second-opinion lane identified E2E coverage that the primary lane omitted.
  • sandbox-survival: The completed second-opinion lane identified E2E coverage that the primary lane omitted.

Second-opinion terminology and E2E selections are advisory. They do not change the primary assessment or E2E / PR Gate.

2 semantic terminology decisions

Terminology decisions are advisory. They affect the assessment only when a separate finding identifies concrete semantic impact.

  • established — MCP bridge at src/lib/actions/sandbox/destroy-flow.test.ts:377: Keep `MCP bridge` in the test titles because it matches established repository usage.
  • justified — confirmed absence at src/lib/actions/sandbox/destroy-execution.ts:301: Keep `confirmed absence`; the modifier states the security-relevant condition for skipping deletion.

E2E guidance

Advisory only. E2E / PR Gate selects and runs jobs independently.

Recommended E2E: onboard-repair, onboard-resume

1 optional E2E recommendation
  • mcp-bridge

Blockers

PRA-1 Blocker — Revalidate remote absence before cleanup after an absent preflight

  • Location: src/lib/actions/sandbox/destroy-execution.ts:301
  • Category: correctness
  • Problem: The absent-preflight branch skips `sandbox delete` but returns `deleteSucceededOrAlreadyGone: true` after local cleanup and MCP finalization. A same-name sandbox can be created after preflight, so the caller can remove the registry and session and stop shared services or remove the gateway without deleting or rechecking that replacement.
  • Impact: A destroy rerun can tear down host services and discard ownership state while a newly created same-name sandbox exists.
  • Fix: Before reporting confirmed deletion from the preflight-absent branch, revalidate that the named sandbox remains absent immediately before cleanup that can retire registry or shared gateway state. If it exists, retain local state and return a controlled failure.
  • Verification: Inspect the gateway calls and state mutations in the absent-preflight branch; simulate a same-name sandbox appearing after preflight in the existing destroy-flow harness.
  • Test coverage: Add a destroy-flow test that reports absence during preflight, creates `alpha` before post-preflight cleanup, and proves the command retains the registry, session, and gateway services.
  • Evidence: src/lib/actions/sandbox/destroy-execution.ts:301-349 skips deletion after preflight and then calls `cleanupShieldsArtifacts` and MCP finalization. src/lib/actions/sandbox/destroy.ts:583-627 removes the registry and session and may clean up the last gateway when `deleteSucceededOrAlreadyGone` is true. src/lib/actions/sandbox/destroy-flow.test.ts:401-430 changes presence only before the rerun and verifies one delete, not a replacement created after the absent preflight.

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

@cv

cv commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator Author

Maintainer review of commit ca7e49cda0db590780250592fcd219862056cc7a on 2026-08-05:

At that commit, PR #8324 was not approval-ready.

  • CodeRabbit correctly required evidence that sandbox deletion completed before agent adapter finalization.
  • PR fix(sandbox): catch McpBridgeError in destroy MCP bridge path (#8103) #8313 contained additional finalization-detail redaction and a later destroy cleanup test. Contributor attribution would need to be preserved if that work were transferred.
  • Build and type-check jobs failed because the base-branch package fixture omitted nemoclaw/dist/shared/openshell-policy-boundary.cjs.
  • Installer integration reported one DGX Station pair-preparation failure after 535 passing tests. That workflow needed a rerun after the deterministic package failure was corrected.

At that review, neither #8313 nor #8324 was eligible. Comparator scores were 13.5/16 for #8313 and 12.5/16 for #8324. This comment records the historical review of commit ca7e49cda0db590780250592fcd219862056cc7a; later commits and current GitHub checks supersede its status.

cv and others added 6 commits August 5, 2026 03:20
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
(cherry picked from commit 5934ec1)
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
(cherry picked from commit adf49fd)
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
(cherry picked from commit 5934ec1)

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/lib/actions/sandbox/destroy-flow.test.ts (1)

377-397: 🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win

Security Test Coverage (CWE-532): Insertion of Sensitive Information into Log File

Exploitability: Theoretical

Cover redaction for MCP preparation errors.

prepareMcpBridgeError has no secret marker, so the test does not prove redaction for preparation failures. Add a credential-shaped marker and assert that it is absent from all captured error arguments. The finalization path currently emits each message as one argument, so the later-argument concern applies only if that output contract changes.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/actions/sandbox/destroy-flow.test.ts` around lines 377 - 397, Update
the MCP preparation failure test using createDestroyHarness and
expectMcpPrepareBridgeErrorAborts to include a credential-shaped secret marker
in prepareMcpBridgeError, then assert the marker is absent from every captured
error argument. Keep the existing finalization test and its single-argument
output contract unchanged.

Source: Path instructions

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/actions/sandbox/destroy-flow.test.ts`:
- Around line 408-418: Update the destroySandbox test around
harness.destroySandbox to assert through the public runOpenshell boundary that
the command for deleting sandbox alpha is invoked exactly once after the retry
path completes. Keep the existing absent-sandbox preparation and MCP bridge
assertions, and ensure the assertion distinguishes the initial delete from any
retry.

---

Outside diff comments:
In `@src/lib/actions/sandbox/destroy-flow.test.ts`:
- Around line 377-397: Update the MCP preparation failure test using
createDestroyHarness and expectMcpPrepareBridgeErrorAborts to include a
credential-shaped secret marker in prepareMcpBridgeError, then assert the marker
is absent from every captured error argument. Keep the existing finalization
test and its single-argument output contract unchanged.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 94e6cc2b-321a-4b83-a6a5-9ce89fa8c13f

📥 Commits

Reviewing files that changed from the base of the PR and between ca7e49c and e386e38.

📒 Files selected for processing (4)
  • src/lib/actions/sandbox/destroy-execution.ts
  • src/lib/actions/sandbox/destroy-flow.test.ts
  • test/helpers/destroy-flow-test-assertions.ts
  • test/helpers/destroy-flow-test-harness.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • test/helpers/destroy-flow-test-assertions.ts
  • src/lib/actions/sandbox/destroy-execution.ts

Comment thread src/lib/actions/sandbox/destroy-flow.test.ts
cv added 2 commits August 5, 2026 03:44
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
(cherry picked from commit d312465)
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
cv and others added 6 commits August 5, 2026 03:52
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Aug 5, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@cv

cv commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by #8313, merged in commit ac24de0. The merged PR carries the structured MCP destroy error handling for #8103 and preserves the contributor history from Yanyun Liao, Apurv Kumaria, and Carlos Villela. The remaining #8324 changes affect the path used when the sandbox is already absent and are not required for that fix.

@cv cv closed this Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: sandbox OpenShell sandbox lifecycle, runtime, config, or recovery bug-fix PR fixes a bug or regression v0.0.103 Release target

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants