Skip to content

fix(shields): refuse a corrupt transition lock without a raw stack trace - #8122

Merged
prekshivyas merged 8 commits into
mainfrom
fix/shields-lock-timeout-clean-failure
Aug 4, 2026
Merged

fix(shields): refuse a corrupt transition lock without a raw stack trace#8122
prekshivyas merged 8 commits into
mainfrom
fix/shields-lock-timeout-clean-failure

Conversation

@laitingsheng

@laitingsheng laitingsheng commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Summary

A shields transition lock whose owner record cannot be parsed is refused on purpose, but the refusal was reached only after the full 30-second wait timeout and then surfaced as a raw Node stack trace. Shields commands now refuse as soon as the unreadable record is observed, and report it through the existing shields exit sentinel, so shields up, shields down, and the read-only shields status all fail closed with a single line plus the recovery hint.

Related Issue

Fixes #8108

Changes

  • src/lib/shields/transition-lock.ts: split the wait-reason text into a reason and a recovery hint so a caller can present them on separate lines. Every existing message is byte-identical; the error's message is still the reason followed by the hint.
  • src/lib/shields/transition-lock.ts: added ShieldsTransitionLockUnavailableError for a lock that cannot be taken, with a name-keyed guard. The guard matches on shape rather than instanceof because dist and src copies of the class can both be loaded, the same reason isDeferredShieldsExit already works that way.
  • src/lib/shields/transition-lock.ts: an owner record that is unreadable and already past the malformed-stale threshold is refused immediately instead of polling until the wait timeout. That state is terminal: it cannot become recoverable by waiting, and no automatic recovery applies to it. A record younger than the threshold still waits, because a writer can finish it.
  • src/lib/shields/index.ts: completeDeferredShieldsExit prints the reason and a Recovery: line, then routes through failShieldsCommand. Both throwOnError settings reach the same sentinel, so shields up and shields down (which pass throwOnError: true) and shields status (which does not) all exit 1 without a stack trace.
  • Tests: src/lib/shields/transition-lock.test.ts covers the immediate refusal on the synchronous and asynchronous acquisition paths and the reason/hint split; test/cli/shields-transition-lock.test.ts runs public shields status, shields up, and shields down commands against a truncated lock file and asserts the exit code, both output lines, the absence of stack frames, preservation of the lock, and completion well inside the old timeout.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: no documented behavior changes. docs/security/tcb-boundary.mdx already states that the transition lock rejects ambiguous owners, and docs/manage-sandboxes/runtime-controls.mdx and docs/manage-sandboxes/backup-restore.mdx describe only which operations take the lock. No page documents the previous wait duration or the stack trace.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Independent nine-category review passed for commit a986513e25472c8654c91c8f54d0c847522a3493 against 49ffca04d0ac3ec3a1715e2fa3455c76db15d803 with no findings.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: None requested; every required check must pass.

Documentation Writer Review

  • Documentation writer subagent reviewed the completed changes
  • Result: no-docs-needed
  • Evidence: The remote commit tree 421952aedc74a60b88238f352644f9f1eeaceacb is identical to the independently reviewed tree. Existing docs/security/tcb-boundary.mdx, docs/reference/host-files-and-state.mdx, and docs/changelog/2026-07-15.mdx already document fail-closed handling and manual recovery for malformed or ambiguous transition-lock owners. This change corrects refusal timing and error presentation without changing command grammar, configuration, supported procedures, or lifecycle behavior. Changed messages and test titles follow the writing rules. GitHub CI provides validation for this commit.
  • Agent: Codex Desktop independent documentation writer

DGX Station Hardware Evidence

  • Tested on DGX Station
  • Tested commit: Not applicable
  • Station profile/scenario: Not applicable
  • Result: Not applicable; this PR does not change scripts/prepare-dgx-station-host.sh.
  • Supporting evidence: Not applicable

Verification

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub.
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run validate:pr passed after refreshing origin/main when hooks were skipped or unavailable — the normal pre-push checks passed for the identical reviewed tree, and GitHub commit lint passed for the contributor's current merge commit.
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — GitHub's aggregate checks gate passed for the current commit.
  • Applicable broad gate passed — GitHub CI is authoritative and pending.
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, credentials, or private keys are added by this diff.
  • npm run docs builds without warnings (doc changes only) — Not applicable; no documentation file changed.
  • Doc pages follow the style guide (doc changes only) — independent writing review passed.
  • New doc pages include SPDX header and frontmatter (new pages only) — Not applicable; no documentation page was added.

GitHub CI is authoritative.
No duplicate local test, documentation, build, or typecheck suite ran after the merge refresh.

Security Review

  • Result: PASS
  • Reviewed commit: a986513e25472c8654c91c8f54d0c847522a3493
  • Base commit: 49ffca04d0ac3ec3a1715e2fa3455c76db15d803
  • Secrets and credentials: PASS
  • Input validation and data sanitization: PASS
  • Authentication and authorization: PASS
  • Dependencies and third-party libraries: PASS
  • Error handling and logging: PASS
  • Cryptography and data protection: PASS
  • Configuration and security controls: PASS
  • Security testing: PASS
  • System security: PASS
  • Evidence: Old malformed owner records are refused without deletion or takeover; recent partial records still wait; existing inode and ownership protections remain. Public tests cover shields status, shields up, and shields down, asserting exit code 1, prompt refusal, bounded recovery output without a raw stack, and an unchanged malformed lock. Synchronous and asynchronous manager paths assert immediate refusal without sleeping.
  • Agent: Codex Desktop independent security reviewer

Signed-off-by: Tinson Lai tinsonl@nvidia.com

Summary by CodeRabbit

  • Bug Fixes
    • Improved handling of corrupted or stale Shields transition locks.
    • Commands now fail promptly with clear recovery guidance instead of waiting for a timeout.
    • Lock files are preserved for troubleshooting.
    • shields status, shields up, and shields down now exit cleanly without stack traces when a malformed lock is detected.
    • Error messages now provide sanitized, actionable details for resolving lock issues.
    • Transition-lock failures now include clearer summaries and recovery instructions.

An unreadable owner record can never be recovered by waiting, but shields
commands polled it for the full wait timeout and then let a plain Error reach
the user. Refuse as soon as the record is observed and report it through the
shields exit sentinel, so up, down, and read-only status all fail closed with
one line plus the recovery hint.

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The Shields transition-lock flow now rejects stale malformed lock records immediately. It exposes structured recovery details, prints sanitized deferred-command errors, preserves the lock file, and adds synchronous, asynchronous, and CLI coverage.

Changes

Shields transition-lock handling

Layer / File(s) Summary
Structured lock errors and diagnostics
src/lib/shields/transition-lock.ts
Wait-state formatting now returns separate summary and recovery text. ShieldsTransitionLockUnavailableError and its type guard expose structured lock failure details.
Fail-fast acquisition and deferred exit handling
src/lib/shields/transition-lock.ts, src/lib/shields/index.ts
Synchronous and asynchronous acquisition reject unrecoverable owners without retry waits. Deferred Shields exit handling prints the structured failure details.
Malformed-lock validation
src/lib/shields/transition-lock.test.ts, test/cli/shields-transition-lock.test.ts
Tests verify immediate failure, lock preservation, clean CLI output, non-zero exit status, and behavior for shields status, shields up, and shields down.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

  • NVIDIA/NemoClaw#8074: Both PRs modify Shields transition-lock failure handling and recovery diagnostics.
  • NVIDIA/NemoClaw#8130: Both PRs modify Shields transition-lock acquisition and recovery behavior.
  • NVIDIA/NemoClaw#8143: Both PRs modify Shields transition-lock handling in src/lib/shields/index.ts.

Suggested labels: area: security

Suggested reviewers: apurvvkumaria

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The changes address issue #8108 by failing fast, preserving corrupt locks, providing recovery guidance, and covering up, down, and status commands.
Out of Scope Changes check ✅ Passed All implementation and test changes directly support corrupt transition-lock handling and the requirements in issue #8108.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: refusing corrupt Shields transition locks without exposing raw stack traces.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/shields-lock-timeout-clean-failure

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit a986513 in the fix/shields-lock-tim... branch remains at 96%, unchanged from commit 5f87afa in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit a986513 in the fix/shields-lock-tim... branch remains at 81%, unchanged from commit 5f87afa in the main branch.

Show a code coverage summary of the most impacted files.
File main 5f87afa fix/shields-lock-tim... a986513 +/-
src/lib/shields...nsition-lock.ts 88% 87% -1%
src/lib/credentials/store.ts 56% 55% -1%
src/lib/sandbox...rce-identity.ts 88% 88% 0%
src/lib/state/g...way-registry.ts 94% 94% 0%
src/lib/state/m...-acquisition.ts 75% 75% 0%
src/lib/shields/index.ts 69% 70% +1%
src/lib/onboard...shboard-port.ts 90% 96% +6%

Updated August 04, 2026 03:10 UTC

@laitingsheng laitingsheng added NV QA Bugs found by the NVIDIA QA Team area: sandbox OpenShell sandbox lifecycle, runtime, config, or recovery bug-fix PR fixes a bug or regression labels Aug 3, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/cli/shields-transition-lock.test.ts`:
- Around line 31-69: Add a corrupt-transition-lock test alongside the existing
cases in the “shields commands with a corrupt transition lock” suite, invoking
the public `alpha shields up` command and verifying exit code 1, clean refusal
output via `expectCleanRefusal`, completion within `REFUSAL_BUDGET_MS`, and
unchanged malformed lock contents. Keep the setup consistent with the existing
`shields status` and `shields down` tests.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 23c0e00a-fe78-4cc1-b32c-80c3727c2f6c

📥 Commits

Reviewing files that changed from the base of the PR and between 4cd4d64 and d1729d9.

📒 Files selected for processing (4)
  • src/lib/shields/index.ts
  • src/lib/shields/transition-lock.test.ts
  • src/lib/shields/transition-lock.ts
  • test/cli/shields-transition-lock.test.ts

Comment on lines +31 to +69
describe("shields commands with a corrupt transition lock", () => {
it(
"refuses read-only status without a raw stack trace (#8108)",
testTimeoutOptions(30_000),
({ testHome }) => {
const { home } = testHome;
writeSandboxRegistry(home);
const lockPath = writeCorruptTransitionLock(home, "alpha");

const startedAt = Date.now();
const status = runWithEnv("alpha shields status 2>&1", testHome.environment());
const elapsedMs = Date.now() - startedAt;

expect(status.code).toBe(1);
expectCleanRefusal(status.out);
expect(elapsedMs).toBeLessThan(REFUSAL_BUDGET_MS);
expect(fs.readFileSync(lockPath, "utf8")).toBe("");
},
);

it(
"refuses shields down without a raw stack trace (#8108)",
testTimeoutOptions(30_000),
({ testHome }) => {
const { home } = testHome;
writeSandboxRegistry(home);
const lockPath = writeCorruptTransitionLock(home, "alpha");

const startedAt = Date.now();
const down = runWithEnv("alpha shields down --reason test 2>&1", testHome.environment());
const elapsedMs = Date.now() - startedAt;

expect(down.code).toBe(1);
expectCleanRefusal(down.out);
expect(elapsedMs).toBeLessThan(REFUSAL_BUDGET_MS);
expect(fs.readFileSync(lockPath, "utf8")).toBe("");
},
);
});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Add corrupt-lock coverage for shields up.

This suite covers shields status and shields down only. Add a shields up case that verifies exit code 1, clean recovery output, prompt refusal, and preservation of the malformed lock file.

As per path instructions, tests must prove that public entrypoints reach the new path. The PR objective includes shields up, shields down, and shields status.

🧰 Tools
🪛 ast-grep (0.45.0)

[warning] 46-46: Filesystem path is not a string literal; a request-/variable-derived path can enable path traversal. Validate and normalize the path before use.
Context: fs.readFileSync(lockPath, "utf8")
Note: [CWE-22] Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal').

(detect-non-literal-fs-filename-typescript)


[warning] 65-65: Filesystem path is not a string literal; a request-/variable-derived path can enable path traversal. Validate and normalize the path before use.
Context: fs.readFileSync(lockPath, "utf8")
Note: [CWE-22] Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal').

(detect-non-literal-fs-filename-typescript)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/cli/shields-transition-lock.test.ts` around lines 31 - 69, Add a
corrupt-transition-lock test alongside the existing cases in the “shields
commands with a corrupt transition lock” suite, invoking the public `alpha
shields up` command and verifying exit code 1, clean refusal output via
`expectCleanRefusal`, completion within `REFUSAL_BUDGET_MS`, and unchanged
malformed lock contents. Keep the setup consistent with the existing `shields
status` and `shields down` tests.

Source: Path instructions

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Model comparison: normalized findings match; normalized terminology decisions differ; normalized E2E selections differ; severity counts match.
4 terminology differences from the second opinion

Advisory only. These are normalized differences from the primary terminology receipt.

  • WaitReasonDescription at src/lib/shields/transition-lock.ts:328: selected only by the second-opinion lane as justified.
  • ShieldsTransitionLockUnavailableError at src/lib/shields/transition-lock.ts:380: selected only by the second-opinion lane as justified.
  • failFastOnUnrecoverableOwner at src/lib/shields/transition-lock.ts:1029: selected only by the second-opinion lane as justified.
  • describeWaitReason at src/lib/shields/transition-lock.ts:333: selected only by the second-opinion lane as replace.
2 additional E2E selections from the second opinion

Advisory only. The primary lane did not select these E2E jobs or targets.

  • shields-config: The completed second-opinion lane identified E2E coverage that the primary lane omitted.
  • ubuntu-repo-cloud-openclaw: The completed second-opinion lane identified E2E coverage that the primary lane omitted.

Second-opinion terminology and E2E selections are advisory. They do not change the primary assessment or E2E / PR Gate.

E2E guidance

Advisory only. E2E / PR Gate selects and runs jobs independently.

Recommended E2E: cloud-inference, security-posture

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

@laitingsheng laitingsheng removed the NV QA Bugs found by the NVIDIA QA Team label Aug 3, 2026
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
@prekshivyas prekshivyas self-assigned this Aug 3, 2026

@apurvvkumaria apurvvkumaria left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed current head d1729d9. Stale malformed owner records fail fast while recent partial writes still wait; the corrupt lock remains untouched, and the CLI path exits cleanly after lock cleanup. Sync, async, and public CLI paths are covered, with required CI and E2E passing. I found no blocking defect.

cv added 4 commits August 3, 2026 16:39
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>

@apurvvkumaria apurvvkumaria left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve — reviewed exact head f070bbe. Old malformed transition locks now fail closed immediately through the typed CLI sentinel, preserve the lock, and render actionable recovery without a raw stack. The current head includes public-entrypoint coverage for status, up, and down, satisfying the still-open automated test thread. Unit coverage passes 47 of 47; the exact CI CLI failure is an unrelated managed-image registry fetch timeout. No blocking correctness, security, compatibility, or regression defect found.

@laitingsheng laitingsheng added the v0.0.102 Release target label Aug 4, 2026
@prekshivyas
prekshivyas merged commit b724932 into main Aug 4, 2026
99 of 101 checks passed
@prekshivyas
prekshivyas deleted the fix/shields-lock-timeout-clean-failure branch August 4, 2026 06:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: sandbox OpenShell sandbox lifecycle, runtime, config, or recovery bug-fix PR fixes a bug or regression v0.0.102 Release target

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Ubuntu 26.04][Security] Corrupt shields transition-lock crashes shields up/down and read-only status with raw Node traceback after 30s hang

4 participants