fix(sandbox): restore residual capability warning behavior

[cv]

Summary

Reverts the residual-capability fail-closed behavior from #4266 and the CI residual-cap opt-in from #4335. This restores the prior behavior where sandbox entrypoints warn about residual dangerous capabilities instead of refusing to start when CAP_SETPCAP is unavailable.

Related Issue

Reverts #4266.
Reverts #4335.

Changes

• Removed the NEMOCLAW_ALLOW_RESIDUAL_CAPS entrypoint opt-in and fail-closed residual-capability policy from scripts/lib/sandbox-init.sh and src/lib/onboard.ts.
• Removed the workflow-level residual-capability opt-in from reusable and nightly E2E workflows.
• Reverted the related E2E, sandbox-init, workflow-contract, and command-reference test/doc updates.

Type of Change

• Code change (feature, bug fix, or refactor)
• Code change with doc updates
• Doc only (prose changes, no code sample modifications)
• Doc only (includes code sample changes)

Verification

• npx prek run --all-files passes
• npm test passes
• Tests added or updated for new or changed behavior
• No secrets, API keys, or credentials committed
• Docs updated for user-facing behavior changes
• npm run docs builds without warnings (doc changes only)
• Doc pages follow the style guide (doc changes only)
• New doc pages include SPDX header and frontmatter (new pages only)

---

Signed-off-by: Carlos Villela cvillela@nvidia.com

Summary by CodeRabbit

• Chores

  • Removed residual-capabilities allowlist from CI workflows and sandbox startup; sandbox now forwards runtime tool tokens and explicitly strips host kube/SSH credentials.
  • Streamlined environment-variable handling for sandbox operations.

• Bug Fixes

  • Improved capability-drop failure handling and residual-capability reporting to avoid forced failures.

• Tests

  • Simplified non-root execution checks and updated workflow contract types and e2e test flows.

• Documentation

  • Removed deprecated onboarding configuration entry.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: a4ba3d96-e255-4b17-a9b9-9d88a2a7628a

📥 Commits

Reviewing files that changed from the base of the PR and between 6183451 and 34ea16f.

📒 Files selected for processing (1)

• src/lib/onboard.ts

---

📝 Walkthrough

Walkthrough

This PR removes the NEMOCLAW_ALLOW_RESIDUAL_CAPS environment variable feature across sandbox initialization, workflow configuration, and test coverage. The sandbox environment injection is replaced with conditional TOOL_GATEWAY_USER_TOKEN support when a Hermes tool broker token is available, and capability enforcement logic is simplified to report without enforcing policy.

Changes

Residual capability allowance removal and tool broker token injection

Layer / File(s)	Summary
Sandbox capability enforcement simplification scripts/lib/sandbox-init.sh, test/sandbox-init.test.ts	drop_capabilities() calls report_residual_capabilities directly without subsequent enforcement. report_residual_capabilities() no longer forces non-zero return when dangerous capabilities are detected. Tests remove capsh stubs and enforce-policy blocks, replacing complex opt-in paths with a fallthrough assertion.
Sandbox environment injection and allowlist documentation src/lib/onboard.ts, test/e2e-gateway-isolation.sh	createSandbox() removes NEMOCLAW_ALLOW_RESIDUAL_CAPS=1 injection, adds conditional TOOL_GATEWAY_USER_TOKEN when hermesToolBrokerToken is present, explicitly strips KUBECONFIG and SSH_AUTH_SOCK, and updates the non-root execution test to assert successful execution under the sandbox UID/GID.
Workflow environment variable and type contract updates .github/workflows/e2e-script.yaml, .github/workflows/nightly-e2e.yaml, test/helpers/e2e-workflow-contract.ts, test/e2e-script-workflow.test.ts	Removes NEMOCLAW_ALLOW_RESIDUAL_CAPS from workflow-level and job-level environment definitions and from env_json payloads. NightlyWorkflow and RunnerWorkflow type contracts drop the optional env field. Test assertion verifying the env variable was removed.
E2E test script security-posture environment cleanup test/e2e/test-full-e2e.sh, test/e2e/test-hermes-e2e.sh	Removes conditional NEMOCLAW_ALLOW_RESIDUAL_CAPS export blocks used under security-posture + non-root host modes, letting Phase 2 proceed without that override.
Documentation reference update docs/reference/commands.mdx	Removes NEMOCLAW_ALLOW_RESIDUAL_CAPS entry from the Onboarding Configuration environment variables table.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• NVIDIA/NemoClaw#4335: Adds NEMOCLAW_ALLOW_RESIDUAL_CAPS and related workflow contract tests; changes are tightly coupled.
• NVIDIA/NemoClaw#4151: Modifies reusable E2E workflow plumbing and contract expectations that this PR removes/updates.
• NVIDIA/NemoClaw#4266: Previously altered sandbox-init capability-dropping logic that this PR further simplifies.

Suggested labels

CI/CD, fix, Sandbox

Suggested reviewers

• ericksoa
• jyaunches

Poem

🐰 Away goes the cap-allowance flag,
A simpler sandbox starts to wag,
No more residual tricks to play,
Tool tokens light the safer way,
Less enforce, more transparency's sway! 🔐✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 60.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the main change: restoring residual capability warning behavior by reverting fail-closed behavior from prior PRs.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch revert-4266-4335

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

ESLint skipped: no ESLint configuration detected in root package.json. To enable, add eslint to devDependencies.

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-4341.docs.buildwithfern.com/nemoclaw

[github-actions]

E2E Advisor Recommendation

Required E2E: test-e2e-gateway-isolation, openclaw-onboard-security-posture-e2e, hermes-onboard-security-posture-e2e, cloud-onboard-e2e
Optional E2E: sandbox-operations-e2e, hermes-e2e

Dispatch hint: openclaw-onboard-security-posture-e2e,hermes-onboard-security-posture-e2e,cloud-onboard-e2e

Auto-dispatched E2E: openclaw-onboard-security-posture-e2e, hermes-onboard-security-posture-e2e, cloud-onboard-e2e via nightly-e2e.yaml at 34ea16fd029fdc491eec19c50af5d2a3940dbee4 — nightly run

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

• test-e2e-gateway-isolation (medium): Directly exercises the sandbox image security/isolation regression suite touched by this PR, including non-root entrypoint behavior and residual capability diagnostics after the sandbox-init policy change.
• openclaw-onboard-security-posture-e2e (high): Validates full OpenClaw non-interactive onboarding on a non-root/security-posture configuration after removal of NEMOCLAW_ALLOW_RESIDUAL_CAPS forwarding and workflow env opt-ins.
• hermes-onboard-security-posture-e2e (high): Validates the same sandbox-start/security-posture path for the Hermes agent, whose E2E script and nightly job env were changed in parallel with OpenClaw.
• cloud-onboard-e2e (high): Covers real installer/onboarding and sandbox creation through the reusable e2e-script workflow after changes to sandbox create env propagation and the workflow runner environment contract.

Optional E2E

• sandbox-operations-e2e (medium): Useful adjacent lifecycle coverage for create/list/status/logs/destroy behavior because sandbox startup semantics changed, but the security-posture onboard jobs are the more targeted blockers.
• hermes-e2e (high): General Hermes full-flow confidence if maintainers want coverage beyond the Hermes security-posture variant; optional because the required Hermes security-posture job already runs test/e2e/test-hermes-e2e.sh with stricter posture settings.

New E2E recommendations

• residual capability fallback on real CI hosts (medium): Existing gateway-isolation coverage is image/container-level and security-posture jobs validate successful onboarding, but there is no dedicated workflow-dispatchable live E2E that asserts the exact CAP_SETPCAP-unavailable residual-capability warning behavior across OpenShell sandbox startup without relying on removed fail-closed assertions.
  • Suggested test: Add a small nightly dispatch job that creates a sandbox on a host known to lack CAP_SETPCAP, captures startup logs, and asserts residual-capability diagnostics plus successful command execution under the intended policy.

Dispatch hint

• Workflow: nightly-e2e.yaml
• jobs input: openclaw-onboard-security-posture-e2e,hermes-onboard-security-posture-e2e,cloud-onboard-e2e

[github-actions]

E2E Scenario Advisor Recommendation

Required scenario E2E: None
Optional scenario E2E: None

Workflow run

Full scenario advisor summary

E2E Scenario Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required scenario E2E

• None. No scenario workflow, scenario metadata, scenario runtime, or validation-suite files changed.

Optional scenario E2E

• None.

Relevant changed files

• None.

[github-actions]

PR Review Advisor

Findings: 2 needs attention, 3 worth checking, 0 nice ideas
Since last review: 0 prior items resolved, 4 still apply, 0 new items found

Review findings

🛠️ Needs attention

• Fail closed when sandbox capability dropping fails (scripts/lib/sandbox-init.sh:226): The sandbox entrypoint now continues when `capsh` is unavailable or when `CAP_SETPCAP` is missing. `report_residual_capabilities()` still logs dangerous residual capabilities, but it no longer returns failure and no policy gate exits. That allows startup with capabilities such as `cap_sys_admin`, `cap_sys_ptrace`, `cap_net_raw`, `cap_dac_override`, or `cap_net_bind_service` still in the bounding set, even though nearby comments describe bounding-set dropping as part of the runtime security model.
  • Recommendation: Restore a fail-closed default when dangerous residual capabilities remain. If a degraded mode is required, keep it behind a narrow explicit operator opt-in with documented risk and evidence that retained capabilities cannot bypass the sandbox isolation model.
  • Evidence: `drop_capabilities()` falls through after `report_residual_capabilities` in the `CAP_SETPCAP`-missing path and after the `capsh not available` warning. The removed `enforce_residual_capability_policy()` was the only refusal/opt-in gate.
• Restore residual-capability negative coverage (test/sandbox-init.test.ts:376): The tests that exercised fail-closed behavior for missing `capsh`, missing `CAP_SETPCAP`, dangerous residual `CapBnd`, and explicit degraded-mode opt-in were removed or replaced with a test that asserts warning-only fallthrough. The E2E isolation test also removed the non-root residual-cap refusal case. This codifies the weaker security behavior without proving the degraded posture is safe.
  • Recommendation: Restore targeted negative/security tests for capability-drop failure and dangerous residual capabilities, or add equivalent runtime tests proving that all required sandbox invariants still hold when dangerous caps are retained.
  • Evidence: `test/sandbox-init.test.ts` now expects `FALLTHROUGH_OK` after `capsh not available`; `test/e2e-gateway-isolation.sh` removes the refusal test and now expects non-root execution without `NEMOCLAW_ALLOW_RESIDUAL_CAPS`.

🔎 Worth checking

• Source-of-truth review needed: scripts/lib/sandbox-init.sh residual-capability fallback: The advisor marked localized patch analysis as missing.
  • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Evidence: `drop_capabilities()` falls through on missing `capsh` or missing `CAP_SETPCAP`, while the explicit opt-in gate, refusal behavior, and related tests/docs are removed.
• Source-of-truth rationale is missing for the residual-capability fallback (scripts/lib/sandbox-init.sh:238): The PR handles the invalid state where the host/runtime cannot drop bounding-set capabilities by locally continuing with a warning. It does not establish why the source boundary cannot be fixed, what invariant remains safe with dangerous caps retained, what regression test proves this cannot regress, or when the workaround can be removed.
  • Recommendation: Prefer making the invalid capability state impossible at the runtime/source boundary. If warning-only fallback is required, document the source-fix constraint, safety argument, regression coverage, and removal condition in code or docs.
  • Evidence: `drop_capabilities()` accepts missing `capsh` or missing `CAP_SETPCAP` as a fallthrough path, while the explicit policy gate, opt-in mechanism, and related tests/docs are removed.
• Capability-drop documentation no longer matches warning-only behavior (docs/reference/commands.mdx:1260): User docs remove the explicit `NEMOCLAW_ALLOW_RESIDUAL_CAPS` degraded-mode variable without replacing it with guidance that startup may continue with dangerous residual capabilities. Internal comments still describe capability dropping as the expected sandbox runtime security model.
  • Recommendation: Either restore the fail-closed invariant in code or update internal and user-facing documentation to clearly state the warning-only behavior, affected hosts, risk, remediation steps, and removal plan.
  • Evidence: `docs/reference/commands.mdx` removes the `NEMOCLAW_ALLOW_RESIDUAL_CAPS` row. `scripts/lib/sandbox-init.sh` still says OpenShell cannot pass `--cap-drop=ALL`, so the entrypoint drops dangerous capabilities at startup using `capsh`.

🌱 Nice ideas

• None.

Since last review details

Current findings:

• Source-of-truth review needed: scripts/lib/sandbox-init.sh residual-capability fallback: The advisor marked localized patch analysis as missing.
  • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Evidence: `drop_capabilities()` falls through on missing `capsh` or missing `CAP_SETPCAP`, while the explicit opt-in gate, refusal behavior, and related tests/docs are removed.
• Fail closed when sandbox capability dropping fails (scripts/lib/sandbox-init.sh:226): The sandbox entrypoint now continues when `capsh` is unavailable or when `CAP_SETPCAP` is missing. `report_residual_capabilities()` still logs dangerous residual capabilities, but it no longer returns failure and no policy gate exits. That allows startup with capabilities such as `cap_sys_admin`, `cap_sys_ptrace`, `cap_net_raw`, `cap_dac_override`, or `cap_net_bind_service` still in the bounding set, even though nearby comments describe bounding-set dropping as part of the runtime security model.
  • Recommendation: Restore a fail-closed default when dangerous residual capabilities remain. If a degraded mode is required, keep it behind a narrow explicit operator opt-in with documented risk and evidence that retained capabilities cannot bypass the sandbox isolation model.
  • Evidence: `drop_capabilities()` falls through after `report_residual_capabilities` in the `CAP_SETPCAP`-missing path and after the `capsh not available` warning. The removed `enforce_residual_capability_policy()` was the only refusal/opt-in gate.
• Restore residual-capability negative coverage (test/sandbox-init.test.ts:376): The tests that exercised fail-closed behavior for missing `capsh`, missing `CAP_SETPCAP`, dangerous residual `CapBnd`, and explicit degraded-mode opt-in were removed or replaced with a test that asserts warning-only fallthrough. The E2E isolation test also removed the non-root residual-cap refusal case. This codifies the weaker security behavior without proving the degraded posture is safe.
  • Recommendation: Restore targeted negative/security tests for capability-drop failure and dangerous residual capabilities, or add equivalent runtime tests proving that all required sandbox invariants still hold when dangerous caps are retained.
  • Evidence: `test/sandbox-init.test.ts` now expects `FALLTHROUGH_OK` after `capsh not available`; `test/e2e-gateway-isolation.sh` removes the refusal test and now expects non-root execution without `NEMOCLAW_ALLOW_RESIDUAL_CAPS`.
• Source-of-truth rationale is missing for the residual-capability fallback (scripts/lib/sandbox-init.sh:238): The PR handles the invalid state where the host/runtime cannot drop bounding-set capabilities by locally continuing with a warning. It does not establish why the source boundary cannot be fixed, what invariant remains safe with dangerous caps retained, what regression test proves this cannot regress, or when the workaround can be removed.
  • Recommendation: Prefer making the invalid capability state impossible at the runtime/source boundary. If warning-only fallback is required, document the source-fix constraint, safety argument, regression coverage, and removal condition in code or docs.
  • Evidence: `drop_capabilities()` accepts missing `capsh` or missing `CAP_SETPCAP` as a fallthrough path, while the explicit policy gate, opt-in mechanism, and related tests/docs are removed.
• Capability-drop documentation no longer matches warning-only behavior (docs/reference/commands.mdx:1260): User docs remove the explicit `NEMOCLAW_ALLOW_RESIDUAL_CAPS` degraded-mode variable without replacing it with guidance that startup may continue with dangerous residual capabilities. Internal comments still describe capability dropping as the expected sandbox runtime security model.
  • Recommendation: Either restore the fail-closed invariant in code or update internal and user-facing documentation to clearly state the warning-only behavior, affected hosts, risk, remediation steps, and removal plan.
  • Evidence: `docs/reference/commands.mdx` removes the `NEMOCLAW_ALLOW_RESIDUAL_CAPS` row. `scripts/lib/sandbox-init.sh` still says OpenShell cannot pass `--cap-drop=ALL`, so the entrypoint drops dangerous capabilities at startup using `capsh`.

Workflow run details

This is an automated advisory review. A human maintainer must make the final merge decision.

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/onboard.ts`:
- Around line 3517-3523: Shorten the comment block describing the env var
allowlist in src/lib/onboard.ts by at least one line to satisfy the
onboard-entrypoint budget; condense sentences (for example combine the first two
lines into one and remove the blank line) while preserving the key details that
the allowlist is whitelist-style and that for the sandbox we must strip
KUBECONFIG and SSH_AUTH_SOCK so the sandbox cannot access the host Kubernetes
cluster or SSH agent.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: fdb131e3-6ce2-4fb5-ae70-8864e5b65306

📥 Commits

Reviewing files that changed from the base of the PR and between be403d3 and 6183451.

📒 Files selected for processing (11)

• .github/workflows/e2e-script.yaml
• .github/workflows/nightly-e2e.yaml
• docs/reference/commands.mdx
• scripts/lib/sandbox-init.sh
• src/lib/onboard.ts
• test/e2e-gateway-isolation.sh
• test/e2e-script-workflow.test.ts
• test/e2e/test-full-e2e.sh
• test/e2e/test-hermes-e2e.sh
• test/helpers/e2e-workflow-contract.ts
• test/sandbox-init.test.ts

💤 Files with no reviewable changes (6)

• test/e2e-script-workflow.test.ts
• test/helpers/e2e-workflow-contract.ts
• docs/reference/commands.mdx
• .github/workflows/e2e-script.yaml
• test/e2e/test-full-e2e.sh
• test/e2e/test-hermes-e2e.sh

[github-actions]

Selective E2E Results — ❌ Some jobs failed

Run: 26529688208
Target ref: 6183451839c5e5961b4e41f0050b7511bd05022c
Workflow ref: main
Requested jobs: openclaw-onboard-security-posture-e2e,hermes-onboard-security-posture-e2e
Summary: 1 passed, 1 failed, 0 skipped

Job	Result
hermes-onboard-security-posture-e2e	❌ failure
openclaw-onboard-security-posture-e2e	✅ success

Failed jobs: hermes-onboard-security-posture-e2e. Check run artifacts for logs.

[cjagwani]

LGTM

[github-actions]

Selective E2E Results — ✅ All requested jobs passed

Run: 26533090069
Target ref: cf491f19ad3d3b518879e41e7515dc3ddbf40c81
Workflow ref: main
Requested jobs: openclaw-onboard-security-posture-e2e,hermes-onboard-security-posture-e2e
Summary: 2 passed, 0 failed, 0 skipped

Job	Result
hermes-onboard-security-posture-e2e	✅ success
openclaw-onboard-security-posture-e2e	✅ success

[github-actions]

Selective E2E Results — ✅ All requested jobs passed

Run: 26533676771
Target ref: 34ea16fd029fdc491eec19c50af5d2a3940dbee4
Workflow ref: main
Requested jobs: openclaw-onboard-security-posture-e2e,hermes-onboard-security-posture-e2e,cloud-onboard-e2e
Summary: 3 passed, 0 failed, 0 skipped

Job	Result
cloud-onboard-e2e	✅ success
hermes-onboard-security-posture-e2e	✅ success
openclaw-onboard-security-posture-e2e	✅ success