Skip to content

fix(runtime): use prependSystemContext to prevent runtime instructions leaking into chat UI#4037

Merged
cv merged 1 commit into
NVIDIA:mainfrom
kagura-agent:fix/runtime-context-system-prompt
Jun 5, 2026
Merged

fix(runtime): use prependSystemContext to prevent runtime instructions leaking into chat UI#4037
cv merged 1 commit into
NVIDIA:mainfrom
kagura-agent:fix/runtime-context-system-prompt

Conversation

@kagura-agent
Copy link
Copy Markdown
Contributor

@kagura-agent kagura-agent commented May 22, 2026

Summary

Fixes #4019 — system runtime instructions (<nemoclaw-runtime> block) leaking into the chat UI on the third message.

Root Cause

registerRuntimeContext() was using prependContext in the before_prompt_build hook return. In the OpenClaw host, prependContext prepends content to the user-visible conversation prompt, which means the sandbox policy instructions could appear in the chat UI — especially with models like Nemotron 3 Super 120B via Ollama that may echo prepended context.

Fix

Switch from prependContext to prependSystemContext, which injects into the system prompt (invisible to users). This matches the intent of these instructions as system-level runtime context that should never be displayed to the user.

Changes

  • nemoclaw/src/runtime-context.ts: prependContextprependSystemContext (1 line)
  • nemoclaw/src/runtime-context.test.ts: Updated test expectations to match (3 test assertions)

Testing

npx vitest run nemoclaw/src/runtime-context.test.ts
# ✓ 6 tests passed (runtime-context.test.ts)

Summary by CodeRabbit

  • Refactor

    • Adjusted how the runtime summary is injected into system prompts so agent turns consistently include the runtime wrapper and sandbox/network/filesystem policy text, and persisted sandbox names are respected.
  • Tests

    • Updated tests to validate the new prompt-injection behavior and the presence of the runtime wrapper and policy content in system prompts.

Signed-off-by: kagura-agent kagura.agent.ai@gmail.com

@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot Bot commented May 22, 2026

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 22, 2026

Review Change Stack

📝 Walkthrough

Walkthrough

The PR renames the hook response field carrying the runtime summary from prependContext to prependSystemContext and updates tests to assert the new field contains the runtime wrapper tags and sandbox policy text (including persisted sandbox name).

Changes

Runtime Context Field Migration

Layer / File(s) Summary
Hook response field rename and test validation
nemoclaw/src/runtime-context.ts, nemoclaw/src/runtime-context.test.ts
registerRuntimeContext's before_prompt_build hook now returns the generated runtime summary under prependSystemContext instead of prependContext. Tests for default and persisted sandbox configurations updated to assert prependSystemContext contains the <nemoclaw-runtime> wrapper and sandbox/network/filesystem policy text.

Estimated Code Review Effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐰 A tiny change, a clearer name,
prependSystemContext stakes its claim.
Sandbox rules wrapped neat and tight,
Hidden from chat, shown where it's right.
Hop, review done — all set for flight!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The PR title accurately describes the main change: switching from prependContext to prependSystemContext to prevent runtime instructions from appearing in the chat UI.
Linked Issues check ✅ Passed The code changes directly address the primary objective from #4019: preventing system instructions from leaking into the chat UI by using prependSystemContext instead of prependContext.
Out of Scope Changes check ✅ Passed All changes are scoped to the runtime context functionality, with modifications limited to the hook field name and corresponding test expectations—no unrelated changes detected.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@wscurran
Copy link
Copy Markdown
Contributor

✨ Thanks for submitting this detailed PR about preventing system runtime instructions from leaking into the chat UI. This proposes a way to improve the security and usability of NemoClaw by ensuring that system-level runtime context is not displayed to the user.


Related open issues:

@kagura-agent
Copy link
Copy Markdown
Contributor Author

Hi 👋 Just checking in — this PR has been open for a week. Let me know if there are any concerns with the approach or if you'd like me to adjust anything. Happy to iterate!

@kagura-agent
Copy link
Copy Markdown
Contributor Author

Friendly ping — this has been open for 10 days. The fix uses prependSystemContext to prevent runtime instructions from leaking into the chat UI. Happy to adjust the approach if needed. 🙏

@kagura-agent
Copy link
Copy Markdown
Contributor Author

Friendly ping — this PR has been open for 11 days. Happy to address any feedback. 🙏

@wscurran wscurran added area: policy Network policy, egress rules, presets, or sandbox policy bug-fix PR fixes a bug or regression feature PR adds or expands user-visible functionality labels Jun 3, 2026
@kagura-agent
Copy link
Copy Markdown
Contributor Author

Friendly ping — this PR has been open for 12 days. Happy to adjust the approach if needed. 🙏

@wscurran wscurran removed fix labels Jun 3, 2026
@cv cv added the v0.0.59 Release target label Jun 4, 2026
@cv cv reopened this Jun 4, 2026
@cv cv enabled auto-merge (squash) June 4, 2026 01:16
@kagura-agent
Copy link
Copy Markdown
Contributor Author

Disregard my previous closing comment — I see @cv approved this. Thanks for the review! Ready to merge whenever convenient. 🙏

@cv
Copy link
Copy Markdown
Collaborator

cv commented Jun 4, 2026

@kagura-agent can you add a DCO to the PR description, please?

auto-merge was automatically disabled June 4, 2026 02:09

Head branch was pushed to by a user without write access

@kagura-agent kagura-agent force-pushed the fix/runtime-context-system-prompt branch from 0c1a648 to 4b4d4fb Compare June 4, 2026 02:09
@kagura-agent
Copy link
Copy Markdown
Contributor Author

Done — rebased on latest main and added DCO signoff. Thanks for the heads-up! 🙏

@cv
Copy link
Copy Markdown
Collaborator

cv commented Jun 4, 2026

@kagura-agent I don't see the DCO Signed-off-by in the PR description, mind adding it?

@cv cv added v0.0.60 Release target and removed v0.0.59 Release target labels Jun 4, 2026
…s leaking into chat UI

The <nemoclaw-runtime> context block was injected via prependContext, which
prepends to the user-visible conversation prompt. On certain models
(e.g. Nemotron 3 Super 120B via Ollama), this caused the sandbox policy
instructions to appear in the chat UI on the third message.

Switch to prependSystemContext, which injects into the system prompt
(invisible to users), matching the intent of these instructions as
system-level runtime context.

Fixes NVIDIA#4019

Signed-off-by: kagura-agent <kagura.agent.ai@gmail.com>
@kagura-agent kagura-agent force-pushed the fix/runtime-context-system-prompt branch from be124be to de3a7e2 Compare June 4, 2026 22:06
@kagura-agent
Copy link
Copy Markdown
Contributor Author

Done — added the DCO Signed-off-by. Thanks for the reminder! 🙏

@kagura-agent
Copy link
Copy Markdown
Contributor Author

Hi @cv — thanks for the approval! DCO signoff is now in place. Would you be able to merge this when convenient? 🙏

@cv
Copy link
Copy Markdown
Collaborator

cv commented Jun 5, 2026

@kagura-agent you must add a string like "Signed-off-by Your Name your.email@example.com" to the PR description, not to the commits.

@kagura-agent
Copy link
Copy Markdown
Contributor Author

Thanks @cv — added the Signed-off-by line to the PR description. Should be good now! 🙏

@cv cv merged commit 5dac380 into NVIDIA:main Jun 5, 2026
21 of 24 checks passed
miyoungc added a commit that referenced this pull request Jun 6, 2026
## Summary
- Adds the `v0.0.60` section to `docs/about/release-notes.mdx` using the
dev announcement from discussion #4877.
- Fills the source-doc gaps found during release-prep review across
inference, policy tiers, command behavior, security boundaries, Hermes
dashboard/tooling, runtime context, and troubleshooting.
- Refreshes generated agent skills under `.agents/skills/` from the
current Fern docs output and upgrades Fern from `5.44.3` to `5.45.0`.

## Source summary
- #4037 -> `docs/reference/architecture.mdx`,
`docs/about/how-it-works.mdx`, `docs/about/release-notes.mdx`: Documents
system-only runtime context that stays out of visible chat.
- #4875 -> `docs/reference/architecture.mdx`,
`docs/about/how-it-works.mdx`, `docs/about/release-notes.mdx`: Documents
try-first sandbox network/filesystem guidance and clearer failure
classification.
- #4788 -> `docs/security/best-practices.mdx`,
`docs/about/release-notes.mdx`: Documents shared OpenClaw
device-approval policy for startup and connect.
- #4768 -> `docs/reference/network-policies.mdx`,
`docs/network-policy/integration-policy-examples.mdx`,
`docs/get-started/quickstart.mdx`,
`docs/get-started/quickstart-hermes.mdx`, `docs/reference/commands.mdx`:
Documents `weather`, `public-reference`, and Hermes managed-tool gateway
preset behavior.
- #3788 and #4864 -> `docs/reference/network-policies.mdx`,
`docs/reference/commands.mdx`: Documents non-interactive policy-tier
fail-fast behavior and interactive prompt fallback.
- #4756 and #4866 -> `docs/reference/commands.mdx`: Documents env-aware
default sandbox resolution for `list`, `status`, and `tunnel` commands.
- #4320 -> `docs/reference/commands.mdx`: Documents `$$nemoclaw tunnel
status` behavior.
- #4328 -> `docs/reference/commands.mdx`: Documents line-scoped policy
preset descriptions in `policy-list`.
- #4580 and #4748 -> `docs/reference/architecture.mdx`: Documents
package-managed OpenShell gateway service and Docker-driver
gateway-marker behavior.
- #4598 -> `docs/manage-sandboxes/lifecycle.mdx`: Documents concurrent
gateway/dashboard cleanup isolation by sandbox name and port.
- #4777 -> `docs/reference/troubleshooting.mdx`: Documents Docker GPU
patch rollback behavior.
- #4610 -> `docs/reference/troubleshooting.mdx`,
`docs/reference/commands.mdx`: Keeps mutable OpenClaw config permission
guidance aligned and removes skipped experimental wording.
- #4868 -> `docs/reference/commands.mdx`: Keeps `.dockerignore` handling
for custom `onboard --from <Dockerfile>` contexts in generated skills.
- #4870 -> `docs/reference/commands.mdx`,
`docs/manage-sandboxes/runtime-controls.mdx`: Documents
`NEMOCLAW_MINIMAL_BOOTSTRAP` and generated skill coverage.
- #4641 -> `docs/inference/inference-options.mdx`,
`docs/reference/troubleshooting.mdx`: Documents local NVIDIA NIM
platform-digest pulls and served-model id adoption.
- #4810 and #4867 -> `docs/inference/inference-options.mdx`: Documents
stable NGC managed-vLLM image lineage and DGX Station DeepSeek V4 Flash
coverage.
- #4852 -> `docs/inference/use-local-inference.mdx`,
`docs/reference/troubleshooting.mdx`: Documents Ollama model fit
filtering, 16K context floor, cold-load retry, and failed-model
exclusion.
- #4847 -> `docs/inference/switch-inference-providers.mdx`: Documents
API-family sync, Hermes `api_mode`, and Bedrock Runtime exception.
- #4800 -> `docs/inference/tool-calling-reliability.mdx`: Documents
Nemotron managed-inference native tool-search fallback.
- #4333 -> `docs/inference/switch-inference-providers.mdx`: Documents
interactive multimodal input prompting.
- #4086 -> `docs/reference/troubleshooting.mdx`: Keeps proxy bypass
normalization in generated troubleshooting coverage.
- #4811 and #4855 -> `docs/get-started/quickstart-hermes.mdx`: Documents
prebuilt Hermes dashboard assets and TUI recovery without runtime
rebuilds.
- #4854 -> `docs/inference/switch-inference-providers.mdx`,
`docs/reference/commands.mdx`: Documents Hermes proxy API-key
placeholder preservation during inference switches.
- #4248 -> `docs/manage-sandboxes/messaging-channels.mdx`,
`.agents/skills/`: Keeps messaging enrollment behavior aligned with
manifest-hook implementation.
- #4771 -> `docs/security/best-practices.mdx`,
`docs/security/credential-storage.mdx`: Documents Hermes
placeholder-only secret boundary for sandbox-visible runtime files.
- #4787 -> `docs/security/best-practices.mdx`,
`docs/about/release-notes.mdx`: Documents expanded memory scanner
examples for OpenAI project keys and Slack app-level tokens.
- #4848 -> `docs/reference/commands.mdx`: Documents OpenClaw skill
install mirroring into the agent home directory.
- #4790 -> `docs/about/release-notes.mdx`: Uses the prior release-prep
structure and generated `.agents/skills/` refresh as the template for
this release.

## Verification
- `python3 scripts/docs-to-skills.py docs/ .agents/skills/ --prefix
nemoclaw-user --doc-platform fern-mdx`
- `python3 scripts/docs-to-skills.py docs/ .agents/skills/ skills/
--prefix nemoclaw-user --doc-platform fern-mdx --dry-run`
- `npm run docs`
- `git diff --check`
- skip-term scan across `docs/`, `.agents/skills/`, and `skills/`
- `npm run build:cli`
- `npm run typecheck:cli`
- Commit and pre-push hook suites, including markdownlint, gitleaks,
env-var docs gate, docs-to-skills verification, and skills YAML tests

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

## Release Notes

* **New Features**
* DeepSeek-V4-Flash now available as default inference model for DGX
Station.
* Hermes dashboard improved with dedicated port and OAuth-authenticated
tool gateway selection.
* Added weather and public-reference policy presets for expanded agent
capabilities.
* Enhanced Ollama model selection with GPU memory filtering and
automatic retry for timeouts.

* **Bug Fixes**
  * Improved policy tier validation to prevent invalid configurations.
* Better sandbox cleanup scoping by port to prevent conflicts across
deployments.
  * Added GPU patch failure recovery with automatic rollback.

* **Documentation**
* Expanded troubleshooting guides for inference, security, and sandbox
lifecycle.
  * Added .dockerignore best practices for custom deployments.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Carlos Villela <cvillela@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: policy Network policy, egress rules, presets, or sandbox policy bug-fix PR fixes a bug or regression feature PR adds or expands user-visible functionality v0.0.60 Release target

Projects

None yet

Development

Successfully merging this pull request may close these issues.

System runtime instructions leaking into chat UI on third message

3 participants