Conversation
… block, rlm#70) Drop the rlm harness's `$HOME/.local/bin/git` refusal shim and the `allow_git` kwarg. Git access from inside the agent is now blocked at the rlm tool layer (PrimeIntellect-ai/rlm#70): shell-escape detection plus an AST walk over `subprocess.run` / `os.system` / `os.popen` for a literal first arg of `git`. Opt out with `RLM_ALLOW_GIT=1` via `ComposableEnv(environment_vars=...)`. Tool-level beats the shim: legitimate sandbox-side git (`pip install git+...`, build tools that internally shell out to git, language toolchains) keeps working, no filesystem mutation of agent-PATH binaries, and every harness/env using rlm gets the block for free. `Harness.post_install_uploads` and `Harness.post_install_script` stay as generic post-install extension points; their docstrings now say so instead of pointing at the rlm git-shim example. Supersedes #1225 / #1244 / #1254.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The rlm harness has been dropping a
gitrefusal shim into$HOME/.local/bin/gitso the agent'sPATHresolvesgitto the shim before/usr/bin/git. The shim's been through three iterations — added on a system PATH dir (#1225), relocated to$HOME/.local/binso scoring's real-gitcalls keep working (#1244), reworked into a rename-to-twit +pre_score_scriptrestoration design (#1254). Each iteration carried more sandbox-plumbing weight while still missing legitimate sandbox-side git usage that happens to traverse the agent's PATH (e.g. an agent-drivenpip install git+..., build tools / language toolchains that internally shell out to git).The block now lives at the rlm tool layer instead: https://github.com/PrimeIntellect-ai/rlm/pull/70.
!cmd,%%bash, bash tool invocation).subprocess.run/subprocess.Popen/os.system/os.popenfor a literal first arg ofgit.RLM_ALLOW_GIT=1(pass throughComposableEnv(environment_vars=...)for envs that genuinely need git inside the agent's tools).pygit2,dulwich,GitPythonAPI) and dynamic-attr / dynamic-import obfuscation. Both are accepted — the block targets the ~95% of incidentalgitinvocations, not adversarial circumvention.Tool-level wins because:
gitinvocation on the agent's PATH, including ones the agent legitimately wants to make:pip install git+https://...and build tools that internally callgit. With the rlm-side block, only directgitfrom the agent (shell escape / subprocess literal) is refused; anything called transitively throughpip/uv/build systems sees the realgit.$HOME/.local/bin/gitshim, no rename-to-twit, no pre-score restoration step.ComposableEnvto wire up.What this PR changes
verifiers/envs/experimental/composable/harnesses/rlm.py: drop_GIT_SHIM_BODY, drop theallow_gitkwarg, drop thepost_install_uploads/post_install_scriptwiring. Docstring now points callers atRLM_ALLOW_GIT=1for the opt-out path.verifiers/envs/experimental/composable/harness.py,composable_env.py: keeppost_install_uploads/post_install_scriptas generic extension points (they are dataclass fields and dispatchers, not git-specific). Docstrings reframed accordingly — they no longer name the rlm git shim as the example.tests/test_rlm_composable_env.py: droptest_rlm_harness_blocks_git_by_defaultandtest_rlm_harness_allow_git_uploads_nothing; replace with a single test asserting the harness leavespost_install_uploads/post_install_scriptasNone. The generic dispatcher tests (test_post_install_uploads_and_script_run_after_install,test_post_install_script_failure_raises) stay — they exercise theHarnessfield plumbing, not the rlm shim.docs/environments.md,environments/AGENTS.md,assets/lab/environments/AGENTS.md: drop "RLM's/usr/local/bin/gitrefusal shim" from theHarnessfield summary.Supersedes / closes
pre_score_scriptdesign is no longer needed; the rlm-side block fully replaces it.Test plan
uv run pytest tests/→ 1059 passed, 2 skipped (pre-existing OpenEnv smoke skips), 0 failed.uv run ruff check verifiers/ tests/→ clean.uv run ruff format --check verifiers/ tests/→ clean.Note
Medium Risk
Medium risk because it changes
rlm_harnessbehavior/API (removesallow_gitand stops installing a git-blocking shim), which may affect environments relying on the previous sandbox post-install hooks.Overview
Removes the RLM harness’s sandbox-side git refusal shim and its associated
allow_gitconfiguration, leavingrlm_harnessto no longer setpost_install_uploads/post_install_script.Updates
ComposableEnv/Harnessdocs and environment docs to describe post-install hooks as generic extension points (not RLM-specific), and adjusts tests to only assert that the RLM harness does not wire post-install hooks while keeping the generic post-install dispatcher coverage.Reviewed by Cursor Bugbot for commit 64c9591. Bugbot is set up for automated code reviews on this repo. Configure here.