test(e2e): add openclaw plugin EXDEV guard for #3513#3761
Merged
Conversation
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds a gated regression E2E job ( ChangesOpenClaw EXDEV Plugin Runtime-Deps Test
Sequence Diagram(s)sequenceDiagram
participant GitHubActions
participant Runner
participant TestScript as test-openclaw-plugin-runtime-exdev.sh
participant NemoClawSandbox as NemoClaw_Sandbox
participant OpenClawAgent as openclaw_agent
participant ArtifactUploader as Artifact_Uploader
GitHubActions->>Runner: trigger regression-e2e workflow
Runner->>TestScript: execute test-openclaw-plugin-runtime-exdev.sh
TestScript->>NemoClawSandbox: prepare sandbox & onboard (nemoclaw onboard --fresh)
TestScript->>NemoClawSandbox: run df to capture filesystem layout
TestScript->>OpenClawAgent: set TMPDIR=/dev/shm and start openclaw agent --agent main
OpenClawAgent-->>TestScript: stdout logs (PONG or errors)
Runner->>ArtifactUploader: upload /tmp/nemoclaw-e2e-openclaw-plugin-exdev-* on failure
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related issues
Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Contributor
E2E Advisor RecommendationRequired E2E: Dispatch hint: Full advisor summaryE2E Recommendation AdvisorBase: Required E2E
Optional E2E
New E2E recommendations
Dispatch hint
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/e2e/test-openclaw-plugin-runtime-exdev.sh`:
- Around line 144-148: The current success check greps for any "PONG" in
AGENT_LOG which can match the prompt and produce false positives; update the
conditional that references AGENT_LOG (the grep -qi 'PONG' check) to instead
search for the explicit response JSON field returned by the agent (e.g., a
pattern matching the response key and value like a JSON pair such as "response":
"PONG" or the exact response_token field your agent emits) so the test asserts
the agent returned the expected JSON response token rather than any occurrence
of PONG in the log.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 09946038-dffd-48f2-86b5-ee0248ed6a10
📒 Files selected for processing (2)
.github/workflows/regression-e2e.yamltest/e2e/test-openclaw-plugin-runtime-exdev.sh
Contributor
There was a problem hiding this comment.
♻️ Duplicate comments (1)
test/e2e/test-openclaw-plugin-runtime-exdev.sh (1)
148-149:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winTighten the success assertion to avoid false positives.
On Line 148,
grep -qi 'PONG'can match the prompt text and pass even when the agent response token is missing. Match a response JSON field/value pair instead.Suggested patch
-if grep -qi 'PONG' "$AGENT_LOG"; then +if grep -qiE '"(content|text|response)"[[:space:]]*:[[:space:]]*"PONG"' "$AGENT_LOG"; then pass "openclaw agent completed without plugin runtime-deps EXDEV despite cross-device staging" else fail "openclaw agent exited 0 but expected response token was missing; see ${AGENT_LOG}"🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@test/e2e/test-openclaw-plugin-runtime-exdev.sh` around lines 148 - 149, The current grep check against AGENT_LOG uses a loose 'PONG' match which can hit unrelated prompt text; replace it with a JSON field/value match so the success assertion only passes when the agent returned the expected response (e.g., look for a JSON pair like "response":"PONG" or similar in AGENT_LOG) and update the conditional that calls pass(...) accordingly; target the grep invocation that currently searches for 'PONG' and change the pattern to a JSON-aware regex (matching the exact field name used by the agent response) so false positives from prompt text are avoided.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Duplicate comments:
In `@test/e2e/test-openclaw-plugin-runtime-exdev.sh`:
- Around line 148-149: The current grep check against AGENT_LOG uses a loose
'PONG' match which can hit unrelated prompt text; replace it with a JSON
field/value match so the success assertion only passes when the agent returned
the expected response (e.g., look for a JSON pair like "response":"PONG" or
similar in AGENT_LOG) and update the conditional that calls pass(...)
accordingly; target the grep invocation that currently searches for 'PONG' and
change the pattern to a JSON-aware regex (matching the exact field name used by
the agent response) so false positives from prompt text are avoided.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 0c10b753-290e-49e1-91f0-69c05691b27e
📒 Files selected for processing (1)
test/e2e/test-openclaw-plugin-runtime-exdev.sh
…runtime-exdev-e2e-guard
cv
approved these changes
May 19, 2026
# Conflicts: # test/e2e/docs/parity-inventory.generated.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a failing-test-first regression guard for #3513 / #3127 in
regression-e2e.yaml.The new
openclaw-plugin-runtime-exdev-e2ejob provisions a fresh sandbox and runs the first in-sandboxopenclaw agentinvocation. On unfixed code, the expected failure is OpenClaw plugin runtime dependency installation aborting withEXDEV: cross-device link not permitted/failed to install bundled runtime deps/PluginLoadFailureError.Expected state before the fix
This PR is intentionally expected to fail when the job is manually dispatched on main-equivalent code. The failure is the executable acceptance criterion for #3513.
Expected RED fragment:
Expected GREEN after the fix:
openclaw agent --agent main --json ...exits 0 and returns a response without plugin runtime-deps EXDEV errors.Workflow placement
.github/workflows/regression-e2e.yamlopenclaw-plugin-runtime-exdev-e2eDispatch:
Related: #3513
Also related: #3127
Summary by CodeRabbit
Tests
Chores