feat(playground): list the environment's tools in the Tools rail - #3622
Conversation
The Tools rail reads the browser's own MCP connections, which environment mode never creates (the backend connects per turn) — so it said "No tools found" while tools visibly executed in chat. New GET /api/web/environments/:environmentId/tools resolves the environment atomically (same read as /preview), opens the same ephemeral authorized connections a chat turn uses — the only path that can reach plugin-contributed servers — lists each server's tools, and disconnects. Per-server failures degrade to per-server rows, so one unreachable server doesn't blank the panel, and the error doubles as a pre-turn health check. In the rail, environment mode swaps in a read-only EnvironmentToolsPane fed by that route: flat searchable list, server badge on colliding names, expandable description + input schema, and the same "connects automatically on every message" copy as the composer menu. Selection comes from the same persisted previewed-environment id the Playground itself uses, fail-closed behind the project-environments flag. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
GitHub Actions
Inactive
dosubot
Bot
added
size:XL
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_7d979945-e554-43eb-9fb9-eae05d56deb0) |
📄 Knowledge reviewDosu skipped reviewing this PR because your organization has used its
|
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2f08e2dfe7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const spec = await resolveEnvironmentForRuntime( | ||
| createConvexClient(bearerToken), | ||
| { projectId, environmentId } | ||
| ); |
There was a problem hiding this comment.
When the user disables a server or pinned plugin in the environment picker, PlaygroundMain.tsx sends those selections as environmentOverrides for subsequent chat turns, but this endpoint always resolves the unmodified environment and the client tools request supplies only the project/environment IDs. The rail therefore continues displaying tools from servers/plugins that the next message cannot use. Pass the current override state through to this request and resolve the same narrowed environment as chat.
Useful? React with 👍 / 👎.
|
Warning Review limit reached
Next review available in: 24 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
WalkthroughThe change adds a server endpoint that lists tools for environment servers. A client hook fetches and validates the results, tracks per-server and route errors, and supports refreshes. The playground passes project scope and renders environment tools when environment mode is active. The new pane supports search, expansion, duplicate-name badges, read-only controls, loading, empty, error, and no-match states. Tests cover the route, hook wiring, and pane states. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
mcpjam-inspector/client/src/hooks/use-environment-tools.ts (1)
43-143: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winRace-prevention logic is correct; consider direct hook tests.
The
requestSeqRefandactiveflag correctly prevent a slow response for a stale environment/project pair from overwriting the current one, andmatchesTargetblanks state for a new target before the effect even runs. This logic works as intended.No test file exercises this hook directly;
EnvironmentToolsPane.test.tsxmocks it entirely. Add a dedicated test foruseEnvironmentToolsthat verifies an out-of-order response (environment A resolves after environment B was selected) does not overwrite B's state.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@mcpjam-inspector/client/src/hooks/use-environment-tools.ts` around lines 43 - 143, Add a dedicated direct test for useEnvironmentTools that switches from environment A to B, resolves B first, then resolves A, and verifies the hook retains B’s servers without stale A data or error. Use a hook-rendering test harness and mock authFetch responses to exercise requestSeqRef and active behavior; keep the existing EnvironmentToolsPane tests unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@mcpjam-inspector/client/src/components/playground/panes/EnvironmentToolsPane.tsx`:
- Around line 227-246: Update the div rendered by ServerErrorRow within
TooltipTrigger to include tabIndex={0}, and ensure the existing Tooltip behavior
opens on keyboard focus so keyboard users can access server.error.
---
Nitpick comments:
In `@mcpjam-inspector/client/src/hooks/use-environment-tools.ts`:
- Around line 43-143: Add a dedicated direct test for useEnvironmentTools that
switches from environment A to B, resolves B first, then resolves A, and
verifies the hook retains B’s servers without stale A data or error. Use a
hook-rendering test harness and mock authFetch responses to exercise
requestSeqRef and active behavior; keep the existing EnvironmentToolsPane tests
unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 57cc4313-fe67-4b8e-9365-c81415b5d003
📒 Files selected for processing (7)
mcpjam-inspector/client/src/components/playground/PlaygroundLeftRail.tsxmcpjam-inspector/client/src/components/playground/PlaygroundTab.tsxmcpjam-inspector/client/src/components/playground/__tests__/EnvironmentToolsPane.test.tsxmcpjam-inspector/client/src/components/playground/panes/EnvironmentToolsPane.tsxmcpjam-inspector/client/src/hooks/use-environment-tools.tsmcpjam-inspector/server/routes/web/__tests__/environments-tools.test.tsmcpjam-inspector/server/routes/web/environments.ts
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
1 issue found across 7 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="mcpjam-inspector/server/routes/web/__tests__/environments-tools.test.ts">
<violation number="1" location="mcpjam-inspector/server/routes/web/__tests__/environments-tools.test.ts:119">
P3: The xaaPolicy/xaaIssuer options the route derives from the environment's own host config are never asserted, so the stated "same enterprise-policy posture as chat-v2" behavior isn't pinned. Consider asserting `expect(createAuthorizedManagerMock.mock.calls[0][7]).toMatchObject({ xaaPolicy: expect.anything(), xaaIssuer: expect.any(String) })` so a regression dropping the policy from the manager build is caught.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| "ps_2", | ||
| ]); | ||
| expect(createAuthorizedManagerMock.mock.calls[0][7]).toMatchObject({ | ||
| serverNames: ["linear", "asana"], |
There was a problem hiding this comment.
P3: The xaaPolicy/xaaIssuer options the route derives from the environment's own host config are never asserted, so the stated "same enterprise-policy posture as chat-v2" behavior isn't pinned. Consider asserting expect(createAuthorizedManagerMock.mock.calls[0][7]).toMatchObject({ xaaPolicy: expect.anything(), xaaIssuer: expect.any(String) }) so a regression dropping the policy from the manager build is caught.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At mcpjam-inspector/server/routes/web/__tests__/environments-tools.test.ts, line 119:
<comment>The xaaPolicy/xaaIssuer options the route derives from the environment's own host config are never asserted, so the stated "same enterprise-policy posture as chat-v2" behavior isn't pinned. Consider asserting `expect(createAuthorizedManagerMock.mock.calls[0][7]).toMatchObject({ xaaPolicy: expect.anything(), xaaIssuer: expect.any(String) })` so a regression dropping the policy from the manager build is caught.</comment>
<file context>
@@ -0,0 +1,199 @@
+ "ps_2",
+ ]);
+ expect(createAuthorizedManagerMock.mock.calls[0][7]).toMatchObject({
+ serverNames: ["linear", "asana"],
+ });
+ const body = await response.json();
</file context>
Review findings: - cubic P1: createAuthorizedManager validates its batch all-or-nothing, so one server's authorization/XAA failure blanked the whole rail. Build one manager per server; an auth failure now degrades to that server's row like every other per-server failure. xaaPolicy stays route-level (a malformed policy is an environment problem, not one server's). - cubic P2: the hook's error parser dropped a top-level message field — probe it before the generic fallback, same order as use-environment-preview. - cubic P3 x2: tests now pin cacheMode: "bypass" and the xaaIssuer option, plus a new test for the authorization-failure row. - a11y (coderabbit/cubic): error-row tooltip trigger is keyboard-focusable; tool rows expose aria-expanded. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_1da40027-e4f2-459b-b2a0-3bedbd0d9929) |
|
Review triage — all findings addressed in the latest commit except one declined by design: Fixed:
Declined (by design) — Codex "apply Playground overrides to the tools request": the rail deliberately shows the environment's OWN resolved set, mirroring the |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4370cdbe5e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| WEB_CALL_TIMEOUT_MS, | ||
| undefined, | ||
| undefined, | ||
| { | ||
| ...(serverNames.length > 0 ? { serverNames: [name] } : {}), | ||
| xaaPolicy, | ||
| xaaIssuer, |
There was a problem hiding this comment.
When an environment's host pins clientInfo, supportedProtocolVersions, or a host/per-server MCP protocol version, this manager is created with clientCapabilities as undefined and without initializePins or mcpProtocolVersionsByServerId. The actual environment turn passes those settings in chat-v2.ts when calling createAuthorizedManager, so a server that requires the pinned wire version or client identity can fail here—or expose a different tool surface—even though chatting connects successfully. Derive and pass the resolved host's connection settings so this health/listing request uses the same handshake as the turn.
Useful? React with 👍 / 👎.
| if (environmentId && projectId) { | ||
| return ( | ||
| <EnvironmentToolsPane | ||
| projectId={projectId} | ||
| environmentId={environmentId} | ||
| /> |
There was a problem hiding this comment.
When the selected environment resolves to a harness host such as Claude Code, this early return discards the harnessBuiltinTools resolved immediately above. Those tools remain executable by the environment's chat turn and were previously passed to both normal Tools panes, but selecting an environment now removes them from the rail entirely. Include the harness built-ins in the environment pane rather than replacing the whole tool surface with server-listed tools only.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
All reported issues were addressed across 4 files (changes from recent commits).
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
…sful listing cubic P2: disconnectAllServers() rejecting inside finally replaced the already-collected tools with the thrown error, which the per-server catch then reported as an unreachable server. A transient close failure would show a healthy server as failed and discard tools already in hand. Teardown failures are now swallowed to one warn — the connection is ephemeral and the request is over either way. Regression test verified to fail without the fix. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
cubic P2 (teardown rejection erases a successful listing) — valid, fixed. Teardown failures are now caught and logged as a single warn rather than propagating — the connection is ephemeral and the request is over either way, so a close failure has no bearing on whether the listing succeeded. Added a regression test ( |
GitHub Actions
Error
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_2ec899a5-744f-4208-b4af-bb73561f3b08) |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e8aa2fbac6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| refreshToken, | ||
| ]); | ||
|
|
||
| const refresh = useCallback(() => setRefreshToken((n) => n + 1), []); |
There was a problem hiding this comment.
If the user clicks Refresh repeatedly while a listing is still loading, every click increments refreshToken and starts another request; the effect cleanup only prevents stale state commits and does not cancel the prior authFetch. Because the server handler opens a live authorized MCP manager for every environment server (server/routes/web/environments.ts), this can create many concurrent external connections and authorization requests from a single pane. Make refresh single-flight, disable it while loading, or abort the previous request.
Useful? React with 👍 / 👎.
1 participant
Why
In environment mode the backend connects the environment's servers per turn — the browser never holds a connection. The Tools rail reads only browser connections, so it showed "No tools found" while tools visibly executed in chat. Fast-follow to #3617 (which fixed the composer's server menu).
What
Server —
GET /api/web/environments/:environmentId/tools?projectId=…(sibling of/preview): resolves the environment atomically, builds the same authorized ephemeral manager a chat turn uses (createAuthorizedManagerover the resolved server ids — the only path that reaches plugin-contributed servers), lists each server's tools withcacheMode: "bypass", disconnects infinally. Enterprise XAA policy comes from the environment's own resolved host config, mirroring chat-v2. Per-server failures return as per-servererrorrows — one dead server doesn't blank the panel, and the row doubles as a pre-turn health check.Client —
useEnvironmentTools(same target-keyed commit discipline asuse-environment-preview: switches blank synchronously, stale responses can't land, live edits refetch via the row'srevision) + a read-onlyEnvironmentToolsPane: flat searchable tool list, server badge on colliding names, expandable description + input schema, per-server failure rows, and the same "connects automatically on every message" copy as the composer menu.ToolsBodyswaps it in when an environment is selected, reading the SAME persisted previewed-environment id the Playground uses, fail-closed behindproject-environments-enabled.Deliberately read-only in v1: execution in environment mode goes through chat, and the rail's run machinery targets browser connections these servers don't have.
Tests
environments-tools.test.ts): projectId/bearer required, resolved ids + names forwarded to the manager builder, per-server failure isolation + teardown, zero-server short-circuit without connecting, ENV_* resolve failure → 409.🤖 Generated with Claude Code
Note
Medium Risk
The new route opens real per-server MCP connections with the same auth/XAA path as chat turns; behavior is well-tested but it touches connection teardown and enterprise policy handling.
Overview
Fixes the Playground Tools rail showing “No tools found” in environment mode, where the backend connects MCP servers per message and the browser never holds those connections.
Adds
GET /api/web/environments/:environmentId/tools: resolves the environment like preview, then opens one ephemeral authorized manager per server (so plugin servers and per-server auth/XAA failures don’t blank siblings), lists tools withcacheMode: "bypass", and tears down connections; failures surface as per-server error rows on an otherwise 200 response.On the client,
useEnvironmentToolsfetches that route with the same target-key / revision discipline as environment preview.EnvironmentToolsPaneis a read-only searchable list (schema on expand, server badges on name collisions, refresh).ToolsBodyswaps it in when a previewed environment is selected (same persisted id and feature flag as the composer), withprojectIdpassed fromPlaygroundTab.Reviewed by Cursor Bugbot for commit e8aa2fb. Bugbot is set up for automated code reviews on this repo. Configure here.
Summary by cubic
Show the environment’s tools in the Playground Tools rail when an environment is selected. Fixes the “No tools found” mismatch by listing tools over the same per-turn connections chat uses, with per-server isolation, clearer errors, and teardown failures that no longer blank the panel.
GET /api/web/environments/:environmentId/tools?projectId=…: resolves the environment, builds one authorized manager per server id, lists each server’s tools withcacheMode: "bypass", surfaces per-server errors, then disconnects; teardown failures are warned and ignored; XAA issuer/policy mirror chat.EnvironmentToolsPane, fail-closed behindproject-environments-enabled.EnvironmentToolsPane: flat searchable list, server badges on colliding names, expandable description and input schema, per-server failure rows, keyboard-focusable error tooltips,aria-expandedon rows, and “connects automatically on every message” guidance.useEnvironmentToolshook: target-keyed commits prevent stale responses on environment switches; refetches on environmentrevision; parses route errors from a top-levelmessage.Written for commit e8aa2fb. Summary will update on new commits.