Skip to content

Add MCP elicitation support for URL-mode auth flows#39

Draft
teallarson wants to merge 14 commits intomainfrom
feat/mcp-elicitation
Draft

Add MCP elicitation support for URL-mode auth flows#39
teallarson wants to merge 14 commits intomainfrom
feat/mcp-elicitation

Conversation

@teallarson
Copy link
Copy Markdown
Collaborator

Summary

  • Adds MCP URL-mode elicitation support to all three agent templates (ai-sdk, mastra, langchain), enabling the Arcade Gateway to request OAuth authorization via the elicitation/create protocol instead of relying solely on tool output parsing
  • Each template uses its framework's native elicitation API: @ai-sdk/mcp onElicitationRequest, @mastra/mcp MCPClient.elicitation.onRequest, and Python mcp SDK Callbacks(on_elicitation=...)
  • Explicitly declares { elicitation: { url: {} } } capability so servers know the client supports URL-mode (required by the MCP spec for sensitive flows like OAuth)
  • The existing extractAuthUrlFromToolOutput fallback is preserved for gateways that don't support elicitation

Changes

Frontend (shared)

  • types/inbox.ts — new elicitation event type in PlanEvent union
  • hooks/use-plan-stream.tsonElicitation callback in PlanStreamCallbacks
  • components/dashboard/auth-prompt.tsx — "Waiting for authorization..." state after clicking Authorize
  • app/dashboard/page.tsx — wires onElicitation to addAuthUrl

Backend (shared partials)

  • arcade-oauth-provider.hbs — elicitation capability declaration + handlers for both AI SDK and Mastra paths
  • plan-route-body.hbs — conditional Mastra (Agent.stream) vs AI SDK (streamText) implementations with elicitation callback wiring
  • extract-auth-url.hbs — marked as fallback

Template-specific

  • mastra/app/api/plan/route.ts.hbs — updated imports for Mastra-native MCP client
  • langchain/app/arcade_oauth.py — elicitation callback bridge + _on_elicitation handler
  • langchain/app/routes/plan.py — drains elicitation events into NDJSON stream

Dependency bumps

  • @ai-sdk/mcp ^1.0.36, @modelcontextprotocol/sdk ^1.27.1
  • @mastra/core ^1.24.1, @mastra/mcp ^1.4.2, @mastra/ai-sdk ^1.3.3
  • langchain-mcp-adapters >=0.2.2, mcp >=1.27

Test plan

  • Scaffold each template (ai-sdk, mastra, langchain) and verify it builds/type-checks
  • Run with a gateway that has tools requiring OAuth — confirm elicitation event appears in NDJSON stream and auth prompt renders
  • Verify fallback extractAuthUrlFromToolOutput still works when elicitation is not available
  • Smoke test: complete an OAuth flow end-to-end (click Authorize, complete auth, tool call resumes)

Made with Cursor

Enables all three agent templates (ai-sdk, mastra, langchain) to handle
MCP elicitation requests from the Arcade Gateway. When a tool requires
OAuth authorization, the gateway sends an elicitation/create request with
a URL; the agent streams it to the frontend as an "elicitation" event,
which the existing auth-prompt UI displays.

Each template uses its framework's native elicitation API:
- ai-sdk: @ai-sdk/mcp ElicitationRequestSchema + onElicitationRequest
- mastra: @mastra/mcp MCPClient.elicitation.onRequest + Agent.stream
- langchain: mcp SDK Callbacks(on_elicitation=...) + ElicitResult

The existing extractAuthUrlFromToolOutput remains as a fallback for
gateways that don't support the elicitation primitive.

Note: page.tsx includes some pre-existing dashboard UX changes
(dismissible callouts) alongside the onElicitation callback wiring.

Made-with: Cursor
The MCP spec requires clients to opt into URL-mode elicitation by
including `url: {}` in the capabilities object. Without this, servers
won't send URL-mode requests (which Arcade uses for OAuth flows).

The Python MCP SDK does this automatically when an elicitation callback
is provided, but the TypeScript side (both @ai-sdk/mcp createMCPClient
and @mastra/mcp MCPClient) requires explicit declaration.

Changes `{ elicitation: {} }` to `{ elicitation: { url: {} } }` in both
the Mastra singleton and the AI SDK client factory.

Made-with: Cursor
Picks up TemplateResponse Starlette fix, favicon, mutation filter,
scaffold pain points, and other upstream improvements. Resolves
conflicts in dashboard page (keep dismissible callouts + upstream
text-left fix + ARCADE_API_KEY removal) and Mastra route (keep
elicitation imports + upstream planPrompt export).

Made-with: Cursor
…flow

- Replace authProvider with per-request fetch in Mastra MCPClient so the
  singleton never triggers its own OAuth handshake and races with the
  connect route's initiateOAuth() (fixes double sign-in)
- Make elicitation.onRequest() lazy via ensureElicitationHandler() to
  avoid eager connect() at module load time; call it from plan route
- Fix onStepFinish callback: Mastra step toolCalls/toolResults are
  ChunkType objects { type, payload: { toolName, ... } } not flat AI SDK
  objects, so access toolName via call.payload.toolName (fixes "Calling
  other: undefined")
- Remove URL-mode elicitation from AI SDK getArcadeMCPClient(): @ai-sdk/mcp
  only supports form-mode, declaring url capability caused gateway to use
  elicitation instead of tool output, breaking extractAuthUrlFromToolOutput
- Rename "Why Arcade?" callout to "Why sign into Arcade?" (Next.js + Python)
- Add dismissible callouts with X/localStorage to empty-state,
  source-auth-gate, and LangChain dashboard/login templates

Made-with: Cursor
Single-tenant templates store Arcade OAuth tokens in .arcade-auth/tokens.json
shared by the whole process. Without clearing on logout, signing out of the
local app and signing in as a different user would still use the previous
user's Arcade identity.

Fix: call clearArcadeTokens() (new export from arcade-oauth-provider.hbs)
on local app logout across all three templates:
- Next.js (ai-sdk + mastra): new POST /api/auth/arcade/disconnect route,
  called from handleLogout in dashboard/page.tsx before authClient.signOut()
- LangChain: existing POST /api/auth/logout now unlinks tokens.json before
  clearing the session cookie

Made-with: Cursor
Arcade's URL elicitation auth URLs redirect back to app.arcade.dev after
the user authorizes a tool — not back to our local app. This left users
stranded on app.arcade.dev with no automatic redirect back to the template.

Fix: remove `capabilities: { elicitation: { url: {} } }` from the Mastra
MCPClient and remove the ensureElicitationHandler / setElicitationCallback
infrastructure. Auth URLs are now surfaced exclusively via
extractAuthUrlFromToolOutput in onStepFinish, which is the same approach
used by the AI SDK template and works correctly.

Both Mastra and AI SDK now use the tool-output-scraping fallback for
surfacing tool auth URLs, shown to the user as AuthPrompt components
with target="_blank".

Made-with: Cursor
…ition

Drop the "I've already signed in — retry" ghost button from all three
templates (Next.js shared, LangChain HTML/JS).

Fix the Mastra (and AI SDK) reconnect race: verifyExistingConnection()
made a live HTTP request on every dashboard load, which could get a
transient 401 from the Arcade gateway immediately after a fresh token
exchange, causing an unnecessary second sign-in prompt. Replace the fast
path with a simple "tokens on disk → connected" trust, matching how the
rest of the auth flow works. Expired tokens surface as errors from the
plan route instead.

Made-with: Cursor
Tool auth redirecting to app.arcade.dev is fine — the frontend opens those
URLs in a new tab. The previous removal was overcorrection; the real issue
was the initial Arcade gateway connection (PKCE) appearing to redirect to
app.arcade.dev, which was a UX confusion, not a technical limit.

Restore capabilities: { elicitation: { url: {} } } on the Mastra MCPClient,
re-add mcpClient.elicitation.onRequest handler, and re-wire
setElicitationCallback from the plan route so URL elicitation events stream
to the frontend as 'elicitation' events.

Also update the AI SDK comment to reflect that Mastra now uses URL-mode
elicitation; the AI SDK still can't because @ai-sdk/mcp only supports
form-mode.

Made-with: Cursor
Two bugs introduced in the previous commit:
1. capabilities: { elicitation: { url: {} } } is not a valid field in
   @mastra/mcp MCPClientOptions.servers — removed it.
2. mcpClient.elicitation.onRequest() at the MCPClient level requires
   (serverName, handler), not just (handler). Was being called without the
   server name, so the handler was never registered. Fix: move registration
   into the plan route with await mcpClient.elicitation.onRequest('arcade', ...)
   before listToolsets().

Also: clear stored tokens when the plan route catches a 401/invalid_auth
error, so the next connect-route check correctly returns needs_auth instead
of a false-positive connected state.

Made-with: Cursor
Add visibilitychange + 3s interval polling while tool auth prompts are
active so the UI reflects authorization within ~3 seconds instead of
waiting for the user to manually return to the tab and trigger a focus
event.

Next.js: use-source-check.ts — replace focus-only listener with
focus + visibilitychange + setInterval(3000) while authGateActive.

LangChain: dashboard.js — add visibilitychange handler and a
startSourceAuthPolling() interval that fires every 3s while auth
prompt cards are visible.

Made-with: Cursor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant