fix(mcpcatalog): tell the model to proceed after enabling an OAuth server#2925
Conversation
…rver
When the model called enable_remote_mcp_server for an OAuth server, the
tool result said "auth: OAuth — an authorization URL will be elicited
on the next turn" and the instructions softly suggested using the
activated tools. The runtime actually drives the OAuth flow inside the
same loop iteration (via reprobe + Tools() → Start() → elicitation), so
by the time the next LLM call begins the tools are live and OAuth is
already complete. But the model still read the previous tool result and
narrated to the user "OAuth will happen on your next request, ask me
again" — forcing the user to repeat themselves.
This change rewrites two strings the model relies on:
- The OAuth branch of handleEnable's success message now tells the
model the host triggers any required authorization transparently
and that the model should continue with the user's original
request immediately.
- Step 3 of the Instructions() workflow is hardened from "use the
newly activated tools as you would any other" into an explicit
"proceed with the user's ORIGINAL request, do NOT stop and ask".
No runtime behaviour changes; this is purely a prompt fix.
docker-agent
left a comment
There was a problem hiding this comment.
Assessment: 🟡 NEEDS ATTENTION
This is a prompt-only fix that rewrites two LLM-facing instruction strings in mcpcatalog.go. No runtime logic is changed. Two findings surfaced during review — both are edge-case wording concerns rather than code bugs.
- Step 3 of Instructions(): make the timing explicit. The previous "Immediately proceed … by calling the newly activated server's tools" wording contradicted step 2's "adds the server's tools to your set on the *next* turn" and could lead a literal model to try to call the new tools in the same response. Reworded to "On your NEXT turn (once the activated server's tools appear in your set), go straight to the user's ORIGINAL request using those tools". - handleEnable's OAuth-branch result message: drop "transparently", which overstated seamlessness for the unmanaged-OAuth path (which requires a browser redirect and explicit grant). Reworded to "the host will surface any required authorization (browser redirect or in-app dialog) the first time the server's tools are used, and resume the tool call once access is granted".
|
Hold on, why does the LLM need to know that there is an oauth thing? |
Per @rumpl's review feedback on docker#2925: the LLM has no agency over the OAuth flow — it can't trigger it, grant it, or recover from it. Every OAuth-related string the catalog exposes to the model is a leaky abstraction, and the original bug (model stops and asks the user to repeat themselves after enabling an OAuth-protected server) is a direct consequence of leaking 'auth: OAuth — elicited on the next turn' in the enable result. This commit removes OAuth as a model-visible concept from every LLM-facing string in the catalog: - handleEnable: drop the per-auth-type switch. The success message is now a single line saying 'tools will appear on your next turn, proceed with the user's original request'. The only conditional emission is the WARNING for unresolved env vars, which is the one case the model has actual agency over (it can ask the user to set the variable and retry). - Instructions() workflow: step 2 drops the 'Authentication (OAuth or API key) is deferred' sentence; step 3 drops 'do not narrate the OAuth flow' (the model no longer learns OAuth exists, so the instruction can shrink to 'do not narrate connection setup'); step 5 reframes reset_remote_mcp_server_auth as 'if a previously enabled server starts rejecting requests, clear any persisted credentials' without mentioning OAuth tokens or authorization URLs. - Tool descriptions: enable_remote_mcp_server no longer mentions 'OAuth flow or API-key check'; reset_remote_mcp_server_auth no longer mentions 'OAuth credentials' or 'fresh authorization flow'. - ResetAuthArgs jsonschema and handleResetAuth result strings: drop every 'OAuth credentials' and 'fresh authorization flow' reference. Internal field names (oauthSuccessHandler, managedOAuth, etc.), Go package-doc comments, and slog debug messages still mention OAuth — those are developer-facing, not LLM-facing, and removing them would obscure intent for anyone reading the package. Tests updated: - TestEnableAddsServerAndFiresChangeNotification now asserts the result does NOT contain 'OAuth' or 'authorization', protecting against a regression where someone re-adds the leaky wording. - TestEnableAPIKeyEnvPresent no longer expects 'auth: API key' (the line is gone); still asserts no WARNING. - TestResetAuthClearsCredentials and TestResetAuthNoOpForNonOAuth updated for the new 'cleared credentials' / 'no persisted credentials' wording. The unresolved-header WARNING test still passes because the WARNING branch survived the rewrite under the unified 'len(missing) > 0' check.
|
@rumpl Good question — and you're right, it shouldn't. The original wording ( c3c85b3 removes OAuth as an LLM-visible concept from the catalog entirely:
Internal field names ( A new test assertion ( PTAL. |
docker-agent
left a comment
There was a problem hiding this comment.
Assessment: 🟢 APPROVE
This is a prompt/message-only fix that removes OAuth-specific wording from LLM-facing tool results and instructions so the model proceeds with the user's original request instead of stopping to ask them to repeat themselves.
What was reviewed:
pkg/tools/builtin/mcpcatalog/mcpcatalog.go—handleEnablemessage consolidation,handleResetAuthmessage updates,Instructions()step 3 hardening, tool description rewritespkg/tools/builtin/mcpcatalog/mcpcatalog_test.go— updated string assertions to match new production strings
Findings summary:
- ✅ No high or medium severity bugs found
- ✅ Auth-type consolidation in
handleEnablecorrectly preserves env-var warnings for all relevant auth types (api_key, custom headers); the removed OAuth-specific branch was informational-only and its removal is intentional - ✅ All updated test assertions (
"enabled","cleared credentials","no persisted credentials",assert.NotContainsfor"OAuth"/"authorization") correctly match the new production string literals - ℹ️ Minor (not flagged):
handleResetAuth's early-exit message"has no persisted credentials"is slightly imprecise for api_key servers (they have credentials, just not ones stored by this tool) — worth a follow-up comment, but not a bug and consistent with the PR's intent of not leaking auth-type details to the model
The fix correctly addresses the root cause: the previous "auth: OAuth — an authorization URL will be elicited on the next turn" string was the signal causing models to pause and narrate instead of proceeding. The new wording and explicit step-3 instruction ("do NOT stop and ask the user to repeat themselves") should resolve the reported behavior.
docker-agent
left a comment
There was a problem hiding this comment.
Assessment: 🟡 NEEDS ATTENTION
One medium-severity issue found in the changed code. The rest of the prompt-text changes look correct and well-motivated.
|
|
||
| msg := strings.Builder{} | ||
| fmt.Fprintf(&msg, "enabled %q (%s) — connection deferred until the host next enumerates tools.\n", id, server.Title) | ||
| fmt.Fprintf(&msg, "enabled %q (%s) — the server's tools will appear on your next turn. Proceed with the user's original request; the host handles any required connection setup.\n", id, server.Title) |
There was a problem hiding this comment.
[MEDIUM] Contradictory model instructions when API-key env vars are missing
When missing is non-empty, the message assembled in handleEnable simultaneously tells the model two conflicting things:
- "Proceed with the user's original request; the host handles any required connection setup."
- "WARNING: the following env var(s) are NOT set: … Ask the user to set them, then call
disable_remote_mcp_serverandenable_remote_mcp_serverfor this id again, otherwise tool calls will fail."
An LLM receiving both instructions in the same tool-result has no explicit prioritisation signal — the Proceed clause comes first and is unconditional, while the WARNING appears later but doesn't say "ignore the previous instruction". Depending on which instruction the model weighs more heavily it may either:
- dutifully proceed and immediately fail every downstream tool call (env vars not set), or
- correctly pause and ask the user to set the vars but now via a path the new wording was designed to remove.
Suggested fix: move the "Proceed with the user's original request" sentence into the else branch (i.e., only emit it when len(missing) == 0), so it is never paired with a WARNING that contradicts it:
msg := strings.Builder{}
fmt.Fprintf(&msg, "enabled %q (%s) — the server's tools will appear on your next turn.\n", id, server.Title)
fmt.Fprintf(&msg, "endpoint: %s\n", server.URL)
if len(missing) > 0 {
fmt.Fprintf(&msg, "WARNING: the following env var(s) are NOT set: %s. Ask the user to set them, then call %s and %s for this id again, otherwise tool calls will fail.\n",
strings.Join(missing, ", "), ToolNameDisable, ToolNameEnable)
} else {
fmt.Fprintf(&msg, "Proceed with the user's original request; the host handles any required connection setup.\n")
}
Problem
When a user asks something that requires an OAuth-protected catalog
server (e.g. "List all pages by David Gageot in Notion"), the model:
search_remote_mcp_servers,enable_remote_mcp_serverfornotion,The user must repeat their original prompt to actually get an answer:
Why this happens
The runtime actually drives the OAuth flow inside the same loop iteration:
processToolCallsrunshandleEnable, which only registers the toolset.reprobe(pkg/runtime/loop.go:608-610) callsgetTools→Tools()on the catalog → first-call lazyStart()on the freshly enabled toolset (mcpcatalog.go:371-372).Start()runs the MCP handshake, the 401 triggers the OAuth elicitation, the user grants,Start()returns success and the server's tools become available.So by the time the next LLM turn begins, OAuth is already complete. The model just doesn't know that — it only sees the previous tool result, which literally said "elicited on the next turn".
Fix
This PR rewrites the two strings the model uses to reason about the post-enable state:
handleEnable's OAuth-branch success message now tells the model the host triggers any required authorization transparently the first time the server's tools are used, and that the model should continue with the user's original request now.Instructions()step 3 is hardened from a soft "Use the newly activated tools as you would any other" into an explicit "Immediately proceed with the user's ORIGINAL request […]; do NOT stop and ask the user to repeat themselves; do NOT narrate the OAuth flow".No runtime behaviour changes; this is a prompt fix only