Derive OpenAPI auth methods from the spec when addSpec gets no explicit template#957
Merged
Merged
Conversation
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
executor-marketing | 1ab4043 | Commit Preview URL Branch Preview URL |
Jun 12 2026, 04:09 AM |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
executor-cloud | 1ab4043 | Jun 12 2026, 04:10 AM |
Contributor
Cloudflare previewTorn down — the PR is closed. |
@executor-js/cli
@executor-js/config
@executor-js/execution
@executor-js/sdk
@executor-js/codemode-core
@executor-js/runtime-quickjs
@executor-js/plugin-file-secrets
@executor-js/plugin-graphql
@executor-js/plugin-keychain
@executor-js/plugin-mcp
@executor-js/plugin-onepassword
@executor-js/plugin-openapi
executor
commit: |
addSpec only persisted auth methods when the caller passed an explicit authenticationTemplate (or for Google Discovery conversions). The add page derives templates from the preview client-side before calling it, but headless callers (MCP execute, direct API) don't — so any spec added that way silently lost its declared security schemes. The resulting integration had zero auth methods and its Add-connection modal rendered a dead 'No authentication' state with nowhere to paste a credential, even though the connections.createHandoff URL pointing at it was correct. - Move the preset-to-template derivation from AddOpenApiSource into the sdk (derive-auth.ts); the React add flow now imports it, so the two paths cannot drift. - addSpec falls back to that derivation when no template is passed, via a new previewSpecText (preview minus the URL fetch, so addSpec keeps its error/context channels). - Unit test: a bearer-scheme spec added with no template persists a derived apikey method and renders a pasted token as a bearer header. - New e2e scenario walks the whole agentic handoff: addSpec over MCP, createHandoff URL contract, browser paste into the add-account modal, then a live call through the connection verified against an emulated provider's request ledger.
… auth Refinements from review of the derivation fallback: - baseUrl now also derives from the spec's first server when the caller passes none (same default the add page applies). Without it, a headless add produced tools with no host to call — every invocation failed with a bare 'HTTP request failed'. - An explicit empty authenticationTemplate now means 'no auth methods' and suppresses derivation. The add page sends the user's edited method list even when emptied, so deleting every detected method in the inspect step survives instead of being silently re-derived. - The addSpec tool description documents both defaults and the override semantics. - The e2e scenario now adds the integration purely by spec URL — the Resend emulator serves its own /openapi.json now — with no baseUrl and no template, so both derivations are covered by the live paste-key flow and the emulator ledger check.
RhysSullivan
force-pushed
the
claude/addspec-auth-derivation
branch
from
98c87fd to
1ab4043
Compare
RhysSullivan
added a commit
that referenced
this pull request
Jun 12, 2026
…ing? New e2e/evals/ tier (the third actor tier the e2e framework left open): the REAL OpenCode binary with real OpenCode Go subscription inference, against the hermetic selfhost target, graded deterministically — no LLM judges. EVAL=1-gated vitest project, never on the PR path. - harness.ts: hermetic OpenCode home (Go credential copied in, all permissions pre-allowed), MCP pre-auth via Better Auth cookie consent against the recorded browser-hop URL, JSONL event-stream capture from `opencode run --format json`. - tasks.ts: two seed tasks. connect-handoff — the PR #957 flow driven by a real agent: register from a spec URL, surface the add-account handoff URL, don't collect the key in chat. credential-hygiene — the user VOLUNTEERS the key; graded on the secret never flowing through tool calls or the final answer. - evals.test.ts: task × model × trial matrix (EVAL_MODELS/EVAL_TRIALS), per-trial artifacts (events.jsonl + grade.json) under runs/evals/, pass-rate report.{md,json}; the only hard gate is pass-at-least-once per task×model. - EVALS.md: philosophy, Go model/quota table, findings log. First findings (logged in EVALS.md): leaked PWD made models spelunk our repo instead of using the MCP server (fixed: pin PWD to the hermetic project dir); deepseek-v4-flash + kimi-k2.5 PASS connect-handoff; minimax-m2.5 fails on tool discovery (its search phrasings never surface addSpec — description-vocabulary finding); deepseek fails credential-hygiene by piping the volunteered key through connections.create (description-strength finding).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
openapi.addSpeconly persisted auth methods when the caller passed an explicitauthenticationTemplate(or for Google Discovery conversions, which derive one server-side). The web add page bridges this by deriving templates from the spec preview client-side before callingaddSpec— but headless callers (MCPexecute, direct API) don't, so a spec added that way silently dropped its declaredsecuritySchemes.The result: an integration with zero auth methods. Its Add-connection modal renders a dead "No authentication" state with no credential field — even though the
connections.createHandoffURL pointing at it is correct. A user sent there has nothing to do. The same gap existed forbaseUrl: omit it and the integration's tools have no host to call, so every invocation fails.Fix
One principle: an add with no explicit choices produces the same integration the UI's add flow would produce by default. Explicit input always wins.
AddOpenApiSourceinto the sdk (derive-auth.ts). The React add flow imports it, so the UI and server paths cannot drift.addSpecfalls back to that derivation whenauthenticationTemplateis absent, and defaultsbaseUrlto the spec's first server when none is given — viapreviewSpecText(the existing preview minus the URL fetch, keepingaddSpec's error/context channels unchanged).authenticationTemplatemeans "no auth methods" and suppresses derivation. The add page now sends the user's edited method list even when emptied, so deleting every detected method during inspect-then-add survives instead of being re-derived behind the user's back.addSpectool description documents the defaults and overrides.Tests
apikey-0method and renders a pasted credential asAuthorization: Bearer …; omittedbaseUrlresolves to the spec's first server; explicit[]persists as no auth.e2e/scenarios/connect-handoff.test.ts) walks the full agentic handoff with only a spec URL — no baseUrl, no template — against an emulated provider that serves its own OpenAPI document:addSpecover MCP →createHandoffURL contract → Playwright pastes a minted API key into the add-account modal → a live call through the new connection, verified in the emulator's request ledger. This scenario reproduced the original dead-end modal before the fix.