fix(oauth): add required API scopes to OpenAI device flow#251
Merged
Conversation
Collaborator
Author
fail |
OpenAI's OAuth device flow was producing tokens without the api.model.read and model.request scopes, causing the provider probe (/v1/models) to fail with HTTP 403 immediately after successful OAuth login. Add OAuthScope to IProviderDescriptor as a default interface member, wire it through OAuthDeviceFlowConfig.FromDescriptor, and include it in the OpenAI proprietary device flow JSON payload. Set OpenAI's scope to "openid profile email offline_access model.request api.model.read". Fixes #173, fixes #191
OpenSpec change `browser-oauth-provider-auth` defines the plan to replace OpenAI's broken proprietary device code flow with browser-based OAuth Authorization Code + PKCE. Includes proposal, design, delta specs for netclaw-model-providers and netclaw-onboarding, and implementation tasks. Key decisions: - Extract shared OAuthPkceService from McpOAuthService - Provider OAuth callback on existing daemon :5199 HTTP server - Three-layer fallback UX: auto-browser, CopyableTextNode URL, paste redirect - Upgrade Termina to 0.8.0 for clipboard and toast support - OpenAI switches from OAuthDevice to OAuthPkce Refs #173, #191
Add OAuthPkceService in Netclaw.Configuration with reusable OAuth Authorization Code + PKCE primitives: code_verifier/code_challenge generation, authorization URL construction, pending flow state tracking, token exchange, and token refresh. This shared service will be used by both provider OAuth (OpenAI browser flow) and MCP OAuth. Upgrade Termina from 0.7.2 to 0.8.0 for CopyableTextNode (OSC 52 clipboard), ToastOverlayNode, and IClipboardService support needed for the browser OAuth fallback UX. Includes 13 unit tests covering PKCE generation, URL construction, token exchange, refresh, and flow state management. Refs #173, #191
…ments Add task group for applying the three-layer browser OAuth UX (auto-browser, CopyableTextNode URL, paste redirect fallback) to the existing MCP OAuth flow. Both provider and MCP OAuth will share OAuthPkceService and the same fallback pattern.
… and spinner fix
- Add OAuthAuthorizationEndpoint and OAuthRedirectUri to IProviderDescriptor
- Switch OpenAiDescriptor to OAuthPkce as preferred auth method with browser
flow endpoints and identity-only scopes
- Add provider OAuth callback endpoints to daemon: /api/provider/oauth/start,
/callback, and /status/{state}
- Add OAuthRedirectParser for paste-redirect-URL fallback
- Fix init wizard OAuth spinner not animating by adding RequestRedraw() after
DynamicLayoutNode invalidation on timer tick
Refs #173, #191
Add browser-based OAuth flow to both ProviderManagerViewModel and InitWizardViewModel with three-layer fallback: auto-open browser, display URL for manual copy, and paste redirect URL input. Wire OAuthPkce auth method routing in both view models and pages. Add AddBrowserOAuthFlow state to ProviderManagerState enum. Add sub-step 6 to init wizard for browser OAuth flow. Update auth method selection UI to show "OAuth Login (recommended)" for OAuthPkce providers instead of device flow. Fix OAuth spinner not animating by adding RequestRedraw() after DynamicLayoutNode invalidation on timer tick for sub-steps 3, 5, 6. Refs #173, #191
…h routing, and browser OAuth UI
Extract OAuthFlowViews static helper with shared auth method label
mapping, label parsing, and browser OAuth flow view builder. Both
InitWizardPage and ProviderManagerPage now delegate to the shared
helper instead of duplicating the UI code.
Fix ProviderManagerPage auth method selection showing raw enum text
("OAuthPkce") by routing through BuildAuthMethodLabels/ParseAuthMethodLabel.
Fix ProviderManagerPage spinner not animating by subscribing to
ProbeElapsedSeconds and EagerProbeElapsedSeconds for invalidation.
Add BrowserDetection.CanOpenBrowser() in Netclaw.Configuration as a shared utility for all OAuth flows (provider and MCP). Checks for DISPLAY/WAYLAND_DISPLAY on Linux to detect headless environments. When no browser is available, the TUI immediately shows the auth URL for manual copy instead of attempting and failing Process.Start.
…n endpoints The AddHttpClient<T> + AddSingleton<T> combination conflicted, preventing the minimal API endpoint resolver from finding OAuthPkceService. Switch to a named HttpClient with explicit singleton factory registration.
…wser detection - Use CopyableTextNode from Termina 0.8.0 with IClipboardService for the auth URL display — Enter key copies to clipboard via OSC 52 - Speed up spinner from 1000ms to 120ms tick interval using SpinnerTick reactive property, matching the chat page tool call spinner speed - Add BrowserDetection.CanOpenBrowser() as shared utility in Netclaw.Configuration — checks DISPLAY/WAYLAND_DISPLAY on Linux - Headless environments immediately show URL fallback instead of attempting Process.Start Refs #173, #191
…tNode CopyableTextNode requires focus navigation which conflicts with the paste input. Instead, display the URL as plain text and handle [C] keypress in the page's global key handler to copy via IClipboardService.
…stener OpenAI's OAuth client (app_EMoamEEZ73f0CkXaXp7hrann) requires redirect_uri to be http://localhost:1455/auth/callback — this is what's registered for the Codex CLI client ID used by all third-party tools. Add temporary HttpListener in OAuthPkceService.ListenForCallbackAsync that binds to the redirect URI's port during the OAuth flow and shuts down after the callback arrives. Add OAuthExtraAuthParams to IProviderDescriptor for provider-specific authorization URL parameters. OpenAI requires id_token_add_organizations, codex_cli_simplified_flow, and originator. Sources documented in OpenAiDescriptor: - OpenCode: anomalyco/opencode#3281 - codex-oauth: https://github.com/7shi/codex-oauth - OpenClaw: https://docs.openclaw.ai/concepts/oauth Refs #173, #191
Set _lastFocusedInput to the redirect URL TextInputNode and call OnFocused() so RouteInputToActiveComponent delivers Enter and other keystrokes to the paste input during browser OAuth flow.
…robe The CLI was transitioning to probe validation after OAuth completed but never received the access token from the daemon. The status endpoint now returns the access token, refresh token, and expiry on completion. Both view models extract the token and set it as the credential for probing.
CompleteAuthorizationAsync removes the flow from _pendingFlows via TryRemove, causing GetFlowStatus to return NotStarted instead of Completed. The CLI's polling loop never saw the token. Add _completedFlows dictionary to store results after token exchange. GetFlowStatus and GetFlowResult now check completed flows first.
Browser flow with identity-only scopes produces the same "Missing scopes: api.model.read" error as the device flow. Explicitly request API scopes alongside identity scopes. Ref: openclaw/openclaw#24720
… list OpenAI OAuth tokens (via Codex CLI client ID) cannot call /v1/models — returns 403 "Missing scopes: api.model.read". The scope names in the API error are NOT valid OAuth scope values; the authorization endpoint rejects them as "invalid scope". This is an OpenAI-specific limitation; other OAuth providers may support live model listing. When an OAuth token is present, return a curated model list instead of probing. API keys still probe /v1/models normally. Revert scope to identity-only (openid profile email offline_access). Add /update-openai-models skill for refreshing the curated list when OpenAI ships new models. Curated list sourced from https://developers.openai.com/api/docs/models/all as of 2026-03-17. Refs #173, #191
The probe was setting the OAuth token as ApiKey on the ProviderEntry, so OpenAiDescriptor's check for OAuthAccessToken never matched and it still called /v1/models. Add auth-method-aware ProbeAsync overload to ProviderDescriptorRegistry that sets OAuthAccessToken for OAuth flows.
Add IProviderProbe.ProbeAsync(ProviderEntry) overload that preserves the OAuthAccessToken vs ApiKey distinction from config. Update model manager, provider manager eager probe, and revalidation to use entry-based probe so OpenAiDescriptor correctly detects OAuth tokens and returns the curated model list instead of hitting /v1/models.
….json SecretsFileWriter encrypts the entire file. Encrypted DateTimeOffset values break IConfiguration.Get<Dictionary<string, ProviderEntry>>() binding, silently dropping the provider entry and crashing the daemon on startup. OAuthTokenExpiry is not a secret — write it to netclaw.json alongside the provider's Type and Endpoint. Tokens (access, refresh) remain encrypted in secrets.json.
…inator Codex OAuth tokens were never granted Chat Completions access — calling /v1/chat/completions returns 429 "insufficient_quota". Switch OpenAiProviderPlugin from ChatClient to ResponsesClient which uses the Responses API (/v1/responses), the only completions endpoint authorized for these tokens. API keys work with both endpoints. Extract duplicated OAuth flow state and orchestration from InitWizardViewModel and ProviderManagerViewModel into a shared OAuthFlowCoordinator composition object. Both VMs delegate browser PKCE, device flow, and paste-redirect operations to the coordinator instead of maintaining ~250 lines of near-identical code each. Additional cleanup: - Add provider OAuth methods to DaemonApi (StartProviderOAuthAsync, GetProviderOAuthStatusAsync, ProviderOAuthCallbackAsync) - Consolidate scattered OpenAiDescriptor comments into single XML doc block - Add doc-comment deprecation on the old 3-arg IProviderProbe.ProbeAsync Closes #173, closes #191
Aaronontheweb
force-pushed
the
claude-wt-openai-oauth-fixes
branch
from
90c0088 to
e8f37cd
Compare
…nAI into api-key + codex-oauth (#173, #191) Provider code was scattered across Configuration, Daemon, and OpenAICompatible. Adding OpenAI Codex OAuth support revealed that OpenAI has two completely separate API surfaces that cannot share a provider type: - api.openai.com (API keys) — standard Responses API - chatgpt.com/backend-api/codex (OAuth tokens) — ChatGPT backend This commit consolidates all provider code into a single Netclaw.Providers assembly organized by vendor affinity, and splits "openai" into two provider types: "openai" (API key only) and "openai-codex" (OAuth only). Assembly extraction: - Created src/Netclaw.Providers/ with vendor directories: OpenAi/, Anthropic/, SelfHosted/, OpenRouter/, OAuth/ - Moved descriptors, plugins, OAuth services, capability resolvers - Absorbed Netclaw.OpenAICompatible into Providers/SelfHosted/ - Deleted Netclaw.OpenAICompatible project - Split LlmProviderServiceExtensions: plugin registration stays in Providers, factory/resilience wiring stays in Daemon (DaemonProviderServiceExtensions) OpenAI bifurcation: - "openai" (TypeKey: openai) — API key auth, ResponsesClient → api.openai.com - "openai-codex" (TypeKey: openai-codex) — OAuth auth, custom client → chatgpt.com/backend-api/codex with ChatGPT-Account-Id header from JWT - New types: OpenAiCodexDescriptor, OpenAiCodexProviderPlugin, OpenAiCodexChatClient, JwtAccountIdExtractor - Config migration: openai + OAuthPkce auto-rewrites to openai-codex at startup Dependency graph: Configuration (SDK-free) → Providers (vendor SDKs) → Daemon (runtime) All 1099 tests pass (19 new for Codex types), 0 warnings, 0 slopwatch violations.
…iry probe, plugin delegation - Add 6 missing OAuth property delegations to ProviderPluginBase - Replace bare EnsureSuccessStatusCode() in OpenAiCodexChatClient with EnsureSuccessAsync() that parses error bodies and gives 401-specific "token expired" guidance - Make OAuthExtraAuthParams a static readonly field (was allocating per read) - Add OAuth-only path in BuildFixCredentialsView so Codex providers show re-authenticate prompt instead of an API key input - Add token expiry check in OpenAiCodexDescriptor.ProbeAsync so expired tokens show red status in provider list - Remove no-op DefaultIgnoreCondition from config migration serialization
…der, shared error helper - Extract ProviderErrorHelper for shared error message parsing between OpenAiCodexChatClient and OpenAiCompatibleChatClient (was copy-pasted) - Add CredentialInputMode.OAuthOnly and use it in OpenAiCodexDescriptor so the TUI routes on CredentialMode instead of ad-hoc auth method checks - Accept SensitiveString (not bare string) in OpenAiCodexChatClient to prevent accidental token exposure in logs/serialization - Inject TimeProvider into OpenAiCodexDescriptor for testable expiry checks (was using DateTimeOffset.UtcNow in violation of constitution) - Collapse StartOAuthReAuth into state setup + SelectAuthMethod call (was duplicating the entire OAuth launch sequence) - Update expiry tests to use FakeTimeProvider for deterministic assertions
…typed OAuth config Replace the flat bag of 16 nullable auth properties on IProviderDescriptor with a compositional IProviderAuth interface and three implementations: - ApiKeyAuth: carries GuidanceUrl (as Uri) - EndpointOnlyAuth: no-auth providers (Ollama, OpenAI-compatible) - OAuthAuth: carries all OAuth config with Uri-typed endpoints IProviderDescriptor drops from 17 members to 6. ProviderPluginBase drops from 12 auth forwarding lines to 1. CredentialInputMode enum deleted — the TUI pattern-matches on the auth type instead. OAuth endpoints are now Uri (not string), giving construction-time URL validation. DeviceFlowServiceFactory and OAuthDeviceFlowConfig take OAuthAuth directly instead of pulling nullable strings off the descriptor.
The Codex backend at chatgpt.com/backend-api/codex rejects requests with Content-Type "application/json; charset=utf-8" (.NET default) — strip the charset suffix. System messages must go in a top-level "instructions" field, not in the input array. The Codex Responses API returns 400 "Instructions are required" otherwise. Non-system messages remain in the input array as before.
…esponsesClient Delete OpenAiCodexChatClient and use the OpenAI SDK's ResponsesClient with a custom PipelinePolicy (OpenAiCodexRequestPolicy) that injects the ChatGPT-Account-Id header and "store": false into outbound requests. This gives the Codex OAuth path full Responses API tool support — tool definitions, function_call output items, streaming delta accumulation, and FunctionCallContent MEAI mapping — without hand-rolling the wire format. Follows the same pattern as OpenRouterProviderPlugin.
The Codex backend at chatgpt.com has stricter validation than api.openai.com: - Move system messages from "input" array to "instructions" field (Codex rejects system role messages in the input array) - Default "instructions" to empty string when absent (required field) - Strip "strict": null from tool definitions (Codex rejects null values; the SDK serializes unset strict as null) Verified end-to-end: tool calls (web_search), streaming, and plain text responses all work against the live Codex backend.
…catalog Enrich OpenAiCodexDescriptor.CuratedModels with ContextWindowTokens and InputModalities/OutputModalities so the runtime doesn't fall back to the 32K default. All Codex models accept text+image input. Add OpenAiCodexCapabilityResolver (static dictionary lookup from the same catalog) and wire it first in the composite resolver chain — it's authoritative for Codex models and costs zero network calls. Context windows: 256K for frontier/codex models, 200K for reasoning models, 1M+ for mini/nano variants.
Single "OpenAI" provider with auth method selection (ChatGPT Subscription vs API Key) instead of two separate entries in the TUI. Core changes: - Add MultiAuth type + ProviderAuthExtensions (GetOAuthConfig, GetApiKeyGuidanceUrl) - Merge OpenAiCodexDescriptor into OpenAiDescriptor with dual-path ProbeAsync - Merge OpenAiCodexProviderPlugin into OpenAiProviderPlugin (branches on AuthMethod) - Delete OpenAiCodexDescriptor, OpenAiCodexProviderPlugin, OpenAiCodexConfigMigration - Remove openai-codex from catalog, DI registrations, and test counts TUI/CLI fixes (init wizard + provider manager): - OAuthFlowViews.BuildAuthMethodLabels accepts IProviderAuth for custom per-provider labels - ProviderManagerPage handles MultiAuth in credentials and fix-credentials views - InitWizardPage: add missing "C" to copy URL handler for browser OAuth - InitWizardPage: add missing GetActiveTextInput case for redirect URL paste - InitWizardPage: fix escape from validation navigating to wrong sub-step - InitWizardViewModel: OAuth callbacks now call StartProbe() matching ProviderManager - OAuthFlowCoordinator: don't set FlowState=Succeeded on paste redirect (let poll complete) - IProviderProbe: add auth-method-aware ProbeAsync overload so probes route OAuth correctly - InitWizardViewModel: write OAuthTokenExpiry to netclaw.json not secrets.json (encrypted DateTimeOffset breaks IConfiguration binding, silently drops provider entry) - Both ViewModels now pass AuthMethod to probe so OAuth tokens route correctly Binary-swap tested: daemon starts, prompts work, provider list shows unified entry.
… writing All three credential-writing call sites (InitWizardViewModel, ProviderManagerViewModel, ProviderCommand) now use ProviderCredentialWriter.WriteProvider instead of inline dict construction. This eliminates the class of bug where OAuthTokenExpiry was written to secrets.json instead of netclaw.json — the shared writer handles placement correctly for all callers. - InitWizard test updated to verify encrypted secrets + decryption round-trip (previously asserted plaintext, which was a gap in test coverage)
…tasks Groups 1 (decision tree) and 2 (OAuth device flow) are fully implemented. Groups 3-5 (model fallback, doctor checks, docs) remain open.
Capture TaskCompletionSource in a local variable instead of re-reading the NextResponseGate field. The actor's async path nulls the field before the test reaches TrySetResult(), causing a NullReferenceException under CI thread scheduling.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
api.model.read,model.request), causing the provider probe (/v1/models) to fail with HTTP 403 immediately after successful OAuth loginOAuthScopetoIProviderDescriptoras a default interface member (null default, zero changes needed for other providers)OAuthDeviceFlowConfig.FromDescriptorand into the OpenAI proprietary device flow JSON payloadopenid profile email offline_access model.request api.model.readFixes #173, fixes #191
Test plan
OpenAiDeviceFlowServiceTestsupdated to verify scope is included in JSON bodyStartDeviceAuthorization_NullScope_OmitsScopeFromJsonfor backward compatibilitydotnet slopwatch analyze— no new violationsnetclaw initornetclaw provider add openai, complete OAuth device flow, verify probe succeeds