Authserver-driven DCR for upstream OAuth 2.1 MCP servers

[jhrozek]

User Story

As a ToolHive operator,
I want to use the authserver as a DCR client,
so that vMCP can proxy to upstream MCP servers (e.g. Datadog) that require OAuth 2.0 / RFC 7591 without hand-registering a client.

Context

Today the ToolHive authserver cannot configure an OAuth2 upstream without a pre-issued client_id. RFC 7591 Dynamic Client Registration primitives already exist under pkg/auth/oauth/ but are not reachable from pkg/authserver/ without introducing an import cycle. This story closes that gap end-to-end across three phases: the primitives are relocated into a new leaf package pkg/oauthproto/ (alongside a rename of the existing pkg/oauth/), the authserver performs RFC 8414 metadata discovery and RFC 7591 registration at startup, credentials are cached (first in-memory, then persistently with memory and Redis backends) keyed on (issuer, redirect_uri, scopes_hash), and the standard upstream OAuth2 flow drives the session after that.

The end-to-end flow this unlocks:

MCP Client -> [Corp OIDC] -> vMCP embedded AS -> [Datadog OAuth] -> Datadog MCP Server

Datadog's MCP server at https://mcp.datadoghq.com is the reference OAuth 2.1 / RFC 7591 target.

Dependencies: None

Acceptance Criteria

Capability-level outcomes a ToolHive operator (or an e2e check) can observe once the work under this story has landed:

• An operator can configure an OAuth2 upstream in the authserver using only a discovery_url (no pre-issued client_id / client_secret), and the authserver starts successfully.
• On startup, the authserver performs RFC 8414 authorization server metadata discovery (with RFC 8414 §3.3 issuer-equality validation) and RFC 7591 Dynamic Client Registration against the upstream, then drives the standard upstream OAuth2 flow with the obtained credentials.
• Configuration validation rejects invalid combinations at startup (e.g., both ClientID and DCRConfig set, or both DiscoveryURL and RegistrationEndpoint set inside DCRConfig) with a clear error.
• After a successful registration, restarting the authserver does not re-register at the upstream — the cached credentials (in-memory in the first delivery, persistent afterwards) short-circuit the discovery + registration network calls.
• Cached credentials survive process restart when the persistent store is configured (memory backend for single-process deployments, Redis backend for shared deployments).
• Secrets (client_secret, registration_access_token, initial_access_token, refresh tokens) never appear in log records at any log level.
• Successful DCR registration is observable via a structured slog Info record; a cache-hit with dcr_age_days > 90 emits a stale-age warning; an invalid_client response at the upstream token endpoint emits a warn-level remediation log.
• An MCP client session can successfully proxy through vMCP to an OAuth 2.1 MCP server upstream (Datadog, or a high-fidelity mock) using the DCR-obtained credentials.
• task build, task test, and task lint-fix all pass on the dcr branch once every sub-task has merged.

Out of Scope

The following are deliberately excluded from this story. The implementation details that realize the acceptance criteria above are tracked as sub-issues; longer-term follow-ups are deferred entirely.

Out of Scope of this story — tracked as sub-issues:

• Rename pkg/oauth to pkg/oauthproto and relocate DCR primitives #4977 — Rename pkg/oauth to pkg/oauthproto and relocate DCR primitives (Phase 1).
• Authserver DCR integration (Phase 2, Steps 2a-2g) #4978 — Authserver DCR integration covering metadata fetch with three-path discovery fallback, DCRUpstreamConfig, resolveDCRCredentials, in-memory DCRCredentialStore stub, RFC 7592 field capture, and structured-log upgrades (Phase 2).
• Persistent DCRCredentialStore backends (memory + Redis) #4979 — Persistent DCRCredentialStore backends (memory + Redis), wiring the persistent store in place of the Phase 2 in-memory stub (Phase 3).

Deferred (not handled by this story or its sub-issues):

• Authserver OTEL metrics and tracing — pkg/authserver/ has no OTEL / Prometheus integration today; adding it is separate work.
• Automatic re-register on invalid_client at the upstream token endpoint (with singleflight + per-key rate limiter). This story ships only the warn-log tripwire; auto-recovery is a follow-up.
• CLI / proxy persistence of registration_client_uri in pkg/auth/remote/config.go (candidate for a small side PR alongside Phase 1).
• RFC 8707 resource indicators confirmation for the existing upstream flow.
• RFC 7592 client_secret rotation via PUT.
• Provider-specific quirks (login.microsoftonline.com resource param, mcp.zoho.com access_type).
• Initial access token rotation / refresh.