feat(sprout-agent): Databricks provider with OAuth 2.0 PKCE auth

[tlongwell-block]

Adds Provider::Databricks to sprout-agent, alongside the existing Anthropic and OpenAI-compat providers. Databricks model-serving is OpenAI-chat-compatible on the wire, so the existing body builder and response parser are reused untouched; the new code is the auth machinery and a small URL/body adapter.

Configuration

SPROUT_AGENT_PROVIDER=databricks
DATABRICKS_HOST=https://your-workspace.cloud.databricks.com
DATABRICKS_MODEL=goose-claude-4-6-sonnet   # or any serving endpoint
DATABRICKS_TOKEN=<optional>                # static-key escape hatch

When DATABRICKS_TOKEN is unset, sprout-agent runs the OAuth 2.0 Authorization Code + PKCE flow (RFC 6749 + RFC 7636) using Databricks' public CLI client (client_id=databricks-cli, scopes all-apis offline_access). The same identifier goose uses, so an existing browser consent on a Block laptop carries over.

Tokens are cached under ~/.config/sprout-agent/oauth/databricks/<sha256>.json and refreshed transparently on expiry; long-running agents survive across token rotations.

Why this shape

This was discussed in https://sprout-oss.stage.blox.sqprod.co channel sprout-agent-auto-databricks-for-block with @max-gpt55. The two key calls:

1. Don't piggyback on goose's cache. It's private state; coupling to it is a hidden break-on-update. We use the same client_id and scopes as goose so the same browser consent works, but we own our cache directory and refresh logic.
2. Don't fork the OpenAI request path. The body shape, retries, auto-upgrade, and parser are reused unchanged. The complete and summarize dispatches collapse to a single OpenAi | Databricks => … match arm. Databricks-specific behavior (URL templating, body model-stripping) lives behind one branch in post_openai.

Implementation

• crates/sprout-agent/src/auth.rs (new, ~500 LOC incl. tests + docs): TokenSource trait, StaticTokenSource, PkceOAuthTokenSource with OIDC discovery, one-shot localhost callback server (axum), PKCE pair generation, refresh-grant POST, atomic on-disk cache write. The PKCE engine is provider-generic — PkceOAuthConfig { discovery_url, client_id, scopes, cache_namespace } — so a future provider that's also browser-PKCE-compatible reuses the engine; Databricks is just the first descriptor.
• llm.rs: Llm now holds Arc<dyn TokenSource> (static for Anthropic/OpenAI-compat, refreshable for Databricks). post_openai fetches the bearer per request and, when provider=Databricks, rewrites the URL to {base}/serving-endpoints/{model}/invocations and strips the model field from the body.
• config.rs: Provider::Databricks variant; DATABRICKS_HOST/DATABRICKS_MODEL required, DATABRICKS_TOKEN optional.

What's intentionally not here

• Responses API path for Databricks (/serving-endpoints/responses for gpt-5.x). Additive — adds a third endpoint-strategy variant when needed. The chat/invocations path covers Sonnet/Haiku, which is all we need today.
• sprout-agent auth databricks CLI subcommand for interactive first-run. The PKCE engine supports it via interactive_login() (already implemented and pub), but the CLI wiring is deferred to a follow-up to keep this PR scoped. First-run today either (a) hits the browser dance lazily on the first inference call, or (b) requires bootstrapping the cache (e.g. seeding from goose's refresh token, as we did for testing).
• Device-code, client-secret, or service-account flows. Different enough that forcing them through the PKCE trait now would warp it. Add as additional TokenSource impls when needed.

Tests

• 5 unit tests in auth.rs cover PKCE pair generation, cache-path construction, token-expiry math (with leeway), and response parsing with refresh-token fallback.
• 3 integration tests in tests/databricks_oauth.rs use a stub axum OIDC server to exercise:
  • cache hit short-circuits the network (zero refresh calls)
  • expired cache silently refreshes via the refresh token
  • refreshed token is persisted to disk
• Existing 17 regression / golden / openai-auto-upgrade tests pass unchanged — Anthropic and OpenAI behavior is untouched.

test result: ok. 33 passed (lib)
test result: ok.  3 passed (databricks_oauth)
test result: ok.  4 passed (golden_transcripts)
test result: ok.  6 passed (openai_auto_upgrade)
test result: ok.  4 passed (fake_llm)
test result: ok. 17 passed (regressions)

Clippy: clean (cargo clippy -p sprout-agent --all-targets -- -D warnings).

End-to-end verification

Tested against block-lakehouse-production with goose-claude-4-6-sonnet:

Path	Time
Cold cache → silent OAuth refresh → inference	~6.0s
Warm cache → in-memory token → inference	~1.8s

Both prompts returned the expected response and stopReason: end_turn. The on-disk cache was rewritten with a fresh access token and rotated refresh token after the cold-cache run, exactly as designed.

cc @max-gpt55 for review.