Support Pi provider and configurable runtime capabilities

[dcramer]

Summary

Add a provider/capability split so Warden can run skills through Pi while keeping the current Claude path working. Warden should own the abstraction: providers expose capabilities, and the rest of the analysis pipeline asks for the capability it needs.

Claude remains the default provider until Pi reaches parity. The first slice adds the provider registry, keeps Claude behind the main agent runtime, models the fast object-generation calls currently handled by Haiku as a second runtime capability, and threads provider/model config through the actual Warden entry points.

Terms

• Provider: a model-system adapter family such as claude or pi.
• Agent runtime: the repo-aware main skill executor. It handles prompts, read-only tool policy, turns, sessions, stderr/auth diagnostics, streaming events, telemetry, and normalized results.
• Fast-model runtime: the quick object-generation path used by extraction repair, merge/dedup, fix gates, and eval judging. It handles structured output, optional tool loops, timeout/retry, telemetry, and normalized usage.

A provider can expose one or both capabilities:

interface RuntimeProvider {
  name: 'claude' | 'pi';
  agent?: AgentRuntime;
  fastModel?: FastModelRuntime;
}

Goals

• Keep Claude as the default provider until Pi reaches parity.
• Add a Pi provider with an agent runtime and, if viable, a fast-model runtime.
• Make the Haiku-style auxiliary calls configurable by provider/model.
• Support choosing which models Pi uses for agent execution and fast model calls.
• Preserve existing Warden output contracts: findings JSON, usage, failed hunk reporting, GitHub reviews/checks, JSONL runs.

Non-goals

• Do not add an AI SDK provider. It was only considered as an implementation detail, not a Warden provider.
• Do not make Warden a general-purpose agent framework.
• Do not expose provider-specific SDK event shapes outside provider/runtime adapters.
• Do not remove Claude support during the Pi rollout.

Proposed Architecture

Provider registry

Add a provider registry that resolves a provider by name and exposes helpers for required capabilities:

getRuntimeProvider('claude')
getAgentRuntime('claude')
getFastModelRuntime('claude')

Unsupported providers or missing capabilities should fail explicitly. This keeps caller errors clear when pi is configured before a capability exists.

AgentRuntime capability

The agent runtime receives:

• systemPrompt
• userPrompt
• repoPath
• skillName
• runtime options such as model, maxTurns, and abort signal

The agent runtime returns:

• final assistant text
• normalized usage
• runtime subtype/status
• normalized errors
• auth error, when surfaced out of band
• captured stderr/diagnostics
• runtime metadata like response id/session id/turn count where available

Implementations:

• claude: wraps @anthropic-ai/claude-agent-sdk and owns Claude-specific event handling, tool names, stderr capture, process options, and telemetry.
• pi: wraps the Pi agent and maps Pi session events/messages into the same Warden result shape.

FastModelRuntime capability

The fast-model runtime supports:

• generateObject(prompt, schema, model?, maxTokens?, maxRetries?)
• generateObjectWithTools(prompt, schema, tools, executeTool, model?, maxIterations?)

Implementations:

• claude: wraps the existing Anthropic Messages/Haiku path, preserving current behavior while allowing model override.
• pi: wraps Pi fast model calls if they can provide reliable structured output and usage metadata.

The fast-model runtime is intentionally smaller than the agent runtime: no repo tool policy, no hunk context, and no provider SDK types in call sites.

Config Shape

Prefer role-based config over global runtime strings:

[defaults.agent]
provider = "claude" # later: "pi"
model = "claude-sonnet-4-6"
maxTurns = 50

[defaults.fastModel]
provider = "claude" # later: "pi"
model = "claude-haiku-4-5"
maxRetries = 5

During migration, keep existing defaults.model, per-skill model, and per-trigger model behavior as legacy shorthand for the agent model. Add explicit agent/fastModel config without breaking existing configs.

For Pi custom providers/models, prefer Pi's existing model configuration. Add project-local config only if Warden needs to pass explicit paths in CI:

[defaults.pi]
agentDir = ".warden/pi"
modelsFile = ".warden/pi-models.json"

Current Slice

• Provider registry and runtime contracts are added.
• Claude exposes both agent and fastModel capabilities.
• [defaults.agent] and [defaults.fastModel] parse and resolve into runner options.
• Actual agent execution resolves getAgentRuntime(configuredProvider).
• Extraction repair, merge, dedup, fix gates, and fix evaluation use getFastModelRuntime(configuredProvider).
• GitHub Action Claude CLI lookup is only required when the configured agent provider is Claude.
• Missing Anthropic auth is no longer rejected before config is known; Claude agent auth is checked once a Claude agent run is selected.

Remaining Work

1. Implement Pi fast-model calls and verify structured-output behavior.
2. Implement Pi agent runtime with a strict read-only tool allowlist.
3. Normalize Pi usage, cost, model id, auth failures, aborts, and turn/tool telemetry.
4. Add Pi-specific auth/config if Warden cannot rely entirely on Pi's existing model configuration.
5. Update docs and examples for provider/model selection.
6. Run evals against Claude and Pi on the same skills/fixtures before switching any default.

Risks / Open Questions

• maxTurns may not map exactly between Claude and Pi. Need to verify Pi's session controls and decide whether Warden emulates a limit.
• Pi tool names and semantics may differ from Claude. The Pi adapter must enforce Warden's read-only policy.
• Usage/cost may be incomplete for custom providers. JSONL should tolerate unknown or zero cost without breaking reports.
• Auth UX needs care: local subscription/OAuth flows and CI API-key flows differ by provider.
• Telemetry attributes currently assume Anthropic in Claude-owned code. Pi should set provider/runtime names dynamically inside its adapter without leaking provider internals.

Acceptance Criteria

• Claude behavior remains unchanged through the provider registry.
• Provider-specific SDK imports are isolated to provider/runtime implementation files.
• Auxiliary calls can choose a provider/model without changing extraction/dedup/fix-gate behavior.
• Pi can run at least one local happy-path Warden skill with read-only tools only.
• Existing tests pass, and new tests cover provider resolution, agent runtime normalization, config resolution, and fast-model runtime behavior.

[Destreyf]

Hey @dcramer this looks interesting, I know this is specifically targetted for pi, but I hope this could maybe be extracted into a more generic implementation, possibly enabling support for opencode via the opencode sdk.

I would be happy to help contribute on that front if it's of interested and once this has been stabilized.

[dcramer]

@Destreyf its actually on the agenda but im told there's going to be some big changes to the opencode sdk

[Destreyf]

I know there's a big 2.0 refactor happening under the hood, but I haven't dug much into what that impacts, it definitely makes sense to try to prevent as much repeated work as possible.

I currently use a mix (for testing purposes) of Claude Code, OpenCode, and Pi.

I have been really enjoying warden so far, it incorporates a lot of idea's I had for my own review agent system, the only thing I think is really missing would be deterministic lookups for potentially affected call sites. I stated prototyping something before finding getsentry/warden that used treesitter to try to map out potential impact surfaces based on the git hunks.

[dcramer]

@Destreyf one big topic has been how we materialize things beyond LLM inference - specifically around things like astgrep and linter-style approaches. Nothing really to show for it yet but @gricha had been thinking about this.

[gricha]

One thing I'm trying to reconcile signal wise for something like this is:

• Do you do it during a warden run, and you just have a really long running analysis that looks through and tries to determine whether a linter would help us - this is tricky, because LLMs like finding something to report, so you may end up with a large swath of low quality lint rules
• Do we just run it independently, say every night/week/month, and inform it both by the existing analysis AND historical warden findings to distill better rules that would've eliminated said issues if they were present originally. I think this is stronger value prop, but complicated the UX..

[Destreyf]

I realize i have derailed this conversation a bit, so if it's better I open this as a separate issue I would be glad to.

I have my own pipeline that I run for local dev & ci that has linter integration pretty solid with per project caches (monorepo), so I personally am not looking for linting specifically.

My ideal setup would be to look at a hunk, determine function/component/class/etc that the hunk impacts, and then using some sort of ast/trace mechanism (treesitter was my plan, astgrep might actually be better) to locate all places that function/component/class might get called, this would be a shortlist for impact, granted an agent can do this pretty well on it's own, but my thought process was to reduce the amount of "surface area" that is LLM inference driven, and provide more relevant contextual data to catch more edge cases.

My thinking around this that if we can provide the agent "Here's the hunk + x lines each side of it" and also "here's all the known call sites", it could possibly reduce some of the tool calling/searching involved.