Wire Claude as primary LLM provider (CLAUDE-A/B/C)

[nuniesmith]

Summary

Resolves the P0 "Wire Claude as Primary Provider" block in TODO.md
(CLAUDE-A, CLAUDE-B, CLAUDE-C). Until now AnthropicClient in
crates/api/src/providers/anthropic.rs was fully built but never
reached from the proxy hot path — every request still dispatched to
Grok. This PR wires Claude in as the primary backend, with two-tier
routing and prompt-cache token accounting.

What changed

Routing & dispatch

• ModelTarget::Claude { model, tier } added alongside Local /
  Remote in src/model_router.rs.
• ClaudeTier { Planner, Executor } enum plus a TaskKind::tier()
  helper so each TaskKind maps cleanly to one of the two slugs.
• ModelRouterConfig gained planner_model, executor_model, and
  anthropic_enabled; route() and on_local_failure() honour
  Anthropic-first ordering when enabled.
• src/api/proxy.rs::dispatch() now has a third arm that calls
  AnthropicClient::send_message() via a small dispatch_claude
  helper, and the streaming handler synthesises a single-delta SSE
  burst from the same blocking call (native streaming via
  stream_message left as a follow-up).
• route_from_model_field picks the right Claude tier from
  claude-opus* / claude-sonnet* slugs (and the anthropic/
  prefix used by OpenClaw).
• handle_list_models advertises claude-opus-4-7 and
  claude-sonnet-4-6 (plus openai/-prefixed aliases).

Prompt caching

• RepoAppState carries an Option<Arc<AnthropicClient>> built
  once at startup in src/server.rs with
  .with_prompt_cache(PromptCache::new("rustcode-proxy")) so the
  cache session persists across requests.
• RaMetadata (and the cached CachedProxyResponse) now surface
  cache_creation_input_tokens and cache_read_input_tokens
  populated from MessageResponse::usage. Both are
  #[serde(skip_serializing_if = "Option::is_none")] so non-Claude
  responses keep their existing payload shape.

Config / docs

• ModelConfig reads ANTHROPIC_API_KEY, RC_PLANNER_MODEL,
  RC_EXECUTOR_MODEL. A new .env.example lists every variable.
• crates/api/src/providers/mod.rs alias "opus" updated from
  claude-opus-4-6 to claude-opus-4-7.
• README.md config table + routing diagram updated.
• TODO.md marks CLAUDE-A/B/C done with implementation notes.

Ride-along fix

• src/config.rs previously declared task_executor and
  task_watcher fields but never initialized them in load() or
  Default, and TaskWatcherConfig was missing Debug, Clone —
  the crate could not compile without this. Both fields are now
  #[serde(skip, default)] and TaskWatcherConfig derives the
  missing traits.

Test plan

• cargo check --workspace clean on a machine with
  cdn.pyke.io reachable (the sandbox here blocks the
  ort-sys prebuilt download, so I could only verify the
  api and runtime crates locally)
• cargo test -p api passes
• cargo test -p rustcode --lib model_router covers the three
  new tier-routing tests (claude_planner_for_review_and_architecture,
  claude_executor_for_scaffold, task_kind_tier_mapping)
• End-to-end smoke: start rc-app with only
  ANTHROPIC_API_KEY set, send a /v1/chat/completions
  request with model: "auto" and a planner-shaped prompt,
  confirm x_ra_metadata.task_kind is ArchitecturalReason
  and the echoed model is Opus
• Cache-hit smoke: send the same request twice; confirm
  cache_read_input_tokens > 0 on the second response
• Grok fallback: unset ANTHROPIC_API_KEY, set XAI_API_KEY,
  confirm classic Grok behaviour still works

Outstanding for follow-up

• Verify the Opus 4.7 slug live (curl https://api.anthropic.com/v1/models)
  per DEPLOY-C before final release.
• Native streaming via AnthropicClient::stream_message (currently
  a single-burst delta, matching the existing Grok path).
• Plumb cache token observation through StreamChunk so streaming
  responses also surface cache_*_input_tokens.

---

Generated by Claude Code

Summary by CodeRabbit

Release Notes

• New Features

  • Added Claude/Anthropic LLM provider support with automatic routing based on task type
  • Implemented two-tier model selection: planner models for architecture/review tasks, executor models for implementation tasks
  • Added prompt caching support for improved performance and reduced token usage

• Documentation

  • Added environment configuration template with Anthropic API key and model override options
  • Updated API documentation describing Claude tier routing and new cache token metadata in responses

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 560a088b-f045-411e-8b75-0caca80b91ba

📥 Commits

Reviewing files that changed from the base of the PR and between f293527 and b6234e4.

📒 Files selected for processing (10)

• .env.example
• README.md
• TODO.md
• crates/api/src/providers/mod.rs
• src/api/proxy.rs
• src/api/repos.rs
• src/config.rs
• src/model_router.rs
• src/server.rs
• src/task_watcher.rs

---

📝 Walkthrough

Walkthrough

This PR integrates Anthropic Claude as a primary LLM provider alongside existing Ollama and Grok routing. Claude dispatch is added to the proxy with two-tier task routing (planner for architecture/review, executor for scaffolding/todo), prompt-cache token tracking, and full streaming support. Configuration, routing logic, and documentation are updated throughout.

Changes

Claude Integration with Two-Tier Routing

Layer / File(s)	Summary
Configuration and Anthropic client initialization src/config.rs, src/server.rs, src/api/repos.rs, src/task_watcher.rs	ModelConfig gains anthropic_api_key, planner_model, and executor_model fields loaded from ANTHROPIC_API_KEY, RC_PLANNER_MODEL, and RC_EXECUTOR_MODEL. At startup, run_server() conditionally builds an AnthropicClient with an attached PromptCache and passes it into RepoAppState.
Model routing with Claude tiering src/model_router.rs	New ClaudeTier enum classifies tasks as Planner (Architectural/Review) or Executor (Scaffold/Todo/etc.), added via TaskKind::tier(). ModelTarget gains a Claude { model, tier } variant. ModelRouter::route() now prefers Claude when enabled, selecting model by tier; on_local_failure falls back to Claude (executor) instead of immediately using Grok. Tests validate tier selection and mapping.
Proxy endpoint Claude dispatch and token metadata src/api/proxy.rs	RaMetadata and cached responses now carry optional cache_creation_input_tokens and cache_read_input_tokens from Anthropic prompt-cache counters. Non-streaming and streaming paths both dispatch to Claude via new dispatch_claude and synthesize Claude responses into OpenAI-compatible format. route_from_model_field maps anthropic/* and claude-* slugs to the appropriate tier and Claude target. /v1/models advertises Claude tier aliases. Cache keys include Claude namespace (tier:model) for proper isolation.
Documentation and configuration examples .env.example, README.md, TODO.md, crates/api/src/providers/mod.rs	New .env.example provides a complete environment template including Anthropic, Grok, Ollama, database, GitHub, and server configuration. README is updated to describe Claude as the primary routing path when configured, with fallback behavior. TODO items for Claude integration are marked completed. Anthropic "opus" alias is bumped to "claude-opus-4-7".

Sequence Diagram(s)

sequenceDiagram
  participant Client as Client
  participant Proxy as /v1/chat/completions
  participant Router as ModelRouter
  participant Dispatch as dispatch()
  participant Claude as AnthropicClient
  Client->>Proxy: POST with model="claude-opus-4-7", task_kind="architecture"
  Proxy->>Router: route(TaskKind::Architecture)
  Router->>Router: Prefer Claude (anthropic_enabled=true)
  Router-->>Proxy: ModelTarget::Claude { model: "claude-opus-4-7", tier: Planner }
  Proxy->>Dispatch: dispatch(ModelTarget::Claude, messages)
  Dispatch->>Claude: send_message() with prompt_cache
  Claude-->>Dispatch: MessageResponse { usage, content: [Text blocks] }
  Dispatch->>Dispatch: extract_text() and compute cache tokens
  Dispatch-->>Proxy: DispatchOutcome { reply, cache_creation_input_tokens, cache_read_input_tokens }
  Proxy->>Proxy: build_oai_response() with RaMetadata::cache_* fields
  Proxy-->>Client: OpenAI response { choices, x_ra_metadata { cache_creation_input_tokens, ... } }

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Poem

A rabbit hops through routing tiers, ✨
Planner thoughts and executor gears,
Claude joins the fray with cache so deep,
Token counts that promises keep. 🐰
Two-tier wisdom, streaming dreams,
All flowing through the proxy streams!

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch claude/review-and-update-todo-iNWcd

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}