feat(llm-provider): add BYO-CLI + Cloud-Paid synthesis paths

[Gradata]

Summary

Extend the unified LLMProvider abstraction with three BYO modes for meta-rule synthesis:

• BYO-Key — Anthropic / OpenAI / generic OpenAI-compat (existing, lightly refactored)
• BYO-CLI — subprocess-shell to claude / codex / gemini, reusing the user's Max-plan OAuth or ChatGPT Plus subscription. No API key required.
• Cloud-Paid — POST to a Gradata-hosted relay; the server holds the LLM key and billing flows through the Gradata subscription.

This lets Claude Pro / ChatGPT Plus subscribers run meta-rule synthesis without spending API credits, and Gradata Cloud customers offload synthesis to a managed relay.

Changes

• enhancements/llm_provider.py
  • New CLIProvider, GradataCloudProvider, GoogleGemmaProvider.
  • Per-instance circuit breaker (3 consecutive fails → open for 5 min, half-open recovery).
  • Centralized SSRF require_https() guard + sanitize_lesson_content pre-prompt hook in the base class.
  • auto factory resolution: cli → anthropic → openai → cloud → gemma → None.
  • Returns None (instead of raising) when auto finds nothing — caller falls back to deterministic synthesis.
• enhancements/meta_rules.py: _try_llm_principle routes through the unified provider when GRADATA_LLM_PROVIDER ∈ {cli, cloud, auto} is explicitly set. Legacy GRADATA_LLM_KEY+GRADATA_LLM_BASE and GRADATA_GEMMA_API_KEY paths preserved so existing tests (which patch internal helpers) stay green.
• tests/test_llm_provider.py: 9 new tests.

Test plan

pytest tests/test_meta_rules.py tests/test_agentic_synthesis.py \
       tests/test_llm_synthesis.py tests/test_rule_synthesizer.py \
       tests/test_llm_provider.py
58 passed in 5.62s

• 44 existing meta-rule / agentic-synthesis / llm-synthesis tests — green.
• 5 rule_synthesizer tests — green.
• 9 new llm_provider tests — green.
• 2 test_dualwrite_atomicity.py failures observed are pre-existing on main (kill-9 + concurrency tests, unrelated to this PR).

Layering check

All changes live inside Layer 1 (enhancements/). No upward imports introduced. No imports from cloud/. No new required dependencies — anthropic and openai SDKs remain optional.

Risk

• Public function signatures unchanged. provider="anthropic" kw on llm_synthesize_rules retained for back-compat.
• Default behavior unchanged when no env var is set: the legacy resolution (GRADATA_LLM_KEY+GRADATA_LLM_BASE → Gemma) still fires.
• Opt-in to BYO-CLI / Cloud requires an explicit GRADATA_LLM_PROVIDER=cli|cloud|auto.

[greptile-apps]

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 6c80707e-5ccd-46d1-b00c-c2f4acf73e8b

📥 Commits

Reviewing files that changed from the base of the PR and between eb465b3 and 29df515.

📒 Files selected for processing (3)

• Gradata/src/gradata/enhancements/llm_provider.py
• Gradata/src/gradata/enhancements/meta_rules.py
• Gradata/tests/test_llm_provider.py

---

📝 Walkthrough

Summary

• New LLM Provider Modes: Extends unified LLMProvider abstraction with three BYO synthesis paths:

  • BYO-CLI: subprocess integration for local/installed CLIs (claude, codex, gemini)
  • Cloud-Paid: HTTP relay to Gradata-hosted server with centralized key management
  • Auto: intelligent factory resolution with precedence (CLI → Anthropic → OpenAI → Cloud → Gemma)

• Reliability & Circuit Breaker: Adds per-instance circuit breaker (opens after 3 consecutive failures, 5-min cooldown, half-open recovery) to prevent cascading failures

• Security Hardening: Centralizes SSRF protection via require_https() guard; adds optional sanitize pre-prompt hook for content validation

• New Public Classes: CLIProvider, GradataCloudProvider, GoogleGemmaProvider for extensible provider implementations

• Breaking Changes:

  • get_provider() return type changed from LLMProvider → LLMProvider | None (now returns None when no backend available)
  • complete() signature adds optional sanitize: bool = False parameter

• Backward Compatibility: Legacy GRADATA_LLM_KEY / GRADATA_GEMMA_API_KEY flows preserved; opt-in via GRADATA_LLM_PROVIDER env var

• Test Coverage: 9 new tests cover CLI happy/failure paths, circuit breaker behavior, cloud endpoint integration, and factory auto-resolution logic; all 58 tests pass

Walkthrough

This PR refactors the LLM provider abstraction from simple implementations to a unified base class with centralized request control, including per-instance circuit-breaker logic and optional pre-prompt sanitization. Four new provider modes are added (CLI, cloud relay, Gemma, and auto-selection), existing providers are adapted to the new pattern, and meta-rule synthesis integrates the unified provider factory.

Changes

Unified Provider Abstraction & Auto-Resolution

Layer / File(s)	Summary
Base Abstraction & Circuit Breaker Gradata/src/gradata/enhancements/llm_provider.py (lines 43–113)	LLMProvider refactored from abstract to a concrete wrapper. Adds per-instance circuit-breaker state (consecutive-failure tracking, temporary blocking with cooldown). complete(...) now centralizes failure recording and optional prompt sanitization via sanitize parameter. Request execution delegated to new abstract _complete_impl(...).
Existing Provider Refactoring Gradata/src/gradata/enhancements/llm_provider.py (lines 120–215)	AnthropicProvider, OpenAIProvider, and GenericHTTPProvider updated to implement _complete_impl(...) instead of overriding complete(...). Shared _openai_complete(...) helper adjusted to use new wrapper. Output validation threshold increased from 500 to 5000 characters across all OpenAI-compatible implementations.
New Provider Implementations Gradata/src/gradata/enhancements/llm_provider.py (lines 217–379)	Added CLIProvider (auto-detects/executes CLI binaries via PATH or explicit name). Added GradataCloudProvider (POST to /meta-rules/synthesize endpoint with bearer auth, handles HTTP 402/429 gracefully). Added GoogleGemmaProvider (native REST call to Gemma generateContent with API key).
Factory & Auto-Resolution Gradata/src/gradata/enhancements/llm_provider.py (lines 381–444)	_PROVIDERS registry extended with cli, cloud, gemma. New _resolve_auto() implements precedence order (PATH-based CLI, Anthropic, OpenAI, cloud, Gemma). Updated get_provider(...) to default to auto, return LLMProvider | None, catch provider-constructor ValueErrors, and log graceful degradation.
Meta-Rules Integration Gradata/src/gradata/enhancements/meta_rules.py (lines 776–903)	Added _get_meta_provider() helper to fetch unified provider. Modified _try_llm_principle() to route synthesis through unified provider when GRADATA_LLM_PROVIDER is set to cli, cloud, or auto; falls back to legacy credential-based paths on None or empty response.
Tests & Validation Gradata/tests/test_llm_provider.py (lines 1–201)	Comprehensive coverage: CLIProvider success/failure and circuit-breaker behavior, GradataCloudProvider HTTP payload/response parsing and error handling, auto-resolution precedence with environment/PATH mocking, and unknown-provider error case.

Sequence Diagram

sequenceDiagram
    participant Caller
    participant LLMProvider
    participant CircuitBreaker
    participant _complete_impl
    participant Backend
    
    Caller->>LLMProvider: complete(prompt, sanitize=False)
    LLMProvider->>CircuitBreaker: Is circuit open?
    alt Circuit open (cooldown active)
        CircuitBreaker-->>LLMProvider: Yes, return None
        LLMProvider-->>Caller: None
    else Circuit closed
        CircuitBreaker-->>LLMProvider: No, proceed
        LLMProvider->>LLMProvider: Optionally sanitize prompt
        LLMProvider->>_complete_impl: _complete_impl(prompt, ...)
        _complete_impl->>Backend: Invoke provider backend (CLI/HTTP/SDK)
        alt Backend success
            Backend-->>_complete_impl: Response text
            _complete_impl-->>LLMProvider: text (validated)
            LLMProvider->>CircuitBreaker: Record success (reset counter)
            LLMProvider-->>Caller: text
        else Backend failure
            Backend-->>_complete_impl: Error/timeout/None
            _complete_impl-->>LLMProvider: None
            LLMProvider->>CircuitBreaker: Record failure (increment counter)
            alt Threshold reached (3 failures)
                CircuitBreaker->>CircuitBreaker: Open circuit for cooldown (60s)
            end
            LLMProvider-->>Caller: None
        end
    end

Loading

sequenceDiagram
    participant Caller
    participant get_provider
    participant _resolve_auto
    participant Environment
    participant PathCheck
    
    Caller->>get_provider: get_provider('auto', **kwargs)
    get_provider->>_resolve_auto: Resolve 'auto' mode
    _resolve_auto->>Environment: Check ANTHROPIC_API_KEY
    _resolve_auto->>Environment: Check OPENAI_API_KEY
    _resolve_auto->>Environment: Check GRADATA_API_KEY
    _resolve_auto->>Environment: Check GOOGLE_API_KEY
    _resolve_auto->>PathCheck: which claude / codex / gemini?
    alt CLI found in PATH
        PathCheck-->>_resolve_auto: Yes
        _resolve_auto-->>get_provider: CLIProvider(...)
    else Anthropic env var set
        Environment-->>_resolve_auto: ANTHROPIC_API_KEY
        _resolve_auto-->>get_provider: AnthropicProvider(...)
    else OpenAI env var set
        Environment-->>_resolve_auto: OPENAI_API_KEY
        _resolve_auto-->>get_provider: OpenAIProvider(...)
    else Gradata cloud env var set
        Environment-->>_resolve_auto: GRADATA_API_KEY
        _resolve_auto-->>get_provider: GradataCloudProvider(...)
    else Gemma env var set
        Environment-->>_resolve_auto: GOOGLE_API_KEY
        _resolve_auto-->>get_provider: GoogleGemmaProvider(...)
    else No match
        _resolve_auto-->>get_provider: None
    end
    get_provider-->>Caller: Provider or None

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

• Gradata/gradata#97: Both PRs touch LLM-based meta-rule synthesis; #97 wires Gemma-native synthesis and credential resolution, while this PR adapts meta_rules to use the new unified LLMProvider factory and auto-resolution routing.

Suggested labels

feature, breaking-change

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/llm-provider-byo-cli-cloud

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 OpenGrep (1.20.0)

OpenGrep fatal error (exit code 2):
┌──────────────┐
│ Opengrep CLI │
└──────────────┘

�[32m✔�[39m �[1mOpengrep OSS�[0m
�[32m✔�[39m Basic security coverage for first-party code vulnerabilities.

�[1m Loading rules from local config...�[0m
[00.39][ERROR]: Error: exception Glob.Lexer.Syntax_error("malformed glob pattern: missing ']'")
Raised at Glob__Lexer.syntax_error in file "libs/glob/Lexer.mll", line 8, characters 2-26
Called from Glob__Lexer.__ocaml_lex_token_rec in file "libs/glob/Lexer.mll", line 29, characters 26-53
Cal

---

_{Review rate limit: 4/5 reviews remaining, refill in 12 minutes.}

_{Comment @coderabbitai help to get the list of available commands and usage tips.}