ADR 0024 provider framework evaluation

ADR-0024: retain native provider adapters after framework evaluation

• Status: Accepted
• Date: 2026-07-19
• Scope: Provider abstraction, caching/batching evaluation, and future dependency decisions

Context

AncestryLLM already has explicit provider adapters, endpoint policy, consent authorization, structured-output validation, timeout handling, and encrypted privacy-minimal audit metadata. LangChain and LiteLLM provide useful wrappers, normalization, caching, batching, and routing primitives, but each introduces additional dependencies and policy boundaries.

The reproducible evaluation is generated by scripts/benchmark_provider_frameworks.py. It is intentionally dependency-free, deterministic, network-free, and based on fictional metadata; it is an architectural comparison, not a claim of measured provider latency.

Decision

Retain the native provider adapters implemented for 0.3.0. Defer LangChain and LiteLLM adoption until a concrete capability cannot be implemented safely behind the existing contract and the added dependency/security surface is accepted through a new ADR.

Dimension	Native adapters	LangChain	LiteLLM
Dependency footprint	Small explicit contract and existing SDK extras	Larger abstraction and transitive dependency surface	Proxy/client plus normalization layer
Provider control	Full lifecycle, timeout, retry, and stream control	Wrappers may obscure provider-specific behavior	Routing can hide rerouting and provider identity changes
Caching/batching	Must be implemented explicitly and policy-aware	Rich primitives require consent/audit integration	First-class options require retention and consent controls
Consent enforcement	Application service authorizes before calls	Must be proven across wrapper boundaries	Proxy defaults must not bypass local policy
Endpoint policy	Central allowlist and loopback rules	Requires enforcement around construction	Requires endpoint, routing, and credential verification
Structured output	Existing schema validation and adapter-specific behavior	Broad integrations require validation at the boundary	Normalization gaps must fail safely
Observability	Encrypted ledger controls fields and redaction	Callback payloads need redaction review	Proxy telemetry must reconcile with local audit metadata
Security surface	Smallest trusted surface	Adds chains, memory, tools, and prompt abstractions	Adds proxy, routing, and possible remote persistence
Maintenance	Existing project ownership and tests	Framework/provider compatibility matrix	Provider and proxy release coordination

Required conditions for reconsideration

A future adoption proposal must identify a capability gap, preserve explicit provider selection and consent-before-network invariants, maintain endpoint allowlisting, validate structured output, retain privacy-minimal audit records, and include an offline test mode. It must also quantify dependency, upgrade, security, and provider-identity tradeoffs rather than relying on framework feature counts.

Consequences

The project keeps a smaller trusted runtime surface and direct control over provider behavior. Operational profiles, bounded scheduling, shared Ollama clients, and process-local exact-result caching are implemented behind the native typed contracts; future batching or persistent-cache work remains project-owned. The evaluation does not prohibit optional research experiments; it prohibits adding these frameworks as runtime requirements based only on this comparison.