ADR 0002 llm provider portability

ADR-0002: LLM provider — portability layer (default Amazon Bedrock)

Status: Accepted (2026-07-22)

Context

Every agent in the pipeline (Supervisor + three sub-agents) reasons with Claude. The agent runs nightly inside AWS CodeBuild (ADR-0001) and reads sensitive infrastructure data — Terraform state (which can contain secrets), live account configuration, and Well-Architected Tool answers. The hosting choice therefore turns on data governance, authentication model, auditability, and prompt-injection controls, not just model quality.

Three ways to reach Claude are viable on AWS:

• Amazon Bedrock (partner-operated) — deepest AWS-native controls (PrivateLink, Bedrock Guardrails, native CloudTrail, KMS-encrypted invocation logs), IAM auth, anthropic.-prefixed model IDs. Serves a subset of features and lags first-party on new model/feature launches.
• Claude Platform on AWS (Anthropic-operated on AWS) — AWS IAM / SigV4 auth and Marketplace billing with same-day feature parity and bare model IDs.
• Direct Anthropic API — full features, but API-key auth and egress outside the AWS trust boundary.

We do not want to hard-couple the agent to one provider: it invites lock-in, leaves no disaster-recovery path if a provider/region is impaired, and turns Bedrock's feature-lag into a rewrite rather than a switch.

Decision drivers

• Data governance — sensitive input must stay in-region, off the public internet, and out of any training set.
• Auth model — reuse the CodeBuild task role; no API key to store or rotate (composes with ADR-0007 least-privilege).
• Auditability — a durable, tamper-evident record of every model invocation for the AI-governance controls.
• Prompt-injection defence — Agent 2 feeds untrusted HCL into the model.
• No lock-in / DR — provider swappable without touching agent logic.

Decision

Reach Claude through a thin provider portability layer, not a direct client.

• A get_client(provider) factory constructs the correct SDK client (AnthropicBedrockMantle for Bedrock, AnthropicAWS for Claude Platform on AWS). Both expose the identical messages.create / .stream surface, so agent code is provider-agnostic.
• A resolve_model_id(logical_name, provider) helper maps a logical model name to the provider-specific ID (claude-sonnet-5 ↔ anthropic.claude-sonnet-5).
• Default provider: Amazon Bedrock. It gives the strongest data-isolation story (PrivateLink keeps sensitive state off the internet), native Bedrock Guardrails for prompt-injection/PII, and CloudTrail + model-invocation logging for the audit trail. The agent needs none of the parity-gated features, so the subset does not bite.
• Provider is selected per run via an environment variable / SSM parameter read at CodeBuild start — Claude Platform on AWS is a config flip, not a code change. Selection is not per-request; one code path runs per invocation.
• Program to the common feature subset (tool use, structured outputs, adaptive/extended thinking, manual prompt caching). Anything Platform-on-AWS-only (automatic prompt caching, server-side web search, Batches) is a deliberate choice that drops Bedrock support and must be flagged.

Model selection (supersedes the source doc's "Claude 3.5 Sonnet"): claude-sonnet-5 for the three analysis sub-agents (throughput/cost on high-volume HCL scanning) and claude-opus-4-8 for the Supervisor's orchestration reasoning — expressed as logical names and resolved per provider.

Consequences

• No API key anywhere — the CodeBuild role is granted bedrock:InvokeModel (or the Platform-on-AWS IAM actions); nothing to rotate or leak.
• Sensitive state never leaves AWS — Bedrock over PrivateLink; data is not used for training.
• Every invocation is audited — CloudTrail + Bedrock model-invocation logging to S3/KMS satisfies the governance doc's AI-decision audit requirement.
• Provider swap is a config value — enabling DR and removing lock-in; the agent code is untouched.
• Portability is a discipline, not a guarantee — it holds only while the code stays on the common subset. Using a Platform-on-AWS-only feature silently breaks Bedrock. This constraint is load-bearing and must be enforced in review.
• Small abstraction cost — the factory + model-ID resolver are the only provider-aware code; everything else is a shared surface.

Alternatives considered

Option	Verdict	Why
Portability layer, default Bedrock	Chosen	Best data-isolation/guardrails/audit story by default; no lock-in; provider is a per-run config flip; cheap because the SDK surface is identical across providers.
Hard-commit to Bedrock	Rejected	Same runtime benefits but no DR path and turns Bedrock's feature-lag into a rewrite.
Hard-commit to Claude Platform on AWS	Rejected	Full parity, but forgoes Bedrock's PrivateLink/Guardrails/native-CloudTrail depth as the default posture for sensitive data. Kept as the selectable alternative.
Direct Anthropic API	Rejected	API-key management and egress outside the AWS trust boundary weaken the data-governance and least-privilege story.
Self-hosted open model	Rejected	Cannot host Claude; GPU/ops burden is disproportionate for a nightly batch.
Runtime fallback / capability routing	Rejected (now)	Per-request switching adds idempotency and partial-state complexity unjustified for a nightly batch; revisit if resilience or parity-gated features demand it.

Revisit trigger

• Move the default to Claude Platform on AWS if the agent needs server-side web search (live WA guidance), Batches (multi-account fan-out cost savings), automatic prompt caching, or a model not yet on Bedrock.
• Introduce runtime fallback (ADR to follow) only if nightly availability SLAs require surviving a single-provider/region outage mid-run.

References

• HLD diagram: ../diagrams/nwaf-agent-hld.png (v1.1)
• Related: ADR-0001 (compute — CodeBuild), ADR-0003 (detection & orchestration — the LLM's bounded reconciliation role), the least-privilege IAM ADR, the RAG-store ADR (provides the WA context, so no server-side web search is needed on the default provider)