Add fixed-capacity API workload isolation pools

[Alexey Soldatchenko (soldatchenko)]

This adds an opt-in workload-isolation capability for Kubernetes data planes. It creates dedicated API Services for ingestion and background work, while keeping the existing API Service as the default pool.

For existing deployments, upgrading to a chart release that includes this change does not enable workload isolation or change the existing API routing. When api.workloadIsolation.enabled: false, the chart renders no additional API pools and preserves the default API Deployment and Service behavior.

For a new deployment, enabling isolation creates dedicated ingest and background API Deployments and Services. The ingress or gateway must own the public route contract below.

For an existing deployment, use two releases:

1. Enable api.workloadIsolation.enabled with api.workloadIsolation.brainstoreAiProxyToBackground: false. Keep classified public paths on the default Service; when using the chart's Istio VirtualService, keep virtualService.workloadIsolation.enabled: false. Verify the dedicated workloads and Services are healthy.
2. Configure the external ingress or gateway routes below and set brainstoreAiProxyToBackground: true in a later release. When the chart owns an Istio VirtualService, also set virtualService.workloadIsolation.enabled: true. This moves the classified public paths and Brainstore's internal scoring/function traffic to the isolated pools.

This feature uses fixed replica counts; it does not configure HPA or other autoscaling behavior.

Route contract

The product-owned, machine-readable contract is braintrust/files/contracts/api-workload-isolation-routes.yaml.

An external ingress or gateway must keep braintrust-api as the fallback backend and route this path set to the dedicated Services. The chart-managed Istio VirtualService can opt into this same contract with virtualService.workloadIsolation.enabled: true; generated routes are ordered before the user's virtualService.http rules.

Pool	Paths
braintrust-api (default)	All requests not matched by an explicit ingest or background route
braintrust-api-ingest	POST /logs3, POST /otel/v1/traces, POST /attachment, POST /attachment/status
braintrust-api-background	POST /v1/eval, POST /v1/eval/*, POST /function/eval, POST /function/sandbox, POST /function/use, POST /function/invoke-async-batch, POST /function/insert-functions, POST /automation/logs/trigger, plus all methods for /v1/proxy/chat/completions and /v1/proxy/responses

The Istio implementation preserves those AWS path and method boundaries: it renders exact /v1/eval plus prefix /v1/eval/, so it does not classify a future /v1/evaluate route. GKE Ingress is path-based rather than method-aware, so its matching paths are deliberately isolated for all HTTP methods; Kubernetes Prefix: /v1/eval has the same exact-and-subpath boundary.

Rollback

First route the isolated public paths and Brainstore's internal proxy target back to the default API Service, then verify it is serving them. For a chart-managed Istio VirtualService, disable virtualService.workloadIsolation.enabled as part of that reroute. Only then disable API workload isolation in the chart.

Validation

• ./test.sh passed: 286 Helm unit tests, Azure/GCP/AWS/minimal renders, and strict lint.
• The default render remains byte-for-byte unchanged from main when workload isolation is disabled.
• Packaged-chart rendering verified the Istio VirtualService consumes the canonical contract, honors configured isolated Service names, preserves the AWS method rules, and retains the existing fallback route after generated routes.
• terraform test passed in the GKE app module: 8 passed, including default routing, staged routing (including the retained Brainstore default target), and invalid route-without-pools coverage.
• Earlier GKE sandbox validation verified distinct healthy default, ingest, and background backends, controlled background-pool failure isolation, and successful ingestion write/read-back recovery.

[Alexey Soldatchenko (soldatchenko)]

Validation completed

• ./test.sh passed: 282 Helm unit tests, Azure/GCP/AWS/minimal renders, and strict lint.
• Deployed in a real GKE sandbox and verified distinct healthy default, ingest, and background backends with the expected path ownership.
• During a controlled background-pool outage, the background route became unavailable as expected while the default health route remained available. All 30 concurrent ingestion writes were acknowledged and subsequently read back.
• Restored the background pool and confirmed the workloads and backends recovered healthy.

[Alexey Soldatchenko (soldatchenko)]

This PR intentionally ports the workload-isolation model introduced in AWS PR #287.

The k8s implementation uses Deployments, Services, and GKE Ingress rather than ECS services and ALB target groups, but preserves the same traffic contract: ingestion is isolated from scoring/evals/proxy background work, while all other API traffic remains on the default pool.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1304354ef5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "Codex (@codex) review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".

[brianvans (brianvans)]

Two last comments re: formatting of the istio virtualservice but not blocking