v0.3.0 — routing strategies, Bedrock SigV4, virtual keys, per-key rate limits, provider retries

Highlights

Routing

• Latency-, cost- and least-connections strategies — latency_based, cost_based and least_connections are now implemented, joining the existing round-robin / random / weighted strategies. The per-provider in-flight count is exposed as maxllm_provider_active_requests_total.

Providers

• AWS Bedrock SigV4 — requests to Bedrock are now signed with AWS SigV4 and the model name is routed into the invocation URL. Adds the aws_sigv4 signing module in maxllm-translate.
• Same-provider retry — num_retries and retry_backoff_ms are now wired: a transient 5xx is retried on the same provider (with exponential backoff) before the gateway fails over. New metric: maxllm_provider_retries_total{provider,status}.

Multi-tenant gateway

• Virtual keys — API keys minted through the admin API are now enforced at the gateway: 401 for unknown/disabled keys, 402 when the spend budget is exhausted, and a model allow-list per key.
• Per-key rate limits — each virtual key can carry its own requests_per_minute / tokens_per_minute budget, enforced before the request is proxied.
• Top-level [auth] and [rate_limit] config sections are now honoured (previously silently ignored) and synthesised into the hot-state plugin chain.

Server

• [server] socket knobs — tcp_reuseport and tcp_fastopen are applied to the listening socket (SO_REUSEPORT / TCP_FASTOPEN).

CLI

• make run / make run-debug and the README now use the real argument syntax (maxllm start --config …, maxllm config check|providers).
• maxllm init writes a starter config with global_plugins at the top level (previously nested under [server], which silently disabled authentication) and the correct requests_per_minute field name.

Quality gates

• cargo clippy --workspace --all-targets -- -D warnings and cargo fmt --all -- --check are both clean — the CI workflow's clippy and format gates now pass.
• Full test suite green: 45 admin / 9 config / 96 gateway / 93 integration (+4 live, ignored by default) / 139 plugin / 177 translate.

Breaking / migration notes

• None — existing configs keep working; the previously ignored top-level [auth] / [rate_limit] sections now take effect, so re-check them if you had leftover defaults.

Full changelog: v0.2.0...v0.3.0