Repository navigation
v0.3.8 — named auto pools (auto = cloud, auto_home = home)
Named auto pools
[auto_models.<key>] declares extra rotation pools next to the default [auto_model], each answering to its own sentinel model name. A deployment can now keep the rotation sets disjoint — auto over cloud vendors, auto_home over the home GPUs — instead of one pool that mixes machines which are off half the day with cloud APIs.
[auto_model]stays the default pool (sentinelauto), so an existing single-pool config is unchanged. A named pool's sentinel is its table key unless the block setsname.- The route's
providernames the pool anauto_model = trueroute rotates over when it matches one; otherwise the first declared pool serves it. RequestCtx.auto_poolpins the pool for the whole request: the sentinel that entered the request also bounds every later failover hop, so a request can no longer drift from one rotation set into another (previously oneautopool was searched for any hop's provider name).GET /v1/modelslists every enabled pool sentinel (owned_by: maxllm-auto).- Pool names must be non-empty and unique; an auto route naming a missing or disabled pool fails validation at startup.
Verification
cargo test --workspace: all green (211 translate tests etc.).cargo test -p maxllm-gateway --test integration: 107 passed, 4 ignored (2 new: pools keep their sets apart; a body-onlyauto_homesentinel is adopted into its named pool).- Binary: musl static, sha256
f80d23577e150935cf3a788149ac5afbd508a220d68ee5919157b6b5a37e3be5.