Skip to content

v2.0.0: adaptive concurrency limits

Choose a tag to compare

@SAY-5 SAY-5 released this 03 Sep 19:58
· 21 commits to main since this release

Every upstream replica now carries an adaptive in-flight limit that grows while calls complete within the replica's no-load latency and is cut back on timeouts, resets, 5xx or a window whose average latency spikes. The pool skips replicas that are at their limit, so a slow replica stops receiving new work while its siblings carry it, and a request is only shed with 503 and Retry-After when no healthy replica has a free slot. Three new metrics expose the limit, the in-flight count and the shed total, and the chaos summary prints shedding alongside retries and failovers. 75 tests.