v0.4.0 — Resilience

throttle-net v0.4.0 — Resilience

The pieces that decide whether to call at all, and how callers wait. v0.4.0 adds the resilience layer on top of the limiters: a circuit breaker that sheds load when a downstream is unhealthy, a bounded deadline-aware queue for orderly waiting, and an exact sliding-window-log limiter for when a token bucket's boundary burst is unacceptable. No breaking changes.

What is throttle-net?

A general-purpose outbound throttling and resilience library. Where rate-net protects your service from being overwhelmed (inbound), throttle-net protects your service from overwhelming the downstreams it calls — and from being banned by them. It paces outbound work, composes limits across dimensions and scopes, retries transient failures, and — as of this release — fails fast when a dependency is sick and queues callers fairly when it is merely busy.

What's new in 0.4.0

CircuitBreaker — fail fast on failures

A limiter paces requests; a breaker stops them. Wrap any Limiter in a breaker and, once the downstream produces enough failures, it trips open and sheds requests immediately — without consuming the wrapped limiter's tokens — giving the dependency room to recover. After a cooldown it goes half-open, admits a trial or two, and closes again on success.

use std::time::Duration;
use throttle_net::{CircuitBreaker, Throttle, Trip};

let breaker = CircuitBreaker::builder()
    .trip(Trip::Consecutive(5))
    .cooldown(Duration::from_secs(10))
    .build(Throttle::per_second(100));

match breaker.acquire().await {
    Ok(permit) => {
        // ... call the downstream ...
        if downstream_ok { permit.success() } else { permit.failure() }
    }
    Err(_shed) => { /* breaker open: fail fast */ }
}

Three trip conditions cover the common policies: Consecutive(n) failures, a failure Ratio over a rolling window of calls, and Windowed failures within a rolling time window. Outcomes are reported through a Permit whose drop counts as a failure, so an early return or a panic is treated conservatively. Behind the circuit-breaker feature. State transitions are verified by a proptest against a reference model; half-open recovery and load-shedding by integration tests. The runnable examples/circuit_breaker.rs walks the full Closed → Open → HalfOpen → Closed lifecycle.

Queue — bounded, deadline-aware waiting

When a limiter is saturated, callers can be rejected or wait. A Queue lets them wait in an orderly way: bounded so it cannot grow without limit, served by priority (and fairly across keys at equal priority), and — the headline — it drops a waiter whose deadline has passed rather than serving it, so a dead request never consumes a token or blocks a live one.

use std::time::Duration;
use throttle_net::{Overflow, Queue, Throttle};

let queue: Queue<Throttle, &str> = Queue::builder()
    .capacity(100)
    .overflow(Overflow::DropOldest)
    .build(Throttle::per_second(50));

// Wait for a slot, but give up after 2 seconds.
queue.acquire("tenant:1", 0, Some(Duration::from_secs(2))).await?;

Overflow policy is configurable — Reject, DropOldest, or DropLowestPriority. Behind the tokio feature (it needs an async runtime to wait).

SlidingWindowLog — exact limiting, no boundary burst

Throttle is a token bucket: smooth and cheap, but it admits a full burst at any instant. SlidingWindowLog is the exact alternative — it records the timestamp of every grant and admits a request only if fewer than limit were granted in the trailing window. No boundary burst, at the cost of remembering recent grants. It implements Limiter, so it drops into a hybrid, a per-key store, a layer, or behind the circuit breaker like any other limiter.

use std::time::Duration;
use throttle_net::SlidingWindowLog;

let limiter = SlidingWindowLog::new(5, Duration::from_secs(1));
for _ in 0..5 {
    assert!(limiter.try_acquire());
}
assert!(!limiter.try_acquire()); // the 6th in this window is refused

Error additions

The #[non_exhaustive] ThrottleError gains CircuitOpen { retry_after } (retryable), QueueFull (retryable), and DeadlineExceeded (not retryable), each carrying the right error-forge retryability metadata.

Breaking changes

None. Everything in this release is additive; the new error variants slot into the existing #[non_exhaustive] enum.

Verification

Run on Windows x86_64, Rust stable 1.93.1; the same commands run in the CI matrix on Linux, macOS, and Windows across stable and MSRV 1.85:

cargo fmt --all -- --check
cargo clippy --all-targets --all-features -- -D warnings
cargo clippy --all-targets --no-default-features -- -D warnings
cargo test --all-features
cargo test --no-default-features
RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --all-features
cargo bench --bench throttle_bench
cargo deny check
cargo audit

All green. Counts at this tag (--all-features): 103 unit tests, 7 property tests, 3 circuit-breaker integration tests, 2 retry integration tests, 61 doctests. The circuit-breaker state machine is proptested against a reference model; the queue's deadline-drop, overflow, and fair priority scheduling are covered by unit tests over the scheduler's pure selection logic plus async integration tests.

What's next

• v0.5.0 — Adaptive. Back off without an explicit rate-limit signal: an AIMD adaptive limiter (additive increase, multiplicative decrease) with floor and ceiling, a latency-based (Vegas-style) limiter keyed on observed p99, a custom adaptive-strategy trait, and an observe(outcome) feedback loop — never violating the underlying hard limit.

Installation

[dependencies]
throttle-net = "0.4"

# The circuit breaker is behind a feature flag:
throttle-net = { version = "0.4", features = ["circuit-breaker"] }

MSRV: Rust 1.85.

Documentation

• README
• API Reference
• CHANGELOG

---

Changelog: CHANGELOG.md.

Full Changelog: v0.3.0...v0.4.0