v1.0.0 — Stable API

[jamesgober]

throttle-net v1.0.0 — Stable

The outbound throttling and resilience library is stable. After the 0.x series built and hardened it, v1.0.0 freezes the public API until 2.0. One library replaces the four you assemble today — a token bucket, a backoff loop, a circuit breaker, and per-provider header parsing — and adds the parts nobody else ships: multi-dimensional cost-aware limits and adaptive throttling. No functional change from 0.9.0; this release is the stability commitment.

What is throttle-net?

Where rate-net protects your service from being overwhelmed (inbound), throttle-net protects your service from overwhelming the downstreams it calls — and from being banned by them. The defining operation is to wait for capacity, not to reject: you pace your own outbound work rather than dropping someone else's request.

The common case is one builder and one acquire().await?:

# async fn run() -> Result<(), throttle_net::ThrottleError> {
use throttle_net::Throttle;

let throttle = Throttle::per_second(100);
throttle.acquire().await?; // returns as soon as a token is free
// ... call the downstream ...
# Ok(())
# }

The hard cases — LLM token budgets, per-tenant quotas, adaptive backpressure — are first-class.

The stable surface

Everything below is frozen until 2.0.

• Limiters — the [Throttle] token bucket (lock-free, one atomic compare-and-swap per acquire) and the exact [SlidingWindowLog] (no boundary burst), each with a waiting, cost-aware acquire and non-blocking try_acquire / peek.
• Composition — [Hybrid] (must pass all), [MultiLimiter] (multi-dimensional cost-aware budgets), [PerKey] (independent per-key state with bounded memory), and [Layered] (global / per-key / per-endpoint scopes) — all with peek-then-commit correctness, so a request never spends in one limiter when another would block it.
• Retry & backoff — [Backoff] (constant / linear / exponential, with full / equal / decorrelated jitter) and [Retry] with per-error classification and Retry-After honoring.
• Resilience — a [CircuitBreaker] that wraps any limiter and fails fast without consuming it, and a bounded, deadline-aware, priority [Queue].
• Adaptive concurrency — an [AdaptiveLimiter] that discovers the right in-flight limit from outcome feedback (AIMD or Vegas), never exceeding the hard ceiling.
• Provider integration — response-header parsers (OpenAI, Anthropic, GitHub, Stripe, AWS, the IETF draft) with limiter synchronization, and LLM tier presets.
• Observability — metrics and tracing events, feature-gated and zero-cost when off.
• The Limiter trait — the Tier-3 extension seam; every algorithm and composite is one Limiter.

The tiered design holds: Tier-1 is the whole common case in a couple of calls, Tier-2 the builders for tuning, Tier-3 the trait seam for custom backends.

Runtime and platform

• Runtimes — the waiting surface runs on tokio (default) or smol with identical call-site code; you pick the timer backend by feature.
• no_std — with std off, the pure algorithm core ([Backoff], [Jitter], [Decision]) compiles without the standard library.
• Platforms — Linux, macOS, and Windows are first-class, verified on stable and MSRV 1.85 across the CI matrix.

How it is held correct

The defining invariants are tested, not assumed:

• Property tests for every limiter invariant — burst never exceeds capacity, the exact window admits at most N per window, composites bind on the tightest scope, per-key floods stay within the eviction bound, backoff stays under its ceiling, and the adaptive limit stays within [floor, ceiling].
• A loom model check that exhaustively explores the adaptive limiter's lock-free slot accounting — no over-admission, no lost slots — under arbitrary thread interleavings.
• Fuzzed parsers — the Retry-After and provider-header parsers never panic on arbitrary input, checked both by cargo-fuzz targets and an always-on deterministic smoke suite. (This caught two real integer-overflow panics during 0.9, both fixed.)
• No unsafe — the crate is #![forbid(unsafe_code)].
• Benchmarks — uncontended try_acquire is ~27 ns (target < 50 ns), benchmarked head-to-head with governor, with a contention sweep at 1 / 4 / 16 / 64 acquirers.

Changes since 0.9.0

• Stable. The public API is frozen until 2.0.
• Removed the unimplemented serde feature flag (it pulled the dependency but wired up no Serialize/Deserialize). Serializable configs may return post-1.0 as an additive feature with a designed, documented representation.

Breaking changes

None from 0.9.0. The only removal is the no-op serde feature flag, which had no functional effect.

After 1.0

Backward-compatible additions under consideration (not yet shipped):

• Serializable limiter configs (serde), with a designed on-disk representation.
• Companion middleware crates (throttle-net-tower, throttle-net-reqwest).
• Distributed limiter state is a 2.0 design topic; 1.x is in-process.

Verification

Run on Windows x86_64, Rust stable; the same commands run in CI on Linux, macOS, and Windows across stable and MSRV 1.85, with dedicated jobs for the smol + no_std matrix, the loom model check, and the fuzz smoke run:

cargo fmt --all -- --check
cargo clippy --all-targets --all-features -- -D warnings
cargo clippy --no-default-features --features smol --all-targets -- -D warnings
cargo clippy --no-default-features --all-targets -- -D warnings
cargo test --all-features
cargo test                                          # default: tokio backend
cargo test --no-default-features                    # no_std core + its doctests
cargo test --no-default-features --features smol    # smol backend
cargo build --no-default-features                   # no_std lib builds
cargo bench --no-run --all-features
RUSTFLAGS="--cfg throttle_loom" cargo test --test loom_throttle \
    --no-default-features --features adaptive --release
cargo +nightly fuzz run retry_after --target x86_64-unknown-linux-gnu -- -max_total_time=30
cargo +nightly fuzz run provider_headers --target x86_64-unknown-linux-gnu -- -max_total_time=30
cargo doc --no-deps && RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --all-features
cargo deny check
cargo audit

All green.

Installation

[dependencies]
throttle-net = "1"

# Optional features:
throttle-net = { version = "1", features = ["circuit-breaker", "adaptive", "provider-llm", "metrics", "tracing"] }

# On smol instead of tokio:
throttle-net = { version = "1", default-features = false, features = ["smol"] }

MSRV: Rust 1.85.

Documentation

• README
• API Reference
• Cookbook
• Migrating from governor
• CHANGELOG

Thanks

throttle-net builds on the first-party stack: better-bucket (token-bucket accounting), clock-lib (mockable time), error-forge (domain errors), and ahash (DoS-resistant shard hashing). It is the outbound companion to rate-net.

---

Changelog: CHANGELOG.md.

---

This discussion was created from the release v1.0.0 — Stable API.