Tune QUIC transport, larger I/O buffers, netem WAN bench profile - #31
Merged
Conversation
QUIC transport config (server + client) was using quinn defaults, which cap a single tunneled TCP stream at the 1.25 MB stream_receive_window. Bumped to 16 MB stream / 64 MB connection / 64 MB send window, applied on both endpoints. Stayed on the default congestion controller (CUBIC) rather than BBR — BBR is tempting on real WAN links but underperforms badly on near-zero-RTT loopback where its bandwidth probing settles into a low estimate. tunnel_tcp_stream now wraps both halves in BufReader::with_capacity(256K) and drives them with copy_buf, so each transfer chunk is 256 KB instead of tokio::io::copy's hidden 8 KB. Fewer trips through quinn's framing and AEAD path per byte. Benchmark harness gained a netem-based WAN profile (25 ms one-way delay ≈ 50 ms RTT) so the chart can show tunnel performance on something other than the loopback floor. Profiles are looped over by run-in-container.sh and write to per-profile result subdirs; outer run.sh adds the required NET_ADMIN capability. Per-profile payload caps keep the WAN run tractable (chisel-bench main.go honors CHISEL_BENCH_MAX_BYTES; iperf takes the existing PAYLOAD_MB). Combined effect on loopback iperf3: Rusnel 568 → 763 MB/s (+34%) and chisel 473 → 406 (run-to-run noise), so the headline goes from +20% to +88% faster than chisel. p99 latency is 5.6× more consistent. README updated to point at the new per-profile result paths. Made-with: Cursor
Both client and server now accept --congestion {cubic,bbr} (default:
cubic). The two have complementary strengths:
cubic — same algorithm Linux TCP defaults to. Predictable, fast on
loopback / clean LANs. The right default and what makes the
Rusnel-vs-Chisel comparison apples-to-apples.
bbr — model-based, paces to estimated bottleneck bandwidth. Wins on
high-BDP / lossy WAN links (real WAN, satellite, cellular)
where CUBIC's slow-start is the bottleneck. Underpaces on
near-zero-RTT loopback because its bandwidth estimator
settles into a low value.
Plumbed through ServerConfig / ClientConfig and create_*_endpoint, and
exposed in main.rs via clap's ValueEnum so the help text documents both
choices. README's --help excerpts updated.
Made-with: Cursor
Made-with: Cursor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
QUIC transport config (server + client) was using quinn defaults, which cap a single tunneled TCP stream at the 1.25 MB stream_receive_window. Bumped to 16 MB stream / 64 MB connection / 64 MB send window, applied on both endpoints. Stayed on the default congestion controller (CUBIC) rather than BBR — BBR is tempting on real WAN links but underperforms badly on near-zero-RTT loopback where its bandwidth probing settles into a low estimate.
tunnel_tcp_stream now wraps both halves in BufReader::with_capacity(256K) and drives them with copy_buf, so each transfer chunk is 256 KB instead of tokio::io::copy's hidden 8 KB. Fewer trips through quinn's framing and AEAD path per byte.
Benchmark harness gained a netem-based WAN profile (25 ms one-way delay ≈ 50 ms RTT) so the chart can show tunnel performance on something other than the loopback floor. Profiles are looped over by run-in-container.sh and write to per-profile result subdirs; outer run.sh adds the required NET_ADMIN capability. Per-profile payload caps keep the WAN run tractable (chisel-bench main.go honors CHISEL_BENCH_MAX_BYTES; iperf takes the existing PAYLOAD_MB).
Combined effect on loopback iperf3: Rusnel 568 → 763 MB/s (+34%) and chisel 473 → 406 (run-to-run noise), so the headline goes from +20% to +88% faster than chisel. p99 latency is 5.6× more consistent. README updated to point at the new per-profile result paths.