Skip to content

perf: TLS connection pool + coalesce tuning for lower latency#751

Merged
therealaleph merged 1 commit intotherealaleph:mainfrom
yyoyoian-pixel:perf/tls-pool-and-coalesce
May 5, 2026
Merged

perf: TLS connection pool + coalesce tuning for lower latency#751
therealaleph merged 1 commit intotherealaleph:mainfrom
yyoyoian-pixel:perf/tls-pool-and-coalesce

Conversation

@yyoyoian-pixel
Copy link
Copy Markdown
Contributor

Summary

  • TLS pool refill loop: maintains ≥8 ready connections in the background so acquire() never pays a cold TLS handshake. Connections are opened one at a time (no burst), with staggered expiry so they roll off gradually.
  • Freshest-first acquire: picks the connection with the most remaining TTL instead of popping whatever is on top.
  • Pool TTL 45→60s: connections live longer, reducing churn.
  • Coalesce step 10→200ms: the dominant bottleneck is the ~1.5s Apps Script round-trip, so 200ms extra wait is negligible but packs 3–5x more ops per batch during page loads.

Test plan

  • Tested on Android (Pixel 6 Pro) with full-mode tunnel
  • Pool hit rate: 100% (zero cold TLS handshakes during requests)
  • Batch sizes improved from mostly 1-op to 3–5 ops during page loads
  • No regressions in connection stability or timeout behavior

🤖 Generated with Claude Code

TLS pool improvements:
- Increase POOL_TTL from 45s to 60s so connections live longer
- Add POOL_MIN (8): background refill loop keeps at least 8 ready
  TLS connections so acquire() never pays a cold handshake
- Refill checks every 5s, only counts connections with ≥20s
  remaining as "healthy" — nearly-expired entries don't count
- warm() now opens sequentially (500ms gaps) with 8s expiry
  offset per connection so they roll off gradually instead of
  all expiring together after a cliff
- acquire() picks the freshest connection (most remaining TTL)
  instead of popping whatever is on top

Coalesce step increase:
- DEFAULT_COALESCE_STEP_MS: 10 → 200. The dominant bottleneck is
  the Apps Script round-trip (~1.5s), so the extra 200ms wait is
  negligible to the user but lets significantly more ops land in
  each batch — measured 3–5 ops/batch vs 1 op/batch at 10ms
  during page loads, cutting round-trips roughly in half.

Tested on Android (Pixel 6 Pro) with full-mode tunnel. Pool
hit rate went from 96% (POOL_MIN=4) to 100% (POOL_MIN=8) —
zero cold TLS handshakes during requests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@therealaleph
Copy link
Copy Markdown
Owner

@yyoyoian-pixel — pool changes are good (refill loop + freshest-first acquire + extended TTL). Merging.

The coalesce step revert from 10ms→200ms is a complete reversal of your own PR #674 from 4 days ago, which I'd want measurement data behind before flipping again — but 200ms is the conservative side and matches what we had before v1.9.8, so the regression risk is bounded. Merging it too.

For future timing-constant work, please attach a quantitative comparison (e.g. tcpdump batch sizes per N seconds across both values, or page-load p50/p95 on a known site) when proposing changes — gut-feel "feels snappier" can flip-flop fast.

Will ship in the next release.


[reply via Anthropic Claude | reviewed by @therealaleph]

@therealaleph therealaleph merged commit ba99d2a into therealaleph:main May 5, 2026
1 check passed
therealaleph added a commit that referenced this pull request May 5, 2026
Wraps three already-merged PRs into a release:
- PR #763 (@yyoyoian-pixel): block_doh: true default; rejects browser DoH at SOCKS5 listener so it falls back to system DNS via tun2proxy virtual DNS instead of paying ~1.5s tunnel round-trip per name lookup. Also fixes the Android tunnel_doh config mismatch (was false on Android, true on Rust — silently broke bypass_doh_hosts).
- PR #751 (@yyoyoian-pixel): TLS pool refill loop keeping ≥8 ready connections, freshest-first acquire, pool TTL 45→60s, coalesce step 10→200ms (more conservative revert from v1.9.8 for full-mode batch packing).
- PR #747 (@Shjpr9): added github.io to Fastly fronting group example.

Tests: 179 lib + 35 tunnel-node green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants