Investigate why dropping socket.io polling fallback degrades p95 under load (#7756 follow-up)

[JohnMcLear]

From the 2026-05 scaling dive, setting `socketTransportProtocols: ["websocket"]` (i.e. dropping the polling fallback) consistently worsens client p95 latency under high concurrency:

Authors	baseline p95	websocket-only p95	apply_mean baseline	apply_mean ws-only
100	11 ms	18 ms	4.16 ms	5.13 ms
140	14 ms	25 ms	4.02 ms	6.09 ms
180	16 ms	68 ms	4.48 ms	9.81 ms
200	22 ms	82 ms	4.95 ms	13.33 ms

Same harness, same runner, same sweep — only difference is the transport setting.

Hypotheses worth checking

1. WS-only forces clients that can't establish WebSocket within socket.io's handshake timeout to retry-loop rather than fall back to polling, producing reconnect storms that drive up server CPU.
2. The WS-only path changes socket.io's handshake protocol routing in a way that interacts badly with load balancers / proxies.
3. Per-message-WebSocket framing overhead becomes significant when emits/sec is high (66k/dwell at 200 authors).
4. The polling fallback acts as a natural coalescer (multiple events per HTTP poll) that we lose when forcing pure-WS.

Hypothesis 4 is particularly interesting because it would mean polling-as-batching is doing real work for us today.

Reproducing

```
gh workflow run "Scaling dive" --repo ether/etherpad-load-test --ref main \
-f core_ref=develop \
-f sweep='authors=20..200:step=20:dwell=10s:warmup=2s'
```

Compare `scaling-dive-baseline` vs `scaling-dive-websocket-only` artifacts. Run 25940112728 is the reference.

Why this matters

If we know why the polling fallback helps so much, we can either preserve that property explicitly (via batching) or stop carrying socket.io's polling-fallback code path.

[JohnMcLear]

Diagnosis

Confirmed hypothesis 4 by reading the engine.io 6.6.5 transport code shipped with etherpad core. The polling fallback is a natural coalescer; WebSocket isn't.

Engine.io socket buffering (`engine.io/build/socket.js:flush`)

sendPacket(type, data, ...) {
  this.writeBuffer.push(packet);
  this.flush();
}

flush() {
  if (closed || !transport.writable || !writeBuffer.length) return;
  const wbuf = this.writeBuffer;
  this.writeBuffer = [];
  this.transport.send(wbuf);   // entire buffer hands off in one call
}

So packets queue in `writeBuffer` and `flush()` is a no-op while `transport.writable === false`.

Polling transport (`transports/polling.js`)

`writable` is false between HTTP poll requests. While a poll response is being written and until the client opens its next long-poll, every `emit()` piles up in `writeBuffer`. Then on the next poll:

send(packets) {
  this.parser.encodePayload(packets, doWrite);
  // -> single HTTP response containing ALL queued packets
}

N packets → 1 HTTP response.

WebSocket transport (`transports/websocket.js`)

`writable` is briefly false during each frame write, then true again essentially immediately. Each `emit()` triggers its own `flush()` while the buffer is still small. And even when the buffer accumulates:

send(packets) {
  for (let i = 0; i < packets.length; i++) {
    const packet = packets[i];
    this.parser.encodePacket(packet, ...);  // ONE WS frame per packet
  }
}

So N packets → N WS frames, regardless of batching at the socket layer.

Why this matters

At 200 authors in our dive, the server emits 6 600 `NEW_CHANGES` per second per client. Polling sends those as roughly 25 HTTP responses/sec (default poll cadence ~25 ms) carrying ~250 packets each — i.e. 25 system calls instead of 6 600. WebSocket sends ~6 600 individual frames per second per client. Hence baseline apply_mean ≈ 5 ms vs ws-only ≈ 13 ms: WebSocket is starving the apply path of CPU on syscall overhead.

This is not an etherpad bug. It's a deliberate engine.io WS-transport design choice. Engine.io's wire protocol can pack multiple packets per WS message (the "payload" encoding the polling transport uses), but the WS transport always sends one packet per frame.

Where to take this

Two paths in priority order:

1. Application-level coalesce (cheap): `NEW_CHANGES_BATCH` message type carrying an array of revisions in a single emit per socket. Cuts the engine.io packet count proportionally, regardless of transport. This is what the dive doc already recommends; the WS finding makes it more urgent.
2. Transport-level pack (deeper): PR engine.io upstream to use payload-encoding on the WS transport — pack multiple queued packets into one WS message. Would help any high-emit-rate socket.io application. Worth filing in socketio/engine.io once we've prototyped it locally.

Keeping `socketTransportProtocols: ["websocket", "polling"]` (current default) remains correct until at least path 1 lands.

[JohnMcLear]

Prototype scored, did not help

Closed #7772. The simple monkey-patch (pack only when send(packets.length > 1)) almost never fires under load because engine.io's socket.flush() drains the writeBuffer immediately whenever transport.writable === true, and WS writes flip writable back within microseconds. Net result: the writeBuffer rarely accumulates more than one packet, so the patch passes through to the legacy single-frame path 99% of the time.

The actual change needed

To get real coalescing on the WS transport, we'd need to deliberately delay the flush — buffer multiple sendPacket calls within a small time window before draining via transport.send. Two viable strategies:

1. Microtask deferral. Replace the synchronous flush() call inside sendPacket() with queueMicrotask(() => flush()). Multiple sends within the same task accumulate. Zero added wall-clock latency (microtasks drain before the next task). Smallest behaviour change.

2. Configurable small delay. Add a setTimeout(flush, debounceMs) with debounceMs like 1-2 ms. Slightly more accumulation across event-loop iterations; adds latency proportional to the delay. Bigger behaviour change but reaches more flush opportunities.

Either path is invasive enough that it should probably be a focused engine.io upstream PR rather than a monkey-patch. Worth opening once we have a clear hypothesis on how much delay is needed to materially shift the WS-frame-per-packet ratio.

Secondary methodology finding

The scoring run also showed websocket-only as the best lever — contradicting every previous dive. The cause is that each GHA matrix entry runs on a separate physical runner, so within-run cross-lever comparisons are cross-hardware. Runner noise can flip lever conclusions. Future dive runs need N ≥ 3 trials per lever to filter that noise out before drawing strong conclusions. The scaling dive doc PR needs a follow-up note about this; existing conclusions in the doc may need re-validation.

[JohnMcLear]

Resolved by #7774

The investigation traced WS-only's performance degradation to engine.io sending one WS frame per packet (vs polling's encodePayload batched HTTP responses). Two prototype directions came out of this:

• First attempt — #7772 — patched transport.send(packets[]) to use encodePayload for multi-packet sends. Almost never fired because engine.io's Socket.flush() drains the writeBuffer immediately when transport.writable === true, and WS writable returns to true within microseconds. Closed.

• Real fix — #7774 — patches Socket.prototype.sendPacket to schedule a coalesced flush via queueMicrotask. Multiple sendPacket calls in the same task accumulate before drain. N=3 scoring shows modest mid-load tail-latency reduction (step 350 max: 122 → 110 ms; step 300 max: 71 → 58 ms). Wire bytes unchanged.

Net effect on the original question — "why does WS-only degrade under load?" — is now answered: socket.io's polling fallback was acting as a natural coalescer at the HTTP boundary; the WS transport doesn't coalesce, so high-emit-rate workloads pay a per-frame syscall cost the polling path naturally amortises. Production should keep socketTransportProtocols: ["websocket", "polling"] (the default), and the flush-deferral approach in #7774 is the right way to claw back some of polling's coalescing property on the WS path without changing wire behaviour.

Closing. The full diagnosis is captured in the scaling-dive doc, section "Lever 8b — engine.io socket flush deferral".