Repository navigation
ws: mask outgoing payload in chunks - #23567
mvanslobbe wants to merge 5 commits into
Conversation
ws_enc_write_payload() masks one byte at a time and calls Curl_bufq_write() once per byte. That limits WebSocket sends (curl_ws_send() and uploads through the ws encode reader) to about 250 MB/s on one core, whatever the frame size. This change masks into a 1 KiB stack buffer, eight bytes at a time, with the mask rotated to the current offset. Each chunk then goes to the bufq in a single call. Partial writes and CURLE_AGAIN behave as before: the xor index and payload_remain advance only by the bytes the bufq accepted. The masking now lives in ws_enc_mask_write(), marked UNITTEST, and the new unit test 2607 drives it with random piece sizes, mask offsets and drains through a hard-limited bufq, so partial writes and CURLE_AGAIN are covered. It checks the output against a byte-by-byte reference. A microbenchmark of the function body (clang 22 -O2, one P-core of an i7-12700H, 64 MiB sent per frame size, median of 7 runs, output compared byte for byte against the old loop): | frame size | before | after | |------------|-----------|-------------| | 16 B | 200 MB/s | 1222 MB/s | | 125 B | 239 MB/s | 7527 MB/s | | 1000 B | 244 MB/s | 31319 MB/s | | 16 KiB | 245 MB/s | 36416 MB/s | | 1 MiB | 245 MB/s | 20796 MB/s | Tests 2300-2304, 2601, 2607 and 2700-2723 pass in a debug build with ASan and UBSan.
I'd personally like these results replicated and confirmed in curl's test server, |
Fair enough. Public run and source:
I looked at tests/perf and scorecard.py first. The perf programs avoid servers and only use the public API and curlx, but sending WebSocket data needs a connection, so
Master → this PR, MB/s:
clang vectorises the 8-byte loop at -O2 and gcc does not, hence the 2x gap. Reshaping the loop helps gcc but slows clang, and getting both to their best would need compiler-specific code, so I've kept it simple here. It's worth exploring separately. I have updated the PR description with these numbers. |
There was a problem hiding this comment.
🟢 Approval recommended
No blocking issues were identified in the bounded masking change, partial-write handling, or test integration.
No open issues 🎉
What changed in this PR
This PR optimizes outgoing WebSocket payload masking by replacing per-byte queue writes with chunked writes, benefiting both curl_ws_send() and WebSocket uploads.
Changes:
- Masks payloads in 1 KiB chunks using eight-byte operations.
- Preserves mask-offset and payload accounting across partial writes and queue backpressure.
- Adds a randomized unit test comparing masked output against a byte-by-byte reference.
| File | Description |
|---|---|
| tests/unit/unit2607.c | Tests masking, partial writes, and full-queue handling. |
| tests/unit/Makefile.inc | Registers the new unit test source. |
| tests/data/test2607 | Defines the WebSocket masking test case and required features. |
| tests/data/Makefile.am | Registers the new test case. |
| lib/ws.c | Extracts a testable helper and implements chunked payload masking. |
Review effort: Balanced
|
Would be cool to get websocket transfers into the scorecard tests so that we can get them into the perf test site. WDYT @icing ? |
I'll see what I can do. |
|
Thanks! |

ws_enc_write_payload() masks one byte at a time and calls
Curl_bufq_write() once per byte. That limits WebSocket sends
(curl_ws_send() and uploads through the ws encode reader) to about
100-250 MB/s on one core, whatever the frame size.
This change masks into a 1 KiB stack buffer, eight bytes at a time, with
the mask rotated to the current offset. Each chunk then goes to the bufq
in a single call. Partial writes and CURLE_AGAIN behave as before: the
xor index and payload_remain advance only by the bytes the bufq
accepted.
The masking now lives in ws_enc_mask_write(), marked UNITTEST, and the
new unit test 2607 drives it with random piece sizes, mask offsets and
drains through a hard-limited bufq, so partial writes and CURLE_AGAIN
are covered. It checks the output against a byte-by-byte reference.
Benchmarked on GitHub Actions with the real ws_enc_write_payload() from
master and from this branch, both built at -O2 and linked against the
same libcurl, output checked identical, median of 7 rounds:
https://github.com/mvanslobbe/curl-fork/actions/runs/39835136699
(source: https://github.com/mvanslobbe/curl-fork/tree/e0412bb)
Master → this PR, MB/s:
clang vectorises the 8-byte loop at -O2 and gcc does not, which
accounts for most of the difference between the two.
Tests 2300-2304, 2601, 2607 and 2700-2723 pass in a debug build with
ASan and UBSan.
AI disclosure
This change was found while reviewing curl's WebSocket send path and written with the help of Claude Code. I reviewed and tested the change myself.