Skip to content

feat: add contains_handshake and assert_contains_handshake helpers#3533

Merged
larseggert merged 2 commits intomozilla:mainfrom
mvanhorn:osc/3447-assert-contains-handshake
Apr 21, 2026
Merged

feat: add contains_handshake and assert_contains_handshake helpers#3533
larseggert merged 2 commits intomozilla:mainfrom
mvanhorn:osc/3447-assert-contains-handshake

Conversation

@mvanhorn
Copy link
Copy Markdown
Contributor

@mvanhorn mvanhorn commented Apr 9, 2026

Summary

Adds contains_handshake() and assert_contains_handshake() to the test fixture assertions module. These scan through coalesced long-header packets in a datagram to check if any is a Handshake packet.

Why this matters

Tests that assert on Handshake packets are fragile when packet sizing causes a small Initial to be coalesced before the Handshake. The existing assert_handshake() only checks the first packet in a datagram. assert_contains_handshake() scans through all coalesced packets.

Changes

  • test-fixture/src/assertions.rs: Added contains_handshake() and assert_contains_handshake() following the same coalesced-packet scanning pattern used by assert_no_1rtt().

Testing

  • Follows the established pattern of assert_no_1rtt for scanning coalesced packets
  • Could not run cargo check locally (NSS system dependency not available); GitHub CI will verify

Fixes #3447

This contribution was developed with AI assistance (Claude Code).

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 9, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.43%. Comparing base (7c0b85f) to head (3325659).
⚠️ Report is 21 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3533      +/-   ##
==========================================
- Coverage   94.53%   94.43%   -0.11%     
==========================================
  Files         127      131       +4     
  Lines       39627    39957     +330     
  Branches    39627    39957     +330     
==========================================
+ Hits        37462    37733     +271     
- Misses       1328     1377      +49     
- Partials      837      847      +10     
Flag Coverage Δ
freebsd 93.49% <ø> (-0.09%) ⬇️
linux 94.49% <ø> (-0.01%) ⬇️
macos 94.44% <ø> (-0.01%) ⬇️
windows 94.49% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
neqo-common 98.61% <ø> (ø)
neqo-crypto 87.08% <ø> (ø)
neqo-http3 93.92% <ø> (ø)
neqo-qpack 95.14% <ø> (ø)
neqo-transport 95.56% <ø> (ø)
neqo-udp 84.90% <ø> (ø)
mtu 86.61% <ø> (ø)

Copy link
Copy Markdown
Collaborator

@larseggert larseggert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Were you planning to actually use these new functions for something?

@mvanhorn
Copy link
Copy Markdown
Contributor Author

mvanhorn commented Apr 9, 2026

Yes - they're intended for tests where packet sizing causes a small Initial to be coalesced before the Handshake. The existing assert_handshake() only checks the first packet in a datagram, so those tests fail intermittently. I have a follow-up PR that uses assert_contains_handshake() to fix some of those flaky assertions, but wanted to land the helpers separately to keep the diff focused.

@larseggert
Copy link
Copy Markdown
Collaborator

I'd suggest to combine those PRs (into this PR), so we can see whether those additions have the right shape overall.

Add helpers that scan through coalesced long-header packets in a
datagram to find a Handshake packet. This reduces test fragility when
packet sizing causes a small Initial to be coalesced before the
Handshake.

Follows the same scanning pattern as assert_no_1rtt.

Fixes mozilla#3447
…ndshake

The two assertions after the server's second handshake burst were
commented out because the datagram can contain a small coalesced
Initial in front of the Handshake packet, which tripped the
first-packet-only check in assert_handshake.

Use the new assert_contains_handshake helper so the check is
deterministic regardless of whether an Initial is coalesced at the
front of the datagram.
@mvanhorn mvanhorn force-pushed the osc/3447-assert-contains-handshake branch from fe90214 to 3325659 Compare April 11, 2026 15:32
@mvanhorn
Copy link
Copy Markdown
Contributor Author

Pushed a consumer in pto_handshake_complete (recovery.rs) that uses assert_contains_handshake.

That test had two assert_handshake(...) calls commented out at the point where the server hands back its second handshake burst, with the inline note that the datagram might have extra Initial data coalesced at the front. That is exactly what this helper is for, so I re-enabled both assertions against assert_contains_handshake.

The rest of the assert_handshake call sites in recovery.rs and handshake.rs run on packets that have already been extracted via split_datagram, or on datagrams where the first packet is deterministically the Handshake (e.g. after client.authenticated(...)), so migrating them would just obscure the assertion.

Ran cargo test -p neqo-transport --lib pto_handshake_complete -- --test-threads=1 five times in a row, all green, and the full connection::tests::recovery (21 tests) and connection::tests::handshake (42 tests) modules also pass. cargo fmt --check (nightly) and cargo clippy -p neqo-transport --tests -- -D warnings are clean.

Happy to grow the migration if you have other specific tests in mind.

@github-actions
Copy link
Copy Markdown
Contributor

Benchmark results

No significant performance differences relative to 7c0b85f.

All results
transfer/1-conn/1-100mb-resp (aka. Download)/mtu-1504: No change in performance detected.
       time:   [204.15 ms 204.65 ms 205.24 ms]
       thrpt:  [487.24 MiB/s 488.65 MiB/s 489.83 MiB/s]
change:
       time:   [-0.6986% -0.3024% +0.0942] (p = 0.13 > 0.05)
       thrpt:  [-0.0941% +0.3033% +0.7035]
       No change in performance detected.
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) high mild
3 (3.00%) high severe
transfer/1-conn/10_000-parallel-1b-resp (aka. RPS)/mtu-1504: No change in performance detected.
       time:   [285.47 ms 287.20 ms 288.91 ms]
       thrpt:  [34.613 Kelem/s 34.819 Kelem/s 35.030 Kelem/s]
change:
       time:   [-1.0760% -0.1804% +0.6795] (p = 0.68 > 0.05)
       thrpt:  [-0.6749% +0.1807% +1.0877]
       No change in performance detected.
transfer/1-conn/1-1b-resp (aka. HPS)/mtu-1504: No change in performance detected.
       time:   [38.818 ms 38.947 ms 39.092 ms]
       thrpt:  [25.581   B/s 25.676   B/s 25.761   B/s]
change:
       time:   [-0.4803% +0.0617% +0.6116] (p = 0.83 > 0.05)
       thrpt:  [-0.6079% -0.0616% +0.4826]
       No change in performance detected.
Found 7 outliers among 100 measurements (7.00%)
3 (3.00%) high mild
4 (4.00%) high severe
transfer/1-conn/1-100mb-req (aka. Upload)/mtu-1504: Change within noise threshold.
       time:   [204.47 ms 204.89 ms 205.33 ms]
       thrpt:  [487.01 MiB/s 488.08 MiB/s 489.06 MiB/s]
change:
       time:   [-0.8461% -0.5221% -0.2056] (p = 0.00 < 0.05)
       thrpt:  [+0.2060% +0.5249% +0.8533]
       Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
streams/walltime/1-streams/each-1000-bytes: No change in performance detected.
       time:   [596.01 µs 597.73 µs 599.81 µs]
       change: [-0.4315% +0.2495% +0.8579] (p = 0.46 > 0.05)
       No change in performance detected.
Found 8 outliers among 100 measurements (8.00%)
2 (2.00%) high mild
6 (6.00%) high severe
streams/walltime/1000-streams/each-1-bytes: Change within noise threshold.
       time:   [12.499 ms 12.541 ms 12.606 ms]
       change: [+0.1341% +0.5267% +1.1002] (p = 0.01 < 0.05)
       Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
streams/walltime/1000-streams/each-1000-bytes: Change within noise threshold.
       time:   [45.550 ms 45.600 ms 45.650 ms]
       change: [-0.6778% -0.3491% -0.1029] (p = 0.01 < 0.05)
       Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
transfer/walltime/pacing-false/varying-seeds: Change within noise threshold.
       time:   [78.503 ms 78.562 ms 78.621 ms]
       change: [-0.9828% -0.7616% -0.5959] (p = 0.00 < 0.05)
       Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
transfer/walltime/pacing-true/varying-seeds: Change within noise threshold.
       time:   [79.758 ms 79.807 ms 79.858 ms]
       change: [+0.0394% +0.1571% +0.2667] (p = 0.01 < 0.05)
       Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
transfer/walltime/pacing-false/same-seed: Change within noise threshold.
       time:   [78.197 ms 78.291 ms 78.402 ms]
       change: [-0.8935% -0.7078% -0.5328] (p = 0.00 < 0.05)
       Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
4 (4.00%) high mild
2 (2.00%) high severe
transfer/walltime/pacing-true/same-seed: Change within noise threshold.
       time:   [79.914 ms 79.972 ms 80.037 ms]
       change: [-0.6600% -0.5234% -0.4054] (p = 0.00 < 0.05)
       Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe

Download data for profiler.firefox.com or download performance comparison data.

@github-actions
Copy link
Copy Markdown
Contributor

Failed Interop Tests

QUIC Interop Runner, client vs. server, differences relative to main at 7c0b85f.

neqo-pr as clientneqo-pr as server
neqo-pr vs. go-x-net: BP BA
neqo-pr vs. haproxy: 🚀L1 BP BA
neqo-pr vs. kwik: S L1 C1 BP BA
neqo-pr vs. linuxquic: run cancelled after 20 min
neqo-pr vs. lsquic: run cancelled after 20 min
neqo-pr vs. msquic: A L1 C1
neqo-pr vs. mvfst: H DC LR M R Z 3 B U A L1 L2 C1 C2 6 BP BA
neqo-pr vs. neqo: Z A
neqo-pr vs. nginx: BP BA
neqo-pr vs. ngtcp2: CM
neqo-pr vs. picoquic: A
neqo-pr vs. quic-go: A ⚠️C1
neqo-pr vs. quiche: BP BA
neqo-pr vs. s2n-quic: BA CM
neqo-pr vs. tquic: S BP BA
neqo-pr vs. xquic: A L1 ⚠️C1
aioquic vs. neqo-pr: Z CM
go-x-net vs. neqo-pr: CM
kwik vs. neqo-pr: Z BP BA CM
lsquic vs. neqo-pr: Z
msquic vs. neqo-pr: Z CM
mvfst vs. neqo-pr: Z A L1 C1 CM
neqo vs. neqo-pr: Z A
openssl vs. neqo-pr: LR M A CM
picoquic vs. neqo-pr: Z
quic-go vs. neqo-pr: CM
quiche vs. neqo-pr: Z 🚀C1 CM
quinn vs. neqo-pr: Z ⚠️L1 V2 CM
s2n-quic vs. neqo-pr: CM
tquic vs. neqo-pr: Z CM
xquic vs. neqo-pr: M CM
All results

Succeeded Interop Tests

QUIC Interop Runner, client vs. server

neqo-pr as client

neqo-pr as server

Unsupported Interop Tests

QUIC Interop Runner, client vs. server

neqo-pr as client

neqo-pr as server

@github-actions
Copy link
Copy Markdown
Contributor

Client/server transfer results

Performance differences relative to b50af73.

Transfer of 33554432 bytes over loopback, min. 100 runs. All unit-less numbers are in milliseconds.

Client vs. server (params) Mean ± σ Min Max MiB/s ± σ Δ baseline Δ baseline
neqo-msquic-cubic 154.3 ± 4.5 147.1 160.9 207.4 ± 7.1 💔 1.3 0.9%
neqo-neqo-newreno-nopacing 98.5 ± 4.2 86.9 107.7 324.9 ± 7.6 💔 2.8 2.9%
s2n-neqo-cubic 220.0 ± 4.6 211.7 236.0 145.5 ± 7.0 💚 -2.1 -1.0%

Table above only shows statistically significant changes. See all results below.

All results

Transfer of 33554432 bytes over loopback, min. 100 runs. All unit-less numbers are in milliseconds.

Client vs. server (params) Mean ± σ Min Max MiB/s ± σ Δ baseline Δ baseline
google-google-nopacing 451.6 ± 3.8 445.7 463.2 70.9 ± 8.4
google-neqo-cubic 270.4 ± 3.9 262.9 278.2 118.4 ± 8.2 -0.8 -0.3%
msquic-msquic-nopacing 180.8 ± 96.4 114.6 579.1 177.0 ± 0.3
msquic-neqo-cubic 199.3 ± 86.0 125.3 498.4 160.6 ± 0.4 -17.4 -8.0%
neqo-google-cubic 756.7 ± 8.7 747.3 810.0 42.3 ± 3.7 1.4 0.2%
neqo-msquic-cubic 154.3 ± 4.5 147.1 160.9 207.4 ± 7.1 💔 1.3 0.9%
neqo-neqo-cubic 97.0 ± 4.4 86.5 106.0 329.9 ± 7.3 0.8 0.8%
neqo-neqo-cubic-nopacing 95.4 ± 4.4 86.8 107.0 335.3 ± 7.3 -1.1 -1.1%
neqo-neqo-newreno 96.7 ± 4.4 86.1 114.7 330.8 ± 7.3 0.4 0.4%
neqo-neqo-newreno-nopacing 98.5 ± 4.2 86.9 107.7 324.9 ± 7.6 💔 2.8 2.9%
neqo-quiche-cubic 190.0 ± 3.6 184.9 203.6 168.4 ± 8.9 0.2 0.1%
neqo-s2n-cubic 220.8 ± 4.0 213.2 227.5 144.9 ± 8.0 -0.1 -0.1%
quiche-neqo-cubic 177.9 ± 4.6 171.0 192.2 179.9 ± 7.0 -0.1 -0.1%
quiche-quiche-nopacing 140.1 ± 5.4 134.5 172.5 228.5 ± 5.9
s2n-neqo-cubic 220.0 ± 4.6 211.7 236.0 145.5 ± 7.0 💚 -2.1 -1.0%
s2n-s2n-nopacing 298.7 ± 27.6 281.2 392.8 107.1 ± 1.2

Download data for profiler.firefox.com or download performance comparison data.

@larseggert larseggert added this pull request to the merge queue Apr 21, 2026
Merged via the queue into mozilla:main with commit f7d28ed Apr 21, 2026
180 of 181 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add and use assert_contains_handshake()

2 participants