chore: Add quic-zig to QNS matrix#3556
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates the QNS PR workflow to include the quic-zig implementation in the set of additional QUIC implementations used for interop pair generation, expanding coverage of Neqo’s PR/baseline interop testing against another external stack.
Changes:
- Add a reference comment for
quic-zigin the workflow’s “non-default implementations” list. - Extend the generated
additional.jsonimplementations map to includequic-zig(image/url/role), so it participates in the QNS interop pair generation.
There was a problem hiding this comment.
Clean, minimal addition of quic-zig to the QNS interop matrix. The new entry is structurally consistent with the existing non-default implementations (tquic, openssl, linuxquic): comment referencing the upstream quic-interop-runner PR, proper JSON with image/url/role, and correct trailing-comma fixup on the preceding linuxquic block. role: "both" will generate client and server pairs against both neqo-pr and neqo-baseline, which is the right default for a full QUIC stack.
No issues found.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3556 +/- ##
==========================================
- Coverage 94.55% 94.43% -0.13%
==========================================
Files 127 131 +4
Lines 39627 39957 +330
Branches 39627 39957 +330
==========================================
+ Hits 37470 37734 +264
- Misses 1318 1377 +59
- Partials 839 846 +7
Flags with carried forward coverage won't be shown. Click here to find out more.
|
Benchmark resultsNo significant performance differences relative to b50af73. All resultstransfer/1-conn/1-100mb-resp (aka. Download)/mtu-1504: No change in performance detected. time: [203.97 ms 204.28 ms 204.63 ms]
thrpt: [488.69 MiB/s 489.51 MiB/s 490.26 MiB/s]
change:
time: [-0.2510% -0.0168% +0.2110] (p = 0.89 > 0.05)
thrpt: [-0.2106% +0.0168% +0.2516]
No change in performance detected.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severetransfer/1-conn/10_000-parallel-1b-resp (aka. RPS)/mtu-1504: Change within noise threshold. time: [284.32 ms 286.12 ms 287.93 ms]
thrpt: [34.731 Kelem/s 34.950 Kelem/s 35.172 Kelem/s]
change:
time: [-2.1356% -1.1894% -0.3035] (p = 0.01 < 0.05)
thrpt: [+0.3044% +1.2037% +2.1822]
Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mildtransfer/1-conn/1-1b-resp (aka. HPS)/mtu-1504: No change in performance detected. time: [38.661 ms 38.799 ms 38.953 ms]
thrpt: [25.672 B/s 25.774 B/s 25.866 B/s]
change:
time: [-1.1479% -0.5082% +0.1171] (p = 0.12 > 0.05)
thrpt: [-0.1169% +0.5108% +1.1612]
No change in performance detected.
Found 6 outliers among 100 measurements (6.00%)
2 (2.00%) high mild
4 (4.00%) high severetransfer/1-conn/1-100mb-req (aka. Upload)/mtu-1504: Change within noise threshold. time: [207.43 ms 207.73 ms 208.03 ms]
thrpt: [480.71 MiB/s 481.40 MiB/s 482.09 MiB/s]
change:
time: [+0.2384% +0.5955% +0.8963] (p = 0.00 < 0.05)
thrpt: [-0.8884% -0.5920% -0.2378]
Change within noise threshold.streams/walltime/1-streams/each-1000-bytes: No change in performance detected. time: [593.03 µs 594.71 µs 596.76 µs]
change: [-0.5091% -0.0290% +0.4428] (p = 0.91 > 0.05)
No change in performance detected.
Found 6 outliers among 100 measurements (6.00%)
6 (6.00%) high severestreams/walltime/1000-streams/each-1-bytes: No change in performance detected. time: [12.438 ms 12.457 ms 12.478 ms]
change: [-0.6573% -0.2574% +0.0467] (p = 0.16 > 0.05)
No change in performance detected.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severestreams/walltime/1000-streams/each-1000-bytes: Change within noise threshold. time: [45.794 ms 45.835 ms 45.878 ms]
change: [+0.5600% +0.6994% +0.8415] (p = 0.00 < 0.05)
Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mildtransfer/walltime/pacing-false/varying-seeds: No change in performance detected. time: [80.254 ms 80.369 ms 80.527 ms]
change: [-0.3882% -0.1364% +0.0861] (p = 0.29 > 0.05)
No change in performance detected.
Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) low mild
1 (1.00%) high mild
1 (1.00%) high severetransfer/walltime/pacing-true/varying-seeds: No change in performance detected. time: [80.296 ms 80.404 ms 80.540 ms]
change: [-0.2695% -0.1179% +0.0684] (p = 0.20 > 0.05)
No change in performance detected.
Found 6 outliers among 100 measurements (6.00%)
1 (1.00%) low mild
5 (5.00%) high severetransfer/walltime/pacing-false/same-seed: No change in performance detected. time: [78.815 ms 78.902 ms 79.012 ms]
change: [-0.2061% -0.0527% +0.1279] (p = 0.55 > 0.05)
No change in performance detected.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severetransfer/walltime/pacing-true/same-seed: Change within noise threshold. time: [80.247 ms 80.317 ms 80.401 ms]
change: [-0.5438% -0.2862% -0.0855] (p = 0.01 < 0.05)
Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) low mild
1 (1.00%) high mild
1 (1.00%) high severeDownload data for |
Failed Interop TestsQUIC Interop Runner, client vs. server, differences relative to
All resultsSucceeded Interop TestsQUIC Interop Runner, client vs. server neqo-pr as client
neqo-pr as server
Unsupported Interop TestsQUIC Interop Runner, client vs. server neqo-pr as client
neqo-pr as server
|
Client/server transfer resultsPerformance differences relative to b50af73. Transfer of 33554432 bytes over loopback, min. 100 runs. All unit-less numbers are in milliseconds.
Table above only shows statistically significant changes. See all results below. All resultsTransfer of 33554432 bytes over loopback, min. 100 runs. All unit-less numbers are in milliseconds.
Download data for |
* Events for SCONE Clean version of #3448. Closes #3448. * Restore missing file * reduce noisiness Co-authored-by: Lars Eggert <lars@eggert.org> Signed-off-by: Martin Thomson <mt@lowentropy.net> * Revert comment change Signed-off-by: Martin Thomson <mt@lowentropy.net> * Address review comments, iterate This uses a loop rather than recursion by popular demand. It also improves the logic for updates, so that updates occur when receiving any packet, in case enough time has elapsed. This resets the state back to "unknown" after one period. Tests to match, plus some improved factoring. * Access PartialOrd directly Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Martin Thomson <mt@lowentropy.net> * Clippy * Factor SCONE to track state on the path * Take a suggestion for commenting Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Martin Thomson <mt@lowentropy.net> * chore: Use Claude PR review from `mozilla/actions` (#3559) * chore: Add `quic-zig` to QNS matrix (#3556) * At least the suggestion could be properly formatted... * OK, that was far worse a suggestion than I thought --------- Signed-off-by: Martin Thomson <mt@lowentropy.net> Co-authored-by: Lars Eggert <lars@eggert.org> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
No description provided.