feat(pyamber): route ranges by unique receivers - #8246
Conversation
Backport auto-label reportThis
|
Automated Reviewer SuggestionsBased on the
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8246 +/- ##
============================================
- Coverage 93.93% 93.89% -0.05%
+ Complexity 4762 4761 -1
============================================
Files 1191 1191
Lines 48308 49051 +743
Branches 5377 5377
============================================
+ Hits 45380 46058 +678
- Misses 1494 1558 +64
- Partials 1434 1435 +1
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 399 | 0.244 | 24,044/36,932/36,932 us | 🔴 +8.2% / 🔴 +137.1% |
| 🔴 | bs=100 sw=10 sl=64 | 810 | 0.495 | 121,866/158,134/158,134 us | 🔴 +6.9% / 🔴 +47.5% |
| 🔴 | bs=1000 sw=10 sl=64 | 899 | 0.548 | 1,110,559/1,229,182/1,229,182 us | 🔴 +5.8% / 🔴 +20.1% |
Baseline details
Latest main 50321e4 from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 399 tuples/sec | 413 tuples/sec | 787.98 tuples/sec | -3.4% | -49.4% |
| bs=10 sw=10 sl=64 | MB/s | 0.244 MB/s | 0.252 MB/s | 0.481 MB/s | -3.2% | -49.3% |
| bs=10 sw=10 sl=64 | p50 | 24,044 us | 22,224 us | 12,593 us | +8.2% | +90.9% |
| bs=10 sw=10 sl=64 | p95 | 36,932 us | 36,763 us | 15,579 us | +0.5% | +137.1% |
| bs=10 sw=10 sl=64 | p99 | 36,932 us | 36,763 us | 18,786 us | +0.5% | +96.6% |
| bs=100 sw=10 sl=64 | throughput | 810 tuples/sec | 833 tuples/sec | 1,008 tuples/sec | -2.8% | -19.6% |
| bs=100 sw=10 sl=64 | MB/s | 0.495 MB/s | 0.509 MB/s | 0.615 MB/s | -2.8% | -19.5% |
| bs=100 sw=10 sl=64 | p50 | 121,866 us | 116,799 us | 100,701 us | +4.3% | +21.0% |
| bs=100 sw=10 sl=64 | p95 | 158,134 us | 147,868 us | 107,244 us | +6.9% | +47.5% |
| bs=100 sw=10 sl=64 | p99 | 158,134 us | 147,868 us | 116,122 us | +6.9% | +36.2% |
| bs=1000 sw=10 sl=64 | throughput | 899 tuples/sec | 909 tuples/sec | 1,042 tuples/sec | -1.1% | -13.7% |
| bs=1000 sw=10 sl=64 | MB/s | 0.548 MB/s | 0.555 MB/s | 0.636 MB/s | -1.3% | -13.8% |
| bs=1000 sw=10 sl=64 | p50 | 1,110,559 us | 1,100,624 us | 981,959 us | +0.9% | +13.1% |
| bs=1000 sw=10 sl=64 | p95 | 1,229,182 us | 1,161,736 us | 1,023,080 us | +5.8% | +20.1% |
| bs=1000 sw=10 sl=64 | p99 | 1,229,182 us | 1,161,736 us | 1,051,697 us | +5.8% | +16.9% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,500.96,200,128000,399,0.244,24043.72,36932.02,36932.02
1,100,10,64,20,2467.87,2000,1280000,810,0.495,121865.97,158133.72,158133.72
2,1000,10,64,20,22258.91,20000,12800000,899,0.548,1110558.57,1229181.92,1229181.92|
I am converting this PR to a draft: please follow our PR template and mark it ready for review. |
|
The description now follows the current template, and this is ready for review. |
What changes were proposed in this PR?
Range shuffle now divides its key range by the deduplicated receiver count used for routing. Repeated topology channels therefore cannot produce an index beyond the receiver list.
Any related issues, documentation, discussions?
Closes #8245
How was this PR tested?
Direct reproduction after the fix:
Was this PR authored or co-authored using generative AI tooling?
Generated-by: Codex