feat(pyamber): compare table row counts - #8230
Conversation
Automated Reviewer SuggestionsBased on the
|
Backport auto-label reportThis
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8230 +/- ##
=========================================
Coverage 93.94% 93.94%
Complexity 4759 4759
=========================================
Files 1185 1185
Lines 48069 48069
Branches 5359 5359
=========================================
Hits 45159 45159
Misses 1480 1480
Partials 1430 1430
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 378 | 0.231 | 25,332/34,588/34,588 us | 🔴 -5.5% / 🔴 +122.0% |
| ⚪ | bs=100 sw=10 sl=64 | 791 | 0.483 | 123,662/157,465/157,465 us | ⚪ within ±5% / 🔴 +46.8% |
| ⚪ | bs=1000 sw=10 sl=64 | 928 | 0.566 | 1,078,004/1,136,247/1,136,247 us | ⚪ within ±5% / 🔴 +11.1% |
Baseline details
Latest main 70c2114 from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 378 tuples/sec | 400 tuples/sec | 787.98 tuples/sec | -5.5% | -52.0% |
| bs=10 sw=10 sl=64 | MB/s | 0.231 MB/s | 0.244 MB/s | 0.481 MB/s | -5.3% | -52.0% |
| bs=10 sw=10 sl=64 | p50 | 25,332 us | 24,686 us | 12,593 us | +2.6% | +101.2% |
| bs=10 sw=10 sl=64 | p95 | 34,588 us | 34,667 us | 15,579 us | -0.2% | +122.0% |
| bs=10 sw=10 sl=64 | p99 | 34,588 us | 34,667 us | 18,786 us | -0.2% | +84.1% |
| bs=100 sw=10 sl=64 | throughput | 791 tuples/sec | 800 tuples/sec | 1,008 tuples/sec | -1.1% | -21.5% |
| bs=100 sw=10 sl=64 | MB/s | 0.483 MB/s | 0.488 MB/s | 0.615 MB/s | -1.0% | -21.5% |
| bs=100 sw=10 sl=64 | p50 | 123,662 us | 121,171 us | 100,701 us | +2.1% | +22.8% |
| bs=100 sw=10 sl=64 | p95 | 157,465 us | 162,898 us | 107,244 us | -3.3% | +46.8% |
| bs=100 sw=10 sl=64 | p99 | 157,465 us | 162,898 us | 116,122 us | -3.3% | +35.6% |
| bs=1000 sw=10 sl=64 | throughput | 928 tuples/sec | 926 tuples/sec | 1,042 tuples/sec | +0.2% | -10.9% |
| bs=1000 sw=10 sl=64 | MB/s | 0.566 MB/s | 0.565 MB/s | 0.636 MB/s | +0.2% | -11.0% |
| bs=1000 sw=10 sl=64 | p50 | 1,078,004 us | 1,080,273 us | 981,959 us | -0.2% | +9.8% |
| bs=1000 sw=10 sl=64 | p95 | 1,136,247 us | 1,151,371 us | 1,023,080 us | -1.3% | +11.1% |
| bs=1000 sw=10 sl=64 | p99 | 1,136,247 us | 1,151,371 us | 1,051,697 us | -1.3% | +8.0% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,528.67,200,128000,378,0.231,25331.69,34587.90,34587.90
1,100,10,64,20,2528.05,2000,1280000,791,0.483,123661.73,157465.42,157465.42
2,1000,10,64,20,21562.54,20000,12800000,928,0.566,1078004.06,1136247.40,1136247.40|
I am converting this PR to a draft: please follow our PR template and mark it ready for review. |
|
The description now follows the current template, and this is ready for review. |
What changes were proposed in this PR?
Require equal row counts before comparing table rows. This prevents the existing
zipcomparison from treating a shorter matching prefix as a fully equal table.Any related issues, documentation, discussions?
Closes #8229
How was this PR tested?
A direct post-fix probe covered different lengths in both directions and the equal case:
Was this PR authored or co-authored using generative AI tooling?
Generated-by: Codex