test(workflow-operator): add unit tests for ImageTaskCodegen - #6659
test(workflow-operator): add unit tests for ImageTaskCodegen#6659Ma77Ball wants to merge 8 commits into
Conversation
Automated Reviewer SuggestionsBased on the
|
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 552 | 0.337 | 17,406/27,952/27,952 us | 🔴 +9.3% / 🔴 +73.4% |
| 🔴 | bs=100 sw=10 sl=64 | 1,162 | 0.709 | 82,135/126,238/126,238 us | 🔴 +5.7% / 🟢 -19.0% |
| ⚪ | bs=1000 sw=10 sl=64 | 1,345 | 0.821 | 739,725/792,238/792,238 us | ⚪ within ±5% / 🟢 +31.5% |
Baseline details
Latest main 964e915 from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 552 tuples/sec | 566 tuples/sec | 760.2 tuples/sec | -2.5% | -27.4% |
| bs=10 sw=10 sl=64 | MB/s | 0.337 MB/s | 0.346 MB/s | 0.464 MB/s | -2.6% | -27.4% |
| bs=10 sw=10 sl=64 | p50 | 17,406 us | 16,832 us | 12,781 us | +3.4% | +36.2% |
| bs=10 sw=10 sl=64 | p95 | 27,952 us | 25,575 us | 16,118 us | +9.3% | +73.4% |
| bs=10 sw=10 sl=64 | p99 | 27,952 us | 25,575 us | 19,074 us | +9.3% | +46.5% |
| bs=100 sw=10 sl=64 | throughput | 1,162 tuples/sec | 1,173 tuples/sec | 988.68 tuples/sec | -0.9% | +17.5% |
| bs=100 sw=10 sl=64 | MB/s | 0.709 MB/s | 0.716 MB/s | 0.603 MB/s | -1.0% | +17.5% |
| bs=100 sw=10 sl=64 | p50 | 82,135 us | 82,405 us | 101,448 us | -0.3% | -19.0% |
| bs=100 sw=10 sl=64 | p95 | 126,238 us | 119,411 us | 107,410 us | +5.7% | +17.5% |
| bs=100 sw=10 sl=64 | p99 | 126,238 us | 119,411 us | 120,258 us | +5.7% | +5.0% |
| bs=1000 sw=10 sl=64 | throughput | 1,345 tuples/sec | 1,340 tuples/sec | 1,023 tuples/sec | +0.4% | +31.5% |
| bs=1000 sw=10 sl=64 | MB/s | 0.821 MB/s | 0.818 MB/s | 0.624 MB/s | +0.4% | +31.5% |
| bs=1000 sw=10 sl=64 | p50 | 739,725 us | 743,371 us | 983,888 us | -0.5% | -24.8% |
| bs=1000 sw=10 sl=64 | p95 | 792,238 us | 788,437 us | 1,030,100 us | +0.5% | -23.1% |
| bs=1000 sw=10 sl=64 | p99 | 792,238 us | 788,437 us | 1,056,513 us | +0.5% | -25.0% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,362.04,200,128000,552,0.337,17406.30,27951.71,27951.71
1,100,10,64,20,1720.65,2000,1280000,1162,0.709,82134.90,126238.26,126238.26
2,1000,10,64,20,14868.02,20000,12800000,1345,0.821,739725.47,792237.73,792237.73
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #6659 +/- ##
============================================
- Coverage 74.96% 74.68% -0.28%
- Complexity 3446 3450 +4
============================================
Files 1160 1157 -3
Lines 45795 45626 -169
Branches 5070 5027 -43
============================================
- Hits 34328 34074 -254
- Misses 9824 9902 +78
- Partials 1643 1650 +7
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Please fix the |
fc7c2b6 to
70db7be
Compare
There was a problem hiding this comment.
Pull request overview
Adds a dedicated ScalaTest spec for the Hugging Face ImageTaskCodegen to validate its handled task set and to regression-test key payload/parse snippet branches (including determinism and “no CodegenContext string leakage”).
Changes:
- Introduce
ImageTaskCodegenSpeccovering the nine-task mapping and core payload/parse branches. - Add regression checks for snippet determinism across
CodegenContextvalues and for absence of raw context-string inlining.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Matthew B. <mgball@uci.edu>
…xera into test/image-task-codegen-spec
Head branch was pushed to by a user without write access
|
This test is failing the CI. |
|
I had to lint the branch. It should work now. |
What changes were proposed in this PR?
common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/huggingFace/codegen/ImageTaskCodegenSpec.scala, a new ScalaTest spec for ImageTaskCodegen, which previously had no dedicated unit tests.Any related issues, documentation, discussions?
Closes: #6658
How was this PR tested?
sbt "WorkflowOperator/testOnly *ImageTaskCodegenSpec", expect all 15 tests passing.Was this PR authored or co-authored using generative AI tooling?
Co-authored with Claude Opus 4.8 in compliance with ASF