Skip to content

Conversation

@gatesn
Copy link
Contributor

@gatesn gatesn commented Oct 27, 2025

  • It's a little aggressive to require an async runtime simply to decompress in-memory data.
  • Async expressions tend to require more powerful APIs anyway. For example, returning partial results since the output can be much larger than the input (not common for in-memory scalar compute). As such, I propose implementing async expressions as layout readers (with an improved API), see Async Expressions #5089

Signed-off-by: Nicholas Gates <nick@nickgates.com>
@gatesn gatesn added the chore Release label indicating a trivial change label Oct 27, 2025
@gatesn gatesn requested a review from connortsui20 October 27, 2025 16:49
@gatesn gatesn enabled auto-merge (squash) October 27, 2025 16:49
@codspeed-hq
Copy link

codspeed-hq bot commented Oct 27, 2025

CodSpeed Performance Report

Merging #5090 will degrade performances by 25.18%

Comparing ngates/sync-operators (ea7fa9b) with develop (10a2c86)1

Summary

⚡ 1 improvement
❌ 1 regression
✅ 1316 untouched
🆕 7 new
⏩ 22 skipped2

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark BASE HEAD Change
new_alp_prim_test_between[f64, 32768] 174.1 µs 232.7 µs -25.18%
new_bp_prim_test_between[i64, 32768] 217.2 µs 158.2 µs +37.33%
🆕 decompress[("alp_for_bp_f64", 0x755670)] N/A 24.2 ms N/A
🆕 decompress[("datetime_for_bp", 0x758950)] N/A 34.9 ms N/A
🆕 decompress[("dict_fsst_varbin_bp_string", 0x757a60)] N/A 14.5 ms N/A
🆕 decompress[("dict_fsst_varbin_string", 0x7573d0)] N/A 14.5 ms N/A
🆕 decompress[("dict_varbinview_string", 0x756090)] N/A 14.7 ms N/A
🆕 decompress[("for_bp_u64", 0x754f20)] N/A 2.5 ms N/A
🆕 decompress[("runend_for_bp_u32", 0x756520)] N/A 2 ms N/A

Footnotes

  1. No successful run was found on develop (e570ee4) during the generation of this report, so 10a2c86 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

  2. 22 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@codecov
Copy link

codecov bot commented Oct 27, 2025

Codecov Report

❌ Patch coverage is 77.94118% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.57%. Comparing base (10a2c86) to head (ea7fa9b).
⚠️ Report is 6 commits behind head on develop.

Files with missing lines Patch % Lines
vortex-array/src/execution/validity.rs 27.27% 8 Missing ⚠️
vortex-array/src/compute/arrays/logical.rs 82.60% 4 Missing ⚠️
vortex-array/src/vtable/operator.rs 0.00% 2 Missing ⚠️
vortex-array/src/execution/mask.rs 91.66% 1 Missing ⚠️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Contributor

@connortsui20 connortsui20 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙏

@gatesn gatesn merged commit bd25e4a into develop Oct 27, 2025
38 of 40 checks passed
@gatesn gatesn deleted the ngates/sync-operators branch October 27, 2025 17:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

chore Release label indicating a trivial change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants