Skip to content

perf: fsst decompress#7486

Open
joseph-isaacs wants to merge 4 commits intodevelopfrom
ji/fsst-decompress
Open

perf: fsst decompress#7486
joseph-isaacs wants to merge 4 commits intodevelopfrom
ji/fsst-decompress

Conversation

@joseph-isaacs
Copy link
Copy Markdown
Contributor

@joseph-isaacs joseph-isaacs commented Apr 16, 2026

Summary

Closes: #000

Testing

@joseph-isaacs
perf
e0977f6 
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@joseph-isaacs joseph-isaacs added the action/benchmark Trigger full benchmarks to run on this PR label Apr 16, 2026
@github-actions github-actions bot removed the action/benchmark Trigger full benchmarks to run on this PR label Apr 16, 2026
@codspeed-hq
Copy link
Copy Markdown

codspeed-hq bot commented Apr 16, 2026 •
edited
Loading

Merging this PR will degrade performance by 23.81%

⚡ 6 improved benchmarks
❌ 24 regressed benchmarks
✅ 1133 untouched benchmarks
⏩ 1457 skipped benchmarks1

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ Simulation take_search[(0.005, 0.05)] 132 µs 168.5 µs -21.65%
❌ Simulation take_search[(0.005, 0.1)] 247.7 µs 320.6 µs -22.74%
❌ Simulation take_search[(0.01, 0.5)] 1.3 ms 1.6 ms -22.21%
❌ Simulation take_search[(0.005, 0.5)] 1.2 ms 1.5 ms -23.67%
❌ Simulation take_search[(0.005, 1.0)] 2.3 ms 3.1 ms -23.81%
❌ Simulation take_search[(0.01, 0.05)] 143.1 µs 179.6 µs -20.32%
❌ Simulation take_search[(0.01, 0.1)] 268.5 µs 341.4 µs -21.35%
❌ Simulation take_search[(0.01, 1.0)] 2.5 ms 3.3 ms -22.34%
❌ Simulation take_search[(0.1, 0.5)] 1.8 ms 2.2 ms -16.94%
❌ Simulation take_search_chunked[(0.005, 0.05)] 169.9 µs 201 µs -15.46%
❌ Simulation take_search[(0.1, 0.1)] 386.1 µs 459 µs -15.89%
❌ Simulation take_search[(0.1, 0.05)] 212.8 µs 249.3 µs -14.64%
❌ Simulation take_search_chunked[(0.01, 0.1)] 346.7 µs 408.7 µs -15.17%
❌ Simulation take_search[(0.1, 1.0)] 3.5 ms 4.3 ms -17.1%
❌ Simulation take_search_chunked[(0.1, 0.1)] 457.3 µs 519.3 µs -11.95%
❌ Simulation take_search_chunked[(0.01, 1.0)] 3.3 ms 3.9 ms -15.76%
❌ Simulation take_search_chunked[(0.01, 0.05)] 182.8 µs 213.9 µs -14.52%
❌ Simulation take_search_chunked[(0.1, 0.05)] 248.8 µs 279.9 µs -11.1%
❌ Simulation take_search_chunked[(0.01, 0.5)] 1.7 ms 2 ms -15.69%
❌ Simulation take_search_chunked[(0.005, 0.5)] 1.5 ms 1.9 ms -16.7%
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Comparing ji/fsst-decompress (edd6b0d) with develop (ce52b71)

Open in CodSpeed

Footnotes

  1. 1457 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2026 •
edited
Loading

Polar Signals Profiling Results

Latest Run

Status Commit Job Attempt Link
🟢 Done edd6b0d 1 Explore Profiling Data
Previous Runs (1)
Status Commit Job Attempt Link
🟢 Done e0977f6 1 Explore Profiling Data

Powered by Polar Signals Cloud

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2026 •
edited
Loading

🚨🚨🚨❌❌❌ SQL BENCHMARK FAILED ❌❌❌🚨🚨🚨

Benchmark PolarSignals Profiling failed! Check the workflow run for details.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2026

File Sizes: PolarSignals Profiling

No file size changes detected.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2026 •
edited
Loading

🚨🚨🚨❌❌❌ SQL BENCHMARK FAILED ❌❌❌🚨🚨🚨

Benchmark FineWeb NVMe failed! Check the workflow run for details.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2026

File Sizes: FineWeb NVMe

No file size changes detected.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2026 •
edited
Loading

🚨🚨🚨❌❌❌ SQL BENCHMARK FAILED ❌❌❌🚨🚨🚨

Benchmark TPC-H SF=1 on NVME failed! Check the workflow run for details.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2026

File Sizes: TPC-H SF=1 on NVME

No file size changes detected.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2026 •
edited
Loading

🚨🚨🚨❌❌❌ SQL BENCHMARK FAILED ❌❌❌🚨🚨🚨

Benchmark TPC-DS SF=1 on NVME failed! Check the workflow run for details.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2026

File Sizes: TPC-DS SF=1 on NVME

No file size changes detected.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2026 •
edited
Loading

🚨🚨🚨❌❌❌ SQL BENCHMARK FAILED ❌❌❌🚨🚨🚨

Benchmark FineWeb S3 failed! Check the workflow run for details.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2026 •
edited
Loading

Benchmarks: Random Access

Vortex (geomean): 0.963x ➖
Parquet (geomean): 1.102x ❌

unknown / unknown (1.057x ➖, 4↑ 24↓)
name PR edd6b0d (ns) base ce52b71 (ns) ratio (PR/base)
random-access/feature-vectors/correlated/lance-tokio-local-disk 2562331 2726422 0.94
random-access/feature-vectors/correlated/lance-tokio-local-disk-footer 🚨 1463771 1293547 1.13
random-access/feature-vectors/correlated/parquet-tokio-local-disk 12303426923 11582036688 1.06
random-access/feature-vectors/correlated/parquet-tokio-local-disk-footer 11695187082 11284649648 1.04
random-access/feature-vectors/correlated/vortex-tokio-local-disk 9014331 9597302 0.94
random-access/feature-vectors/correlated/vortex-tokio-local-disk-footer 7886986 7999156 0.99
random-access/feature-vectors/uniform/lance-tokio-local-disk 🚨 1788926 1485583 1.20
random-access/feature-vectors/uniform/lance-tokio-local-disk-footer 🚨 2812094 2405726 1.17
random-access/feature-vectors/uniform/parquet-tokio-local-disk 11741877379 11281254010 1.04
random-access/feature-vectors/uniform/parquet-tokio-local-disk-footer 11729077252 11180983607 1.05
random-access/feature-vectors/uniform/vortex-tokio-local-disk 14981991 14878159 1.01
random-access/feature-vectors/uniform/vortex-tokio-local-disk-footer 14759574 14974703 0.99
random-access/lance-tokio-local-disk 🚨 818030 733918 1.11
random-access/lance-tokio-local-disk-footer 🚨 1635893 1340238 1.22
random-access/nested-lists/correlated/lance-tokio-local-disk 🚨 297433 269118 1.11
random-access/nested-lists/correlated/lance-tokio-local-disk-footer 🚨 805587 705123 1.14
random-access/nested-lists/correlated/parquet-tokio-local-disk 🚨 151091538 133197659 1.13
random-access/nested-lists/correlated/parquet-tokio-local-disk-footer 🚨 149972515 133355701 1.12
random-access/nested-lists/correlated/vortex-tokio-local-disk 🚨 797656 722035 1.10
random-access/nested-lists/correlated/vortex-tokio-local-disk-footer 🚨 991997 736863 1.35
random-access/nested-lists/uniform/lance-tokio-local-disk 🚨 1328278 1170298 1.13
random-access/nested-lists/uniform/lance-tokio-local-disk-footer 🚨 1897896 1645958 1.15
random-access/nested-lists/uniform/parquet-tokio-local-disk 🚨 150962989 133760520 1.13
random-access/nested-lists/uniform/parquet-tokio-local-disk-footer 🚨 148323787 133861061 1.11
random-access/nested-lists/uniform/vortex-tokio-local-disk 2701910 2574511 1.05
random-access/nested-lists/uniform/vortex-tokio-local-disk-footer 🚨 2858369 2523234 1.13
random-access/nested-structs/correlated/lance-tokio-local-disk 475088 461327 1.03
random-access/nested-structs/correlated/lance-tokio-local-disk-footer 726828 703200 1.03
random-access/nested-structs/correlated/parquet-tokio-local-disk 🚨 31285349 26712286 1.17
random-access/nested-structs/correlated/parquet-tokio-local-disk-footer 🚨 31142358 26625604 1.17
random-access/nested-structs/correlated/vortex-tokio-local-disk 1050945 978159 1.07
random-access/nested-structs/correlated/vortex-tokio-local-disk-footer 1023468 980045 1.04
random-access/nested-structs/uniform/lance-tokio-local-disk 3234488 3046717 1.06
random-access/nested-structs/uniform/lance-tokio-local-disk-footer 3493413 3285254 1.06
random-access/nested-structs/uniform/parquet-tokio-local-disk 🚨 31199374 26447669 1.18
random-access/nested-structs/uniform/parquet-tokio-local-disk-footer 🚨 31544423 26810339 1.18
random-access/nested-structs/uniform/vortex-tokio-local-disk 2125983 2039583 1.04
random-access/nested-structs/uniform/vortex-tokio-local-disk-footer 2092808 2018732 1.04
random-access/parquet-tokio-local-disk 195127070 180823993 1.08
random-access/parquet-tokio-local-disk-footer 196086067 184294576 1.06
random-access/taxi/correlated/lance-tokio-local-disk 🚨 1058382 959869 1.10
random-access/taxi/correlated/lance-tokio-local-disk-footer 🚨 2018987 1636725 1.23
random-access/taxi/correlated/parquet-tokio-local-disk 288751273 270207785 1.07
random-access/taxi/correlated/parquet-tokio-local-disk-footer 295864135 269375309 1.10
random-access/taxi/correlated/vortex-tokio-local-disk 🚀 2020066 3075902 0.66
random-access/taxi/correlated/vortex-tokio-local-disk-footer 🚀 2239685 2970291 0.75
random-access/taxi/uniform/lance-tokio-local-disk 🚨 9543847 8415928 1.13
random-access/taxi/uniform/lance-tokio-local-disk-footer 🚨 10222907 9126608 1.12
random-access/taxi/uniform/parquet-tokio-local-disk 308400640 287190371 1.07
random-access/taxi/uniform/parquet-tokio-local-disk-footer 312522526 287932353 1.09
random-access/taxi/uniform/vortex-tokio-local-disk 5302826 5787580 0.92
random-access/taxi/uniform/vortex-tokio-local-disk-footer 5177777 5690417 0.91
random-access/vortex-tokio-local-disk 🚀 1492283 1868164 0.80
random-access/vortex-tokio-local-disk-footer 🚀 1522170 1944123 0.78

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2026 •
edited
Loading

🚨🚨🚨❌❌❌ SQL BENCHMARK FAILED ❌❌❌🚨🚨🚨

Benchmark Statistical and Population Genetics failed! Check the workflow run for details.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2026

File Sizes: Statistical and Population Genetics

No file size changes detected.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2026 •
edited
Loading

🚨🚨🚨❌❌❌ SQL BENCHMARK FAILED ❌❌❌🚨🚨🚨

Benchmark TPC-H SF=10 on NVME failed! Check the workflow run for details.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2026

File Sizes: TPC-H SF=10 on NVME

No file size changes detected.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2026 •
edited
Loading

🚨🚨🚨❌❌❌ SQL BENCHMARK FAILED ❌❌❌🚨🚨🚨

Benchmark Clickbench on NVME failed! Check the workflow run for details.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2026

File Sizes: Clickbench on NVME

File Size Changes (1 files changed, -0.0% overall, 0↑ 1↓)
File Scale Format Base HEAD Change %
duckdb.db 1.0 vortex-compact 268.00 KB 0 B 268.00 KB -100.0%

Totals:

  • vortex-compact: 7.03 GB → 7.03 GB (-0.0%)
  • vortex-file-compressed: 13.88 GB → 13.88 GB (0.0%)

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2026 •
edited
Loading

🚨🚨🚨❌❌❌ SQL BENCHMARK FAILED ❌❌❌🚨🚨🚨

Benchmark TPC-H SF=1 on S3 failed! Check the workflow run for details.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2026 •
edited
Loading

BENCHMARK FAILED

Benchmark Compression failed! Check the workflow run for details.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2026 •
edited
Loading

🚨🚨🚨❌❌❌ SQL BENCHMARK FAILED ❌❌❌🚨🚨🚨

Benchmark TPC-H SF=10 on S3 failed! Check the workflow run for details.

@joseph-isaacs joseph-isaacs requested a review from a10y April 16, 2026 18:40
@joseph-isaacs
perf
a298dc7 
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@joseph-isaacs
perf
249d691 
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@joseph-isaacs joseph-isaacs added the changelog/performance A performance improvement label Apr 16, 2026
@joseph-isaacs joseph-isaacs marked this pull request as ready for review April 16, 2026 19:50
@joseph-isaacs
perf
edd6b0d 
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@joseph-isaacs joseph-isaacs added the action/benchmark Trigger full benchmarks to run on this PR label Apr 16, 2026
@github-actions github-actions bot removed the action/benchmark Trigger full benchmarks to run on this PR label Apr 16, 2026
a10y
a10y reviewed Apr 16, 2026
View reviewed changes
Comment thread vortex-array/src/arrays/varbinview/build_views.rs
lens: &[P],
) -> (Vec<ByteBuffer>, Buffer<BinaryView>) {
let mut views = BufferMut::<BinaryView>::with_capacity(lens.len());
let views_dst = views.spare_capacity_mut().as_mut_ptr().cast::<BinaryView>();
Copy link
Copy Markdown
Contributor

@a10y a10y Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lol, how much perf are we talkin about here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@a10y a10y a10y left review comments

@connortsui20 connortsui20 Awaiting requested review from connortsui20

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

changelog/performance A performance improvement

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@joseph-isaacs @a10y