Skip to content

vortex-cuda format in benchmarks#6468

Closed
onursatici wants to merge 1 commit intodevelopfrom
os/vortex-cuda-files
Closed

vortex-cuda format in benchmarks#6468
onursatici wants to merge 1 commit intodevelopfrom
os/vortex-cuda-files

Conversation

@onursatici
Copy link
Contributor

Does this PR closes an open issue or discussion?

  • Closes #.

What changes are included in this PR?

allows benchmarks to create and ran against vortex-cuda files, which are vortex files that only include cuda compatible encodings

What is the rationale for this change?

How is this change tested?

Are there any user-facing changes?

Signed-off-by: Onur Satici <onur@spiraldb.com>
@onursatici onursatici added the changelog/chore A trivial change label Feb 12, 2026
@codspeed-hq
Copy link

codspeed-hq bot commented Feb 12, 2026

Merging this PR will degrade performance by 21.88%

❌ 16 regressed benchmarks
✅ 1119 untouched benchmarks
⏩ 1268 skipped benchmarks1

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation old_raw_prim_test_between[f32, 2048] 113.4 µs 144.7 µs -21.59%
Simulation old_raw_prim_test_between[f64, 16384] 267.6 µs 298.6 µs -10.39%
Simulation old_raw_prim_test_between[f64, 2048] 118.3 µs 149.8 µs -21.04%
Simulation old_raw_prim_test_between[f32, 16384] 232.3 µs 262.4 µs -11.49%
Simulation old_raw_prim_test_between[i32, 16384] 233.3 µs 263.3 µs -11.39%
Simulation old_raw_prim_test_between[i64, 2048] 120.2 µs 151.2 µs -20.46%
Simulation old_raw_prim_test_between[u32, 16384] 236.4 µs 266.1 µs -11.16%
Simulation old_raw_prim_test_between[i32, 2048] 113.2 µs 144.9 µs -21.88%
Simulation old_raw_prim_test_between[u64, 2048] 121 µs 151.3 µs -20.03%
Simulation old_alp_prim_test_between[f32, 2048] 139.7 µs 178 µs -21.5%
Simulation old_raw_prim_test_between[u32, 2048] 114.3 µs 145.3 µs -21.35%
Simulation old_alp_prim_test_between[f64, 2048] 146.2 µs 178 µs -17.87%
Simulation old_bp_prim_test_between[i16, 2048] 133 µs 161.9 µs -17.82%
Simulation old_bp_prim_test_between[i16, 16384] 267.9 µs 297.8 µs -10.04%
Simulation old_bp_prim_test_between[i32, 2048] 133 µs 162.6 µs -18.2%
Simulation old_bp_prim_test_between[i64, 2048] 140.4 µs 170.9 µs -17.84%

Comparing os/vortex-cuda-files (e878f8e) with develop (18b5846)2

Open in CodSpeed

Footnotes

  1. 1268 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on develop (001c920) during the generation of this report, so 18b5846 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@robert3005
Copy link
Contributor

If we want to use this we should also add GPU runners to benchmarks. I think it's too early to focus on benchmarking these

Copy link
Contributor

@joseph-isaacs joseph-isaacs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you clean this up once its gone

@onursatici
Copy link
Contributor Author

If we have different strategies we expose, we should be able to generate benchmark data with all of those. This one is named cuda compatible but it is just a subset of encodings like the compact encodings, and I should be able to generate those and run my existing benchmark code on it. If I choose to run the query benchmark as is, it will run but will be executed on the cpu. If I have a gpu benchmark in the future, I still can use the data-gen binary to generate the files and now run on the gpu.

That being said, I am happy to keep this on my branch

@onursatici onursatici closed this Feb 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/chore A trivial change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants