Add h2o SQL benchmark#22660
Open
Omega359 wants to merge 1 commit into
Open
Conversation
adriangb
approved these changes
Jun 2, 2026
Contributor
adriangb
left a comment
adriangb
left a comment
There was a problem hiding this comment.
This looks good to my eyes, it's mechanical. It's very nice how the framework you designed already fits all of these shapes including the single dataset multi benchmark setup. I kicked off a Copilot review just to see if there's anything I didn't catch.
Contributor
There was a problem hiding this comment.
Pull request overview
Adds the H2O SQL benchmark suite to DataFusion’s SQL benchmark harness as part of the ongoing migration to SQL-driven benchmarks (#21706). This introduces benchmark definitions for the H2O groupby/join/window subgroups plus corresponding dataset “load” SQL scripts.
Changes:
- Added H2O
.benchmarkfiles forgroupby,join, andwindowsubgroups. - Added H2O
init/SQL scripts to register external tables for CSV and Parquet across small/medium/big dataset sizes.
Reviewed changes
Copilot reviewed 45 out of 45 changed files in this pull request and generated 18 comments.
Show a summary per file
| File | Description |
|---|---|
| benchmarks/sql_benchmarks/h2o/init/load_window_small_parquet.sql | Registers “small” window parquet dataset as external table |
| benchmarks/sql_benchmarks/h2o/init/load_window_small_csv.sql | Registers “small” window csv dataset as external table |
| benchmarks/sql_benchmarks/h2o/init/load_window_medium_parquet.sql | Registers “medium” window parquet dataset as external table |
| benchmarks/sql_benchmarks/h2o/init/load_window_medium_csv.sql | Registers “medium” window csv dataset as external table |
| benchmarks/sql_benchmarks/h2o/init/load_window_big_parquet.sql | Registers “big” window parquet dataset as external table |
| benchmarks/sql_benchmarks/h2o/init/load_window_big_csv.sql | Registers “big” window csv dataset as external table |
| benchmarks/sql_benchmarks/h2o/init/load_join_small_parquet.sql | Registers “small” join parquet datasets (x/small/medium/large) |
| benchmarks/sql_benchmarks/h2o/init/load_join_small_csv.sql | Registers “small” join csv datasets (x/small/medium/large) |
| benchmarks/sql_benchmarks/h2o/init/load_join_medium_parquet.sql | Registers “medium” join parquet datasets (x/small/medium/large) |
| benchmarks/sql_benchmarks/h2o/init/load_join_medium_csv.sql | Registers “medium” join csv datasets (x/small/medium/large) |
| benchmarks/sql_benchmarks/h2o/init/load_join_big_parquet.sql | Registers “big” join parquet datasets (x/small/medium/large) |
| benchmarks/sql_benchmarks/h2o/init/load_join_big_csv.sql | Registers “big” join csv datasets (x/small/medium/large) |
| benchmarks/sql_benchmarks/h2o/init/load_groupby_small_parquet.sql | Registers “small” groupby parquet dataset as external table |
| benchmarks/sql_benchmarks/h2o/init/load_groupby_small_csv.sql | Registers “small” groupby csv dataset as external table |
| benchmarks/sql_benchmarks/h2o/init/load_groupby_medium_parquet.sql | Registers “medium” groupby parquet dataset as external table |
| benchmarks/sql_benchmarks/h2o/init/load_groupby_medium_csv.sql | Registers “medium” groupby csv dataset as external table |
| benchmarks/sql_benchmarks/h2o/init/load_groupby_big_parquet.sql | Registers “big” groupby parquet dataset as external table |
| benchmarks/sql_benchmarks/h2o/init/load_groupby_big_csv.sql | Registers “big” groupby csv dataset as external table |
| benchmarks/sql_benchmarks/h2o/benchmarks/window/q01.benchmark | Window benchmark query Q01 definition |
| benchmarks/sql_benchmarks/h2o/benchmarks/window/q02.benchmark | Window benchmark query Q02 definition |
| benchmarks/sql_benchmarks/h2o/benchmarks/window/q03.benchmark | Window benchmark query Q03 definition |
| benchmarks/sql_benchmarks/h2o/benchmarks/window/q04.benchmark | Window benchmark query Q04 definition |
| benchmarks/sql_benchmarks/h2o/benchmarks/window/q05.benchmark | Window benchmark query Q05 definition |
| benchmarks/sql_benchmarks/h2o/benchmarks/window/q06.benchmark | Window benchmark query Q06 definition |
| benchmarks/sql_benchmarks/h2o/benchmarks/window/q07.benchmark | Window benchmark query Q07 definition |
| benchmarks/sql_benchmarks/h2o/benchmarks/window/q08.benchmark | Window benchmark query Q08 definition |
| benchmarks/sql_benchmarks/h2o/benchmarks/window/q09.benchmark | Window benchmark query Q09 definition |
| benchmarks/sql_benchmarks/h2o/benchmarks/window/q10.benchmark | Window benchmark query Q10 definition |
| benchmarks/sql_benchmarks/h2o/benchmarks/window/q11.benchmark | Window benchmark query Q11 definition |
| benchmarks/sql_benchmarks/h2o/benchmarks/window/q12.benchmark | Window benchmark query Q12 definition |
| benchmarks/sql_benchmarks/h2o/benchmarks/join/q01.benchmark | Join benchmark query Q01 definition |
| benchmarks/sql_benchmarks/h2o/benchmarks/join/q02.benchmark | Join benchmark query Q02 definition |
| benchmarks/sql_benchmarks/h2o/benchmarks/join/q03.benchmark | Join benchmark query Q03 definition |
| benchmarks/sql_benchmarks/h2o/benchmarks/join/q04.benchmark | Join benchmark query Q04 definition |
| benchmarks/sql_benchmarks/h2o/benchmarks/join/q05.benchmark | Join benchmark query Q05 definition |
| benchmarks/sql_benchmarks/h2o/benchmarks/groupby/q01.benchmark | Groupby benchmark query Q01 definition |
| benchmarks/sql_benchmarks/h2o/benchmarks/groupby/q02.benchmark | Groupby benchmark query Q02 definition |
| benchmarks/sql_benchmarks/h2o/benchmarks/groupby/q03.benchmark | Groupby benchmark query Q03 definition |
| benchmarks/sql_benchmarks/h2o/benchmarks/groupby/q04.benchmark | Groupby benchmark query Q04 definition |
| benchmarks/sql_benchmarks/h2o/benchmarks/groupby/q05.benchmark | Groupby benchmark query Q05 definition |
| benchmarks/sql_benchmarks/h2o/benchmarks/groupby/q06.benchmark | Groupby benchmark query Q06 definition |
| benchmarks/sql_benchmarks/h2o/benchmarks/groupby/q07.benchmark | Groupby benchmark query Q07 definition |
| benchmarks/sql_benchmarks/h2o/benchmarks/groupby/q08.benchmark | Groupby benchmark query Q08 definition |
| benchmarks/sql_benchmarks/h2o/benchmarks/groupby/q09.benchmark | Groupby benchmark query Q09 definition |
| benchmarks/sql_benchmarks/h2o/benchmarks/groupby/q10.benchmark | Groupby benchmark query Q10 definition |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| @@ -0,0 +1 @@ | |||
| CREATE EXTERNAL TABLE x STORED AS CSV LOCATION 'data/h2o/G1_1e7_1e7_100_0.csv'; | |||
| @@ -0,0 +1 @@ | |||
| CREATE EXTERNAL TABLE x STORED AS PARQUET LOCATION 'data/h2o/G1_1e7_1e7_100_0.parquet'; No newline at end of file | |||
| @@ -0,0 +1 @@ | |||
| CREATE EXTERNAL TABLE x STORED AS CSV LOCATION 'data/h2o/G1_1e8_1e8_100_0.csv'; | |||
| @@ -0,0 +1 @@ | |||
| CREATE EXTERNAL TABLE x STORED AS PARQUET LOCATION 'data/h2o/G1_1e8_1e8_100_0.parquet'; No newline at end of file | |||
| @@ -0,0 +1 @@ | |||
| CREATE EXTERNAL TABLE x STORED AS CSV LOCATION 'data/h2o/G1_1e9_1e9_100_0.csv'; No newline at end of file | |||
Comment on lines
+1
to
+5
| CREATE EXTERNAL TABLE x STORED AS CSV LOCATION 'data/h2o/J1_1e7_NA_0.csv'; | ||
|
|
||
| CREATE EXTERNAL TABLE small STORED AS CSV LOCATION 'data/h2o/J1_1e7_1e1_0.csv'; | ||
|
|
||
| CREATE EXTERNAL TABLE medium STORED AS CSV LOCATION 'data/h2o/J1_1e7_1e4_0.csv'; |
Comment on lines
+1
to
+5
| CREATE EXTERNAL TABLE x STORED AS PARQUET LOCATION 'data/h2o/J1_1e8_NA_0.parquet'; | ||
|
|
||
| CREATE EXTERNAL TABLE small STORED AS PARQUET LOCATION 'data/h2o/J1_1e8_1e2_0.parquet'; | ||
|
|
||
| CREATE EXTERNAL TABLE medium STORED AS PARQUET LOCATION 'data/h2o/J1_1e8_1e5_0.parquet'; |
Comment on lines
+1
to
+5
| CREATE EXTERNAL TABLE x STORED AS CSV LOCATION 'data/h2o/J1_1e8_NA_0.csv'; | ||
|
|
||
| CREATE EXTERNAL TABLE small STORED AS CSV LOCATION 'data/h2o/J1_1e8_1e2_0.csv'; | ||
|
|
||
| CREATE EXTERNAL TABLE medium STORED AS CSV LOCATION 'data/h2o/J1_1e8_1e5_0.csv'; |
Comment on lines
+1
to
+5
| CREATE EXTERNAL TABLE x STORED AS PARQUET LOCATION 'data/h2o/J1_1e9_NA_0.parquet'; | ||
|
|
||
| CREATE EXTERNAL TABLE small STORED AS PARQUET LOCATION 'data/h2o/J1_1e9_1e3_0.parquet'; | ||
|
|
||
| CREATE EXTERNAL TABLE medium STORED AS PARQUET LOCATION 'data/h2o/J1_1e9_1e6_0.parquet'; |
Contributor
There was a problem hiding this comment.
@Omega359 this does look like a real finding comparing to the existing TPCH scripts:
Comment on lines
+1
to
+5
| CREATE EXTERNAL TABLE x STORED AS CSV LOCATION 'data/h2o/J1_1e9_NA_0.csv'; | ||
|
|
||
| CREATE EXTERNAL TABLE small STORED AS CSV LOCATION 'data/h2o/J1_1e9_1e3_0.csv'; | ||
|
|
||
| CREATE EXTERNAL TABLE medium STORED AS CSV LOCATION 'data/h2o/J1_1e9_1e6_0.csv'; |
This was referenced Jun 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Part of #21706
Rationale for this change
Continue work on sql benchmark migration.
What changes are included in this PR?
h2o sql benchmark
Are these changes tested?
Yes
BENCH_NAME=h2o BENCH_SUBGROUP=groupby H2O_BENCH_SIZE=small H2O_FILE_TYPE=csv cargo bench --bench sqlBENCH_NAME=h2o BENCH_SUBGROUP=groupby H2O_BENCH_SIZE=small H2O_FILE_TYPE=parquet cargo bench --bench sqlBENCH_NAME=h2o BENCH_SUBGROUP=groupby H2O_BENCH_SIZE=medium H2O_FILE_TYPE=csv cargo bench --bench sqlBENCH_NAME=h2o BENCH_SUBGROUP=groupby H2O_BENCH_SIZE=medium H2O_FILE_TYPE=parquet cargo bench --bench sqlBENCH_NAME=h2o BENCH_SUBGROUP=join H2O_BENCH_SIZE=small H2O_FILE_TYPE=csv cargo bench --bench sqlBENCH_NAME=h2o BENCH_SUBGROUP=join H2O_BENCH_SIZE=small H2O_FILE_TYPE=parquet cargo bench --bench sqlBENCH_NAME=h2o BENCH_SUBGROUP=join H2O_BENCH_SIZE=medium H2O_FILE_TYPE=csv cargo bench --bench sqlBENCH_NAME=h2o BENCH_SUBGROUP=join H2O_BENCH_SIZE=medium H2O_FILE_TYPE=parquet cargo bench --bench sqlBENCH_NAME=h2o BENCH_SUBGROUP=window H2O_BENCH_SIZE=small H2O_FILE_TYPE=csv cargo bench --bench sqlBENCH_NAME=h2o BENCH_SUBGROUP=window H2O_BENCH_SIZE=small H2O_FILE_TYPE=parquet cargo bench --bench sqlBENCH_NAME=h2o BENCH_SUBGROUP=window H2O_BENCH_SIZE=medium H2O_FILE_TYPE=csv cargo bench --bench sqlBENCH_NAME=h2o BENCH_SUBGROUP=window H2O_BENCH_SIZE=medium H2O_FILE_TYPE=parquet cargo bench --bench sqlI was unable to run the following because of limited memory:
BENCH_NAME=h2o BENCH_SUBGROUP=groupby H2O_BENCH_SIZE=big H2O_FILE_TYPE=csv cargo bench --bench sqlBENCH_NAME=h2o BENCH_SUBGROUP=groupby H2O_BENCH_SIZE=big H2O_FILE_TYPE=parquet cargo bench --bench sqlBENCH_NAME=h2o BENCH_SUBGROUP=join H2O_BENCH_SIZE=big H2O_FILE_TYPE=csv cargo bench --bench sqlBENCH_NAME=h2o BENCH_SUBGROUP=join H2O_BENCH_SIZE=big H2O_FILE_TYPE=parquet cargo bench --bench sqlBENCH_NAME=h2o BENCH_SUBGROUP=window H2O_BENCH_SIZE=big H2O_FILE_TYPE=csv cargo bench --bench sqlBENCH_NAME=h2o BENCH_SUBGROUP=window H2O_BENCH_SIZE=big H2O_FILE_TYPE=parquet cargo bench --bench sqlAre there any user-facing changes?
No