Skip to content

test: scale remaining sort-merge join (SMJ) benchmark queries#21200

Merged
mbutrovich merged 2 commits intoapache:mainfrom
mbutrovich:smj-bench
Mar 27, 2026
Merged

test: scale remaining sort-merge join (SMJ) benchmark queries#21200
mbutrovich merged 2 commits intoapache:mainfrom
mbutrovich:smj-bench

Conversation

@mbutrovich
Copy link
Copy Markdown
Contributor

@mbutrovich mbutrovich commented Mar 27, 2026

Which issue does this PR close?

  • Closes #.

Rationale for this change

Our SMJ benchmark queries finish too quickly to demonstrate improvements that aren't massive. For example, I am working on an optimization that introduces DynComparator (part of #20910) and it's about a 10% improvement, but only when you actually make the queries run long enough. The new queries for #21184 are scaled enough to see improvements, but we need to scale the older queries.

I am also continuing to see SMJ issues with Comet when running joins with billions (sometimes trillions) of rows. We can't do that for microbenchmarks, but we can at least start hitting millions of rows to look at more than a handful of batches.

What changes are included in this PR?

Bring our SMJ queries into alignment with some of the newer ones (Q21-23) to demonstrate further performance wins.

Are these changes tested?

I ran the benchmark. On my M3 Max, here's how long it takes:

Query Join Type Rows Keys Filter Median (ms)
Q1 INNER 1M×1M 1:1 16.3
Q2 INNER 1M×10M 1:10 117.4
Q3 INNER 1M×1M 1:100 74.2
Q4 INNER 1M×10M 1:10 1% 17.1
Q5 INNER 1M×1M 1:100 10% 18.4
Q6 LEFT 1M×10M 1:10 129.3
Q7 LEFT 1M×10M 1:10 50% 150.2
Q8 FULL 1M×1M 1:10 16.6
Q9 FULL 1M×10M 1:10 10% 153.5
Q10 LEFT SEMI 1M×10M 1:10 53.1
Q11 LEFT SEMI 1M×10M 1:10 1% 15.5
Q12 LEFT SEMI 1M×10M 1:10 50% 65.0
Q13 LEFT SEMI 1M×10M 1:10 90% 105.7
Q14 LEFT ANTI 1M×10M 1:10 54.3
Q15 LEFT ANTI 1M×10M 1:10 partial 51.5
Q16 LEFT ANTI 1M×1M 1:1 10.3
Q17 INNER 1M×50M 1:50 5% 75.9
Q18 LEFT SEMI 1M×50M 1:50 2% 50.2
Q19 LEFT ANTI 1M×50M 1:50 partial 336.4
Q20 INNER 1M×10M 1:100 GROUP BY 763.7
Q21 INNER 10M×10M 1:1 50% 186.1
Q22 LEFT 10M×10M 1:1 50% 10,193.8
Q23 FULL 10M×10M 1:1 50% 10,194.7

Note that Q22 and Q23 will be about 20x faster when #21184 merges, so taking 10 seconds to run is just a short-term issue.

Are there any user-facing changes?

No.

Copy link
Copy Markdown
Member

@andygrove andygrove left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @mbutrovich

@mbutrovich mbutrovich added this pull request to the merge queue Mar 27, 2026
Merged via the queue into apache:main with commit 37c1b75 Mar 27, 2026
31 checks passed
@mbutrovich mbutrovich deleted the smj-bench branch March 27, 2026 15:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants