Skip to content

Conversation

@Dousir9
Copy link
Contributor

@Dousir9 Dousir9 commented Apr 10, 2023

I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/

Summary

Summary about this PR

Optimize the probe of hash join:

  • Reuse the probe_indexes in the ProbeState
  • Merge the same probe indexes: [1, 1, 1, 2, 2, 2, 2, 3] => [(1, 3), (2, 4), (3, 1)].
  • Reduce deref operations and using the doubling method to copy memory.

benchmark TPCH 30G:

Query main optimization ratio(old/new)
Q1 2.731681 2.721138 101%
Q2 2.08444 1.732778 124%
Q3 2.044172 1.665409 127%
Q4 5.111453 5.687737 97%
Q5 5.773698 5.058759 120%
Q6 0.091207 0.09686 91%
Q7 10.751493 10.53939 103%
Q8 9.866906 8.289139 119%
Q9 17.446036 15.923825 115%
Q10 2.757232 2.415082 118%
Q11 0.705935 0.535113 141%
Q12 1.149247 1.063082 111%
Q13 10.697853 10.668142 100%
Q14 1.154865 1.169206 99%
Q15 0.273866 0.273614 100%
Q16 0.645702 0.450822 148%
Q17 4.301655 3.813419 113%
Q18 17.302344 15.921257 108%
Q19 1.641281 1.366417 122%
Q20 4.306966 4.38217 98%
Q21 59.795661 57.065519 104%
Q22 6.076152 5.993872 102%

@vercel
Copy link

vercel bot commented Apr 10, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
databend ⬜️ Ignored (Inspect) Visit Preview Apr 12, 2023 6:30pm

@mergify mergify bot added the pr-feature this PR introduces a new feature to the codebase label Apr 10, 2023
@Dousir9 Dousir9 changed the title feat(query): explore a new hashtable for hashjoin feat(query): optimize the probe of hash join Apr 11, 2023
@Dousir9 Dousir9 marked this pull request as ready for review April 11, 2023 13:06
@Dousir9 Dousir9 marked this pull request as draft April 11, 2023 13:30
@Dousir9 Dousir9 marked this pull request as ready for review April 11, 2023 14:22
@andylokandy andylokandy requested a review from leiysky April 11, 2023 15:27
Dousir9 and others added 2 commits April 12, 2023 14:57
…n/probe_join/inner_join.rs

Co-authored-by: xudong.w <wxd963996380@gmail.com>
@sundy-li sundy-li merged commit 72092e6 into databendlabs:main Apr 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-feature this PR introduces a new feature to the codebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants