-
Notifications
You must be signed in to change notification settings - Fork 722
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore(query): refine hash join bitmap and support fast path for datablock take #13213
Conversation
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
Docker Image for PR
|
Dev / linux / sqllogic_cluster_tpch:
|
There is a same issue here in another PR: |
This issue has been addressed by #13212. |
…lock take (datafuselabs#13213) * add set_true_validity * fast path for take --------- Co-authored-by: sundyli <543950155@qq.com>
I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/
Summary
After jorgecarleitao/arrow2#1574, If a
Bitmap
is all true or all false,Bitmap.slice()
has almost no overhead, we can benefit a lot from it, for example, we can keep an all true bitmaptrue_validity
in hash joinProbeState
, when we need to wrap aColumn
to aNullableColumn
, we can just slice thetrue_validity
to avoid creating a new bitmap, it can also save a lot of memory(all bitmaps is refer to thetrue_validity
), besides, we can introduce a fast path intake.rs
andtake_chunks.rs
to avoid iterating column to generate a new bitmap when a bitmap is all true or all false.This change is![Reviewable](https://camo.githubusercontent.com/23b05f5fb48215c989e92cc44cf6512512d083132bd3daf689867c8d9d386888/68747470733a2f2f72657669657761626c652e696f2f7265766965775f627574746f6e2e737667)