Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DictionaryBlock for build page in LookupJoinPageBuilder #16475

Open
arunthirupathi opened this issue Jul 23, 2021 · 0 comments
Open

DictionaryBlock for build page in LookupJoinPageBuilder #16475

arunthirupathi opened this issue Jul 23, 2021 · 0 comments

Comments

@arunthirupathi
Copy link

LookupJoin uses block builder for the build side and does not use the dictionary block. https://github.com/prestodb/presto/blob/master/presto-main/src/main/java/com/facebook/presto/operator/LookupJoinPageBuilder.java#L33

Is there some caveats to fixing this issue or is this just a matter of implementation ? If it is just code change, I will try to fix this problem.

Assume the build side has 5 rows. The probe side has million rows. The LookupPageBuilder will create a new MapBlock builder and create a million element map block builder.

MapBlockBuilder after #16346 additional hash tables lazily (MapBlock computed them lazily for a long time).

Due to the cardinality increase 1) It will hold on to many copies of the data, including hash tables. 2) If the hash tables are computed lazily, it will waste CPU in repeated computation of the hash tables.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant