-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Further parallelize index creation #5812
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I made some nitpicky observations, but it looks good :)
One more thought I had, but I'm not sure of the added benefit over the existing solution:
Maybe we want to create a NodeChildIterator instead of the NextPosAndByte functions?
That would encapsulate the position, so you don't have to keep track of this position yourself and keep giving it back to the function.
I have a list of possible code refactoring ideas/improvements for the ART. I will prolly put them in an issue these days, and I think that this should go there, too. Instead of adding it to this PR. This iterator would also make sense for similar functions iterating node children. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! Just one minor comment - perhaps you can fix it in a future PR:
Before this PR, we sorted the index key columns and executed key column expressions in the
PhysicalCreateIndex
operator. However, this limited the sorting performance and decreased the index creation's performance in general.This PR moves the sorting and the expression execution out of the
PhyscialCreateIndex
operator and adds them as extra operators to the CREATE INDEX pipeline. There are also some other minor code improvements.