-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Merge Iterator alternate implementation #1080
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
This commit introduces fast merge iterator which is faster than normal merge iterator if there are less than 18 tables.
…h-io/badger into ibrahim/mergeiterator-fast
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
✅ A review job has been created and sent to the PullRequest network.
@jarifibrahim you can click here to see the review status or cancel the code review job.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The change looks good to me. I added a few notes about best practices and code readability.
Reviewed with ❤️ by PullRequest
ashish-goswami
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 4 of 6 files at r1, 2 of 2 files at r2.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @jarifibrahim and @manishrjain)
manishrjain
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 4 of 6 files at r1, 2 of 2 files at r2.
Reviewable status: all files reviewed, 7 unresolved discussions (waiting on @jarifibrahim and @pullrequest[bot])
table/merge_iterator.go, line 47 at r2 (raw file):
func (n *node) setIterator(iter y.Iterator) { n.iter = iter n.merge, _ = iter.(*MergeIterator)
Please add a comment saying that even if the conversion fails, n.merge would be nil, which is OK and you handle that.
table/merge_iterator.go, line 106 at r2 (raw file):
if &mi.right == mi.small { mi.swapSmall() }
return
table/merge_iterator.go, line 108 at r2 (raw file):
} } if mi.reverse {
if cmp == 0 { .. ; return}
if cmp < 0 { .. if reverse then do, else do. }
else { .. if reverse then do, else do }
table/merge_iterator.go, line 194 at r2 (raw file):
mi.left.setIterator(iters[0]) mi.right.setIterator(iters[1]) mi.small = &mi.left
Add a comment that assign one randomly, because fix would fix this.
jarifibrahim
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Dismissed @pullrequest[bot] from 2 discussions.
Reviewable status: 5 of 6 files reviewed, 5 unresolved discussions (waiting on @ashish-goswami, @manishrjain, and @pullrequest[bot])
table/merge_iterator.go, line 55 at r1 (raw file):
Previously, pullrequest[bot] wrote…
By actively ignoring the second argument here, you're effectively silencing the error in type check. This might be ok given that it's an unexported function, but it might be better for the long term if the second argument was either assigned and checked or left out for a panic to happen.
Done.
table/merge_iterator.go, line 136 at r1 (raw file):
Previously, jarifibrahim (Ibrahim Jarif) wrote…
I feel the existing code is more readable.
Done.
table/merge_iterator.go, line 47 at r2 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
Please add a comment saying that even if the conversion fails, n.merge would be nil, which is OK and you handle that.
Done.
table/merge_iterator.go, line 106 at r2 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
return
Done.
table/merge_iterator.go, line 108 at r2 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
if cmp == 0 { .. ; return} if cmp < 0 { .. if reverse then do, else do. } else { .. if reverse then do, else do }
Done.
table/merge_iterator.go, line 194 at r2 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
Add a comment that assign one randomly, because fix would fix this.
Done.
This PR introduces an alternate implementation of #1045
Benchmarks
master-x.txt and new-x.txt are the result of
ReadMergedbenchmarks with x tables.Each table contains 10e6 entries.
This change is