-
-
Notifications
You must be signed in to change notification settings - Fork 69
add key-value file! #2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
19 commits
Select commit
Hold shift + click to select a range
8335a0b
add key-value file!
ruclz 156142a
Completed the key-value sort with uint64_t.
ruclz 473427c
Merge branch 'main' into main
ruclz a05f062
first
ruclz fad1753
first
ruclz 7821e01
run clang-format and add unit test for key-value sort
ruclz ae02b58
add unit test and run clang_format
ruclz 5363ba4
Rename Makefile to Makefile.bak2
ruclz cd836c7
Rename Makefile.bak to Makefile
ruclz 541f914
Delete Makefile.bak2
ruclz 8873ea1
Update meson.build
ruclz 7ea8230
Creat key-value sort hpp file.
ruclz 09ef43a
Creat key-value sort hpp file.
ruclz 17b534c
Creat key-value sort hpp file.
ruclz 4927fe2
Sorry, I forgot to merge the test_all.cpp in the last commit.
ruclz e73aa3f
Delete duplicated code and add benchmarking code of key value sort
ruclz b73ba45
Move the get_povit_64bit function and sort_zmm_64bit function to avx5…
ruclz c083424
Fix the authors name.
ruclz ecc94cf
Change the author of avx512-64bit-common.h file to Rahu.
ruclz File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
# Prerequisites | ||
*.d | ||
|
||
# Compiled Object files | ||
*.slo | ||
*.lo | ||
*.o | ||
*.obj | ||
|
||
# Precompiled Headers | ||
*.gch | ||
*.pch | ||
|
||
# Compiled Dynamic libraries | ||
*.so | ||
*.dylib | ||
*.dll | ||
|
||
# Fortran module files | ||
*.mod | ||
*.smod | ||
|
||
# Compiled Static libraries | ||
*.lai | ||
*.la | ||
*.a | ||
*.lib | ||
|
||
# Executables | ||
*.exe | ||
*.out | ||
*.app | ||
|
||
**/.vscode | ||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure if std::sort is a fair comparison. The way the key-values are arranged in memory for
std::vector<sorted_t<K, V>>
is very different when compared two distinct vectorsstd::vector<K>, std::vector<V>
. I would think this can alter performance significantly because you can no longer load continuous chunk of memory into ZMM registers. Does Oceanbase have two separate vectors for key and values (I assume they do cos that's the way you implemented the sort). How does Oceanbase currently sort them?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ObStoreRow is the base unit to do the sort in Oceanbase. The key and value are all ObObject class type in the ObStoreRow. There is no sperate vector for key and value. The ObStoreRow is passed to the std::sort in Oceanbase. std::sort first get the key type from ObObject and then do the sort work.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To enable the AVX512 sort, we need extract the key from the ObStoreRow and then pass the key and ObStoreRow as the value to avx512_sort(key, value) like API. That's way we need the key-value interface