A Proper lookup table that uses HashTable's API #4536
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
amosbird
force-pushed
the
fixedhashtable
branch
2 times, most recently
from
910f579
to
c8e54a4
Compare
dbms/src/Common/HashTable/HashMap.h
Outdated
| const value_type & getValue() const { return value; } | ||
|
|
||
| static Key & getKey(value_type & value) { return value.first; } | ||
| static const Key & getKey(value_type & value) { return value.first; } |
There was a problem hiding this comment.
No need for this method?
| #include <Common/HashTable/HashMap.h> | ||
| #include <Common/HashTable/HashSet.h> | ||
|
|
||
|
|
There was a problem hiding this comment.
Missing comments with brief description.
| }; | ||
|
|
||
|
|
||
| template <typename Key, typename TMapped, typename TState = HashTableNoState> |
There was a problem hiding this comment.
Why don't make a separate header file for map?
| friend class TwoLevelHashTable; | ||
|
|
||
| template <typename, typename, typename, typename, typename, typename> | ||
| friend class HashTableWithSSO; |
There was a problem hiding this comment.
But there is no HashTableWithSSO yet.
| /// Set the mapped value, if any (for HashMap), to the corresponding `value`. | ||
| void setMapped(const value_type & /*value*/) {} | ||
|
|
||
| struct CellExt |
|
Ok, but I don't understand what is the source of performance improvement. In this PR we still store keys in lookup table and the FixedHashSet/Map doesn't look much different to the HashSet/HashMap. |
|
It's Ok as a first step, if the motivation will be mentioned in comments. |
amosbird
force-pushed
the
fixedhashtable
branch
2 times, most recently
from
e25e295
to
c7e5602
Compare
This is the first step of allowing heterogeneous cells in hash tables. performance test results are ``` 1. HashMap<UInt16, UInt8, TrivialHash, HashTableFixedGrower<16>>; 2. NewLookupMap<UInt16, UInt8> ResolutionWidth 30000 1 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................223550276.46 ResolutionWidth 30000 2 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................248772721.24 Best: 2 - 24877272124 ResolutionWidth 100000 1 ..........................................................................................................................................................................................................................................................238498413.99 ResolutionWidth 100000 2 ..........................................................................................................................................................................................................................................................261808889.98 Best: 2 - 26180888998 ResolutionWidth 300000 1 ...................................................................................239307348.81 ResolutionWidth 300000 2 ...................................................................................257592761.30 Best: 2 - 25759276130 ResolutionWidth 1000000 1 .........................240144759.26 ResolutionWidth 1000000 2 .........................257093531.91 Best: 2 - 25709353191 ResolutionWidth 5000000 1 .....241573260.35 ResolutionWidth 5000000 2 .....259314162.79 Best: 2 - 25931416279 ResolutionDepth 30000 1 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................217108119.84 ResolutionDepth 30000 2 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................249459504.41 Best: 2 - 24945950441 ResolutionDepth 100000 1 ..........................................................................................................................................................................................................................................................229065162.17 ResolutionDepth 100000 2 ..........................................................................................................................................................................................................................................................253769105.64 Best: 2 - 25376910564 ResolutionDepth 300000 1 ...................................................................................233079225.18 ResolutionDepth 300000 2 ...................................................................................256316273.78 Best: 2 - 25631627378 ResolutionDepth 1000000 1 .........................234184633.51 ResolutionDepth 1000000 2 .........................261100491.57 Best: 2 - 26110049157 ResolutionDepth 5000000 1 .....233118795.66 ResolutionDepth 5000000 2 .....252436160.41 Best: 2 - 25243616041 ```
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en
This is the first step of allowing heterogeneous cells in hash tables.
performance test results are