Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speedup hashing of interned objects. #1053

Merged
merged 1 commit into from
Aug 18, 2021
Merged

Conversation

ryzhyk
Copy link
Contributor

@ryzhyk ryzhyk commented Aug 18, 2021

We recently changed the implementation of Hash for interned objects to
hash the value of the object itself instead of hashing the pointer,
which is non-deterministic in some scenarios. So obviously hashing
became much more expensive for interned types. Of course the biggest
user of hashing is the internment library itself, as it hashes objects
in order to intern them. So, interning an object whose fields are also
interned values became many times more expensive. This for example
affected the performance of OVN in some scenarios.

To solve this, we change the way interned objects are hashed once again.
We already store a hash with each interned object, used to speedup
comparison operators by comparing by hash instead of by value.
Similarly, we now hash the interned object by hashing the stored hash
instead of hashing the object itself. We also increase the stored hash
size to 64 bit.

Signed-off-by: Leonid Ryzhyk lryzhyk@vmware.com

We recently changed the implementation of `Hash` for interned objects to
hash the value of the object itself instead of hashing the pointer,
which is non-deterministic in some scenarios.  So obviously hashing
became much more expensive for interned types.  Of course the biggest
user of hashing is the internment library itself, as it hashes objects
in order to intern them.  So, interning an object whose fields are also
interned values became many times more expensive.  This for example
affected the performance of OVN in some scenarios.

To solve this, we change the way interned objects are hashed once again.
We already store a hash with each interned object, used to speedup
comparison operators by comparing by hash instead of by value.
Similarly, we now hash the interned object by hashing the stored hash
instead of hashing the object itself.  We also increase the stored hash
size to 64 bit.

Signed-off-by: Leonid Ryzhyk <lryzhyk@vmware.com>
@ryzhyk ryzhyk requested a review from mihaibudiu August 18, 2021 18:28
@mihaibudiu
Copy link

Does this restore the OVN perf to its prior status?

@blp
Copy link
Contributor

blp commented Aug 18, 2021

Does this restore the OVN perf to its prior status?

Yes, my measurements show that:

                       GB    elapsed time    CPU s
0.38                 16.8        5:13        277.5
0.42.1               16.5        5:11        271.2
0.42.1 + interning   12.9        4:44        248.0
0.43                 16.5       10:05        564.2
0.45.1               16.6        9:52        550.6
0.45.1 + interning   12.8       11:47        665.1
0.45.1 + 64b intern  12.9       11:21        640.3
0.45.1 + PR#1053     12.8        5:08        267.4

@ryzhyk ryzhyk merged commit 2ce3ba7 into vmware:master Aug 18, 2021
@ryzhyk ryzhyk deleted the 64bit_hash branch August 18, 2021 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants