This repository has been archived by the owner on Oct 21, 2024. It is now read-only.
Improve 128-bit hash performance #6
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit makes some improvements to the 128-bit hash performance, mainly to the Hash128Writer but also to the standalone Hash128 function Prior to this, the writer would copy all incoming data in to a 16-byte buffer and process it block by block, which incurred a lot of memmmove overhead. This has been changed to unsafe read the incoming data in 16-byte blocks and only copying the remainder to a tail buffer. This ends up being much more efficient for larger blocks of input as well as inputs that are sized in to 16-byte blocks. The other change here is to use `bits.RotateLeft64` where appropriate instead of hand-rolling the code for this. Using `bits.RotateLeft64` leverages assembly intrinsics (ROLQ) where available which speeds this up The changes to HashWriter128 are backwards compatible but new methods have been added (AddString/AddBytes) that allow callers not to have to bother with an error result. The Sum128() method was added and returns a Hash128Value, similar to the standalone hash function Some tests & benchmarks from the twmb/murmur3 project were borrowed to test edge cases around the handling of tail bytes in the writer.
brycekahle
reviewed
May 28, 2020
jmoiron
reviewed
Aug 4, 2020
| @@ -199,160 +186,190 @@ func Hash128x64(key []byte) []byte { | |||
| return Hash128(key).Bytes() | |||
| } | |||
|
|
|||
| // WriteHash128x64 is a | |||
| // WriteHash128x64 creates a hash of key and writes it to ret | |||
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit makes some improvements to the 128-bit hash performance, mainly to the Hash128Writer but also to the standalone Hash128 function
Prior to this, the writer would copy all incoming data in to a 16-byte buffer and process it block by block, which incurred a lot of memmove overhead. This has been changed to unsafe read the incoming data in 16-byte blocks and only copying the remainder to a tail buffer. This ends up being much more efficient for larger blocks of input as well as inputs that are sized in to 16-byte blocks.
The other change here is to use
bits.RotateLeft64where appropriate instead of hand-rolling the code for this. Usingbits.RotateLeft64leverages assembly intrinsics (ROLQ) where available which speeds this upThe changes to HashWriter128 are backwards compatible but new methods have been added (AddString/AddBytes) that allow callers not to have to bother with an error result. The Sum128() method was added and returns a Hash128Value, similar to the standalone hash function
Some tests & benchmarks from the twmb/murmur3 project were borrowed to test edge cases around the handling of tail bytes in the writer.
Below are benchmarks for the standalone
Hash128function:And benchmarks for
HashWriter128: