optimize hash_tree_root using hashtree #35
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mratsim
reviewed
Aug 16, 2022
ssz_serialization/merkleization.nim
Outdated
|
|
||
| func best_sha256_implementation(): Hasher {.importc.} | ||
|
|
||
| let digest64* = block: |
There was a problem hiding this comment.
digest64 is a really bad name. Maybe digest2x or digest_2way?
There was a problem hiding this comment.
well .. it's 64 bytes which is the significant requirement here .. ie unlike the other digest functions, it only works for blocks of that length.
There was a problem hiding this comment.
just call it digestBlockSize :)
ssz_serialization/merkleization.nim
Outdated
| h.update a | ||
| h.update b | ||
| h.update c | ||
| func digest(a, b, c: openArray[byte]): Digest {.noinit.} = |
There was a problem hiding this comment.
when joining a 32-byte data chunk with a <32-byte chunk - c is the zero padding mandated by ssz
arnetheduck
force-pushed
the
hashtree
branch
3 times, most recently
from
6397a6a
to
84985e0
Compare
arnetheduck
force-pushed
the
hashtree
branch
3 times, most recently
from
8685304
to
9aeaa38
Compare
arnetheduck
force-pushed
the
hashtree
branch
from
9aeaa38
to
fd5f3ee
Compare
20% from digest64 10% from working around nim-lang/Nim#20232 brings `hash_tree_root(BeaconState)` down ~480ms to ~320ms from the padding optimization alone - further optimization possible by parallelizing the hashing of each tree level, but that requires more surgery.
arnetheduck
changed the title
arnetheduck
added a commit
to status-im/nimbus-eth2
that referenced
this pull request
May 2, 2024
status-im/nim-ssz-serialization#35 brings in https://github.com/prysmaticlabs/hashtree as a supported backend for SHA256, giving a nice little performance boost to all hash_tree_root calls on supported platforms / compilers. Expected gains are on the order of 30% which in the case of a replayed state nets us 0.2-0.3s improvement. More about this design here: https://hackmd.io/@potuz/BJyrx9DOF - kudos to @potuz for this excellent library!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
brings
hash_tree_root(BeaconState)down ~800ms to ~660ms - further optimization is possible by hashing multiple blocks in a single call, but this requires more significant surgery.