execution/commitment: cache nibblized keccak(addr) for storage keys#22185
Merged
Conversation
Add KeyToHexNibbleHashWithCache that memoizes the first 64 nibbles (nibblized keccak(addr)) keyed by [20]byte address. Integrated into Updates via addrNibblesCache field, used in TouchPlainKeyDirect for ModeParallel. For whale storage (one address, many slots), avoids redundant keccak computations and nibble expansion per slot. Benchmarks show ~43-45% latency reduction on both whale (1×1000) and spread (5×200, 100×10) scenarios with zero additional allocations. Co-authored-by: awskii <artem.tsskiy@gmail.com> Co-authored-by: shuo <shuo@erigon.dev>
taratorio
approved these changes
Jul 3, 2026
…allel Replace the per-address map cache with a single-entry last-address cache that reuses the nibblized keccak(addr) prefix across a run of storage keys sharing one address (whale storage). Route it through Updates.hashKey so it applies to serial (ModeDirect/ModeUpdate) and parallel commitment alike, gated to the nibblizing hasher whose key layout the reuse assumes. The map was never cleared in Reset() and its cold per-batch allocation plus rehash regressed scattered writes. The single-entry cache is fixed-size (nothing to leak), adds zero allocations, and matches or beats the map on grouped workloads; Reset now clears it. Nibblization microbench (1000 keys): whale 1x1000 -46%, spread 5x200 -45%, spread 100x10 -40%, scatter 1000x1 +6%.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR optimizes commitment key hashing by caching and reusing the nibblized keccak(addr) prefix across consecutive storage keys that share the same address, reducing repeated address-hash work for “whale” contracts with many dirty storage slots.
Changes:
- Added a single-entry last-address cache (
addrHashCache) and a cached hashing variant (keyToHexNibbleHashCached) to reuse the 64-nibble address prefix for storage keys. - Routed plain-key hashing through
Updates.hashKeyfor bothTouchPlainKeyandTouchPlainKeyDirect, and gated the optimization toKeyToHexNibbleHashviahasherReusesAddrPrefix. - Cleared the cache on
Updates.Reset()and added targeted unit tests + microbenchmarks.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
execution/commitment/keys_nibbles.go |
Introduces cached nibblization for storage keys (address-prefix reuse) and a shared nibble-expansion helper. |
execution/commitment/keys_nibbles_cache_test.go |
Adds correctness tests covering hit/miss behavior and benchmarks comparing cached vs uncached hashing. |
execution/commitment/commitment.go |
Adds Updates-level cache plumbing (hashKey, gating via hasher identity, reset behavior) so all modes benefit. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+70
to
+71
| nibblized := make([]byte, 128) | ||
| addr := [20]byte(key[:length.Addr]) |
mh0lt
added a commit
that referenced
this pull request
Jul 3, 2026
Bring the cache stack current with main (parallel/streaming commitment correctness fixes #22184/#22113, nibblized-keccak cache #22185, erigondb.toml commitment referencing #21452, trie io.Writer trace #21859, etc.). One conflict in commitment_context.go's trieContext: keep both this branch's probeSd/probeTx (adaptive trunk-pin probe) and main's traceW (io.Writer trace).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Commitment re-nibblizes keccak(addr) for every storage slot, so a whale contract (one address, thousands of dirty slots) pays the same address hash thousands of times per commitment.
Reuse the nibblized keccak(addr) prefix across a run of storage keys that share an address, via a single-entry last-address cache routed through
Updates.hashKey. Applies to serial (ModeDirect/ModeUpdate) and parallel commitment, gated to the nibblizing hasher whose key layout the reuse assumes. keccak(addr) is immutable, so a hit is always correct and a miss simply recomputes.Changes
addrHashCache+keyToHexNibbleHashCached: one fixed-size cache entry; a hit copies the 64-nibble prefix, a miss recomputes and repopulates.Updates.hashKeywraps every plain-key hash site in bothTouchPlainKeyandTouchPlainKeyDirect, so all three modes benefit; enabled only when the hasher isKeyToHexNibbleHash.Reset()clears the cache.Nibblization microbench (1000 keys/op): whale 1x1000 -46%, spread 5x200 -45%, spread 100x10 -40%, scatter 1000x1 +6%. Zero added allocations.
End-to-end commitment over a 200K-slot single-address whale (touch + fold, benchstat n=6): ModeDirect -7.5%, ModeParallel -15.3% (p=0.002); allocations unchanged. The gain scales with how whale-heavy the storage writes are — a normal mixed block sees less.