Replace collision-chain eviction with global LRU in#97867
Open
crakjie wants to merge 1 commit intoClickHouse:masterfrom
Open
Replace collision-chain eviction with global LRU in#97867crakjie wants to merge 1 commit intoClickHouse:masterfrom
crakjie wants to merge 1 commit intoClickHouse:masterfrom
Conversation
|
|
Author
|
closes #62178 |
The previous implementation coupled storage position to key hash, limiting eviction to a 10-cell collision chain. Frequently accessed keys could be evicted while cold keys in other chains survived. ClickHouse#62178 Decouple storage from hashing by using a for key-to-slot lookup and a flat array for cell storage. Eviction now uses a clock algorithm (GCLOCK) that sweeps all cells globally, giving recently accessed entries a higher survival count. This provides O(1) amortized eviction with approximate LRU semantics.
crakjie
force-pushed
the
cache-dictionary-clock-lru
branch
from
656ed00 to
c9b7730
Compare
alexey-milovidov
added
the
can be tested
Contributor
|
Workflow [PR], commit [c9b7730] Summary: ❌
|
clickhouse-gh
bot
added
the
pr-improvement
Author
|
Can someone explain me what is AST fuzzer and BuzzHouse and how to deal with that? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The previous implementation coupled storage position to key hash, limiting
eviction to a 10-cell collision chain. Frequently accessed keys could be
evicted while cold keys in other chains survived.
Decouple storage from hashing by using a for key-to-slot lookup
and a flat array for cell storage. Eviction now uses a clock algorithm
(GCLOCK) that sweeps all cells globally, giving recently accessed entries
a higher survival count. This provides O(1) amortized eviction with
approximate LRU semantics.
Changelog category (leave one):
Changelog entry :
Replace collision-chain eviction by approximate LRU in CacheDictionary. Evicted entries should be more predictable.
Documentation entry for user-facing changes
When accessed entries in CacheDictionary are updated to keep track of the recent usages. When when a new entry need a room, the cache evict one of the least recently used key.
With the previous implementation a frequently accessed key can be evicted simply because its collision chain is full of other keys, while completely cold keys in other chains remain untouched. In other words, the cache does not provide LRU semantics — access frequency has little influence on which entries survive