Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Rewrite ConcurrentLruCache implementation
Prior to this commit, the `ConcurrentLruCache` implementation would not perform well under certain conditions. As long as the cache capacity was not reached, the cache would avoid maintaining an eviction queue (reordering entries depending with least/most recently read). When the cache capacity was reached, the LRU queue was updated for each read/write operation. This decreased performance significantly under contention when the capacity was reached. This commit completely rewrites the internals of `ConcurrentLruCache`. `ConcurrentLruCache` is now a specialized version of the `ConcurrentLinkedHashMap` [1]. This change focuses on buferring read and write operations, only processing them at certain times to avoid contention. When a cached entry is read, a read operation is queued and buffered operations are drained if the buffer reached a fixed limit. When a new cache entry is added or removed, a write operation is queued and triggers a drain attempt. When the capacity is outgrown, the cache polls items from the eviction queue, which maintains elements with the least recently used ones first. Entries are removed until the capacity is under control. The behavior described here and the buffer sizes are optimized with the number of available processors in mind. Work is localized as much as possible on a per-thread basis to avoid contention on the eviction queue. The new implementation has been tested with the JMH benchmark provided here, comparing the former `COncurrentLruCache`, the new implementation as well as the `ConcurrentLinkedHashMap` [1]. When testing with a cache reaching capacity, under contention, with a 10% cache miss, we're seeing a 40x improvement compared to the previous implementation and performance on par with the reference. See [2] for how to replicate the benchmark. [1] https://github.com/ben-manes/concurrentlinkedhashmap [2] https://github.com/spring-projects/spring-framework/wiki/Micro-Benchmarks Closes gh-26320
- Loading branch information