OAK-12214 : segment cache notify L2 on L1 hits to keep frequency/recency accurate#2903
Conversation
|
Build is failing due to resource leakage in DocumentNodeStoreIT test. To be fixed by : #2905 |
| * L1-to-L2 propagation; downstream code that constructed ids with a {@code Runnable} must be updated. | ||
| */ | ||
| public SegmentId(@NotNull SegmentStore store, long msb, long lsb, @NotNull Runnable onAccess) { | ||
| public SegmentId(@NotNull SegmentStore store, long msb, long lsb, @NotNull Consumer<SegmentId> onAccess) { |
There was a problem hiding this comment.
Not backwards compatible public function.
| } | ||
|
|
||
| @Override | ||
| public void putSegment(@NotNull Segment segment) { |
There was a problem hiding this comment.
Also flagged, but not related to this PR: calling putSegment twice for the same id might over-counts stats.currentWeight.
jsedding
left a comment
jsedding
left a comment
There was a problem hiding this comment.
Looks good to me.
If the fix proves effective (which I have no doubt), I think we should remove the memoization (aka L1 cache) entirely.
| registerCloseable(store); | ||
|
|
||
| // OAK-12214: bug-fix toggle (default on) so L2 eviction policy sees L1 memoised hits | ||
| registerCloseable(cfg.getWhiteboard().register(FeatureToggle.class, |
There was a problem hiding this comment.
Maybe better would be to use utility org.apache.jackrabbit.oak.spi.toggle.Feature#newFeature
| public abstract void recordHit(@NotNull SegmentId id); | ||
|
|
||
| /** | ||
| * Feature toggle name to enable embedded verification for full GC mode for Mongo Document Store |
There was a problem hiding this comment.
this file is outdated. latest one doesn't have this wrong comment.
| * @param lsb least significant bits of this id | ||
| * @param onAccess callback invoked whenever the locally memoised segment is accessed | ||
| * ({@link #getSegment()}); receives {@code this} (e.g. to notify {@link SegmentCache}). | ||
| * <p><strong>API note (Oak 2.1, OAK-12214):</strong> this parameter type changed from |
There was a problem hiding this comment.
Oak 2.1? Should be 2.2.0, right?
…ency accurate L1 memoization in SegmentId serves most segment reads without touching L2. This means W-TinyLFU admission sketches and LRU recency lists never see hot segments, and the eviction policy treats them as cold — potentially evicting them under pressure. SegmentCache.recordHit(SegmentId) now calls cache.getIfPresent(id) to register the access with the L2 backing store. The behaviour is controlled by feature toggle FT_NOTIFY_L2_OAK-12214 (enabled by default). Also adds cleanUp() to flush pending Caffeine maintenance before reading eviction stats. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…s internal frequency updated such that we don't evict hot segments
|
5 participants
No description provided.