perf: use bounded LRU for local cache#9
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replace DialCache's hand-written per-use-case Map/FIFO local cache with the mature
lru-cachev11 implementation.The local layer now provides:
DialCacheinstanceundefinedsupport through the existing entry wrapperMotivation
The previous implementation created one Map per use case and applied
localMaxSizeindependently to each Map. As the number of registered use cases grew, total local entries were not bounded by the configured limit. Eviction followed insertion order, so frequently read entries did not gain retention priority.Using a well-tested local-cache library keeps TTL and eviction mechanics behind the existing local-cache abstraction while enforcing the configured process-local bound.
Design
LocalCacheowns oneLRUCache<string, LocalEntry<unknown>>keyed by the existing full DialCache URN. The URN already includes the use case and all key dimensions, so a single cache preserves key isolation while enforcing one aggregate limit.The cache uses sparse
maxSizeaccounting with every entry weighted as one, avoiding allocation proportional to a configured limit. TTL remains configured per entry:setttlResolution: 0checks expiry on every accessperformance.now()clock keeps TTL timing monotonic and avoids the library's zero-timestamp sentinellru-cache's strict expiry comparison, preserving DialCache's historical exact-boundary behaviorValues remain wrapped in
LocalEntry, allowing an application result ofundefinedto remain distinguishable from a missing cache entry.Consumer impact
The exported symbols, method signatures, and public type shapes are unchanged.
localMaxSizealready existed onDialCacheConfig; this PR documents and tightens its contract:DialCacheinstance0disables local storageRangeErrorThe package engine changes from Node
>=18.17to20 || >=22, matchinglru-cachev11. Redis behavior, request coalescing, runtime ramping, metrics APIs, invalidation semantics, cache-key construction, and local TTL units remain unchanged.Validation
pnpm check— typecheck, 99 unit tests with coverage thresholds, build, and packaged ESM/CJS/TypeScript consumerpnpm test:integration— 32 Docker-backed Redis/Valkey testspnpm audit --prod— no known vulnerabilitiesgit diff --check