fix: skip post-idle compaction when cache warmer kept the cache warm#258
Merged
Conversation
onIdleResume() unconditionally set postIdleCompact=true, forcing Layer 1+ with a tight raw budget on resume. When the cache warmer (automated or /keep) had recently refreshed the upstream cache, this produced different bytes than what the warmer replayed — busting the cache and wasting the warming cost. Add a skipCompact parameter to onIdleResume() (default false). The gateway pipeline checks whether the cache was warmed within the current TTL window via a new isCacheWarm() helper and passes the result through. All other idle-resume housekeeping (cache clearing, cameOutOfIdle, LTM refresh) still runs — only the compaction is skipped. Also fix vectorSearch test isolation: beforeEach was scoped to one project but vectorSearch() queries globally, causing cross-suite leaks.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
onIdleResume()unconditionally setpostIdleCompact=true, forcing Layer 1+ with a tight raw budget. When the cache warmer (automated or/keep) had recently refreshed the upstream cache, the compacted context produced different bytes than the warmed prefix — busting the cache and wasting the warming cost.skipCompactparameter toonIdleResume()(defaultfalse, backwards-compatible). The gateway pipeline checks whether the cache was warmed within the current TTL window via a newisCacheWarm()helper and skips compaction when the cache is warm. All other idle-resume housekeeping (cache clearing,cameOutOfIdle, LTM refresh) still runs.vectorSearchtest isolation:beforeEachwas scoped to one project butvectorSearch()queries globally, causing 3 pre-existing test failures from cross-suite leaks.Changes
packages/core/src/gradient.tsskipCompactparam toonIdleResume(), conditionalpostIdleCompactpackages/gateway/src/pipeline.tsisCacheWarm()helper, importresolveWarmingProfile, passskipCompactat call sitepackages/core/test/gradient.test.tsskipCompactbehaviorpackages/core/test/embedding.test.tsvectorSearchdescribe cleanup to be globalTest plan
bun test— 1187 pass, 0 fail (was 1184 pass, 3 fail before the embedding fix)bun run typecheck— clean across all 4 packages