fix: prevent splitSegments infinite recursion and add global background work throttling#355
Merged
Merged
Conversation
…nd work throttling Two fixes for issues triggered by importing 5 projects simultaneously: 1. splitSegments() infinite recursion (RangeError: Maximum call stack size exceeded) — a single message exceeding maxSegmentTokens (16384) caused unbounded recursion because findSplitIndex() returned messages.length for a 1-element array, producing left=same message, right=empty. Added base case: if (messages.length <= 1) return. 2. Background 429 Too Many Requests — no global concurrency limit on background LLM calls meant N idle sessions could fire N×4 simultaneous requests. Added a global p-limit(2) background limiter with a circuit breaker that trips on 429, pausing all background work for the Retry-After duration. Wired into idle scheduler, pipeline incremental distillation, and in-flight curation. Urgent distillation (client waiting) is intentionally excluded.
…time, add tests, fix docs - Re-check isBackgroundPaused() when task reaches front of p-limit queue, not just at submission time (prevents queued tasks from executing after a 429 trips the breaker while they wait) - Fix module doc comment: auto-import is NOT wrapped by runBackground (it runs sequentially per-process; circuit breaker still protects via llm-adapter.ts) - Add clearQueue() behavior comment in resetBackgroundLimiter() - Add test for error propagation through runBackground - Add test for circuit breaker tripping while tasks are queued - Add .d.ts/.d.ts.map build artifacts to .gitignore
…tion The resetWorkerModelState test used absolute callCount assertions that broke when another test file's async resetPipelineState() cleared the model cache mid-test, causing unexpected re-fetches against the mock. Fix: reset cache + counter before assertions, use relative deltas.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes two distinct background errors reported by a user who imported old chats from 5 projects simultaneously:
Recursive crash during idle distillation —
RangeError: Maximum call stack size exceededinsplitSegments()when a single temporal message exceedsmaxSegmentTokens(16384 tokens / ~49KB content). Added a base case to prevent infinite recursion on indivisible oversized messages.Upstream 429 Too Many Requests — background LLM work (distillation, curation) had no global concurrency limit, allowing N idle sessions to fire N×4 simultaneous requests. Added a global
p-limit(2)background limiter with a circuit breaker that trips on 429 and pauses all background work for theRetry-Afterduration.Changes
Bug 1:
splitSegments()infinite recursionpackages/core/src/distillation.ts: Addedif (messages.length <= 1) return [messages]base case after thetotalTokens <= maxTokensguard (1 line)packages/core/test/distillation.test.ts: Added 2 test cases — single oversized message and oversized message among normal messagesBug 2: Background 429 rate limiting
packages/gateway/src/background-limiter.ts(new): Globalp-limit(2)concurrency limiter + circuit breaker modulepackages/gateway/src/llm-adapter.ts: Trips circuit breaker on non-urgent 429 responsespackages/gateway/src/idle.ts: WrapsdoIdleWork()throughrunBackground()— at most 2 sessions do idle work simultaneouslypackages/gateway/src/pipeline.ts: Wraps incremental distillation and in-flight curation throughrunBackground(). Urgent distillation (client waiting, gradient in overflow) is intentionally not throttled. Added cleanup inresetPipelineState().packages/gateway/test/background-limiter.test.ts(new): 7 tests for concurrency limiting and circuit breaker behaviorpackages/gateway/package.json: Addedp-limitdependencyDesign decisions
Retry-Afterduration (default 60s). Only extends, never shortens an active pause.p-limit+ circuit breaker provides dampening. Each of 5lore runprocesses independently backs off on 429.