Fix: Optimize token usage for scratchpad tool - #245
Merged
ea-rus merged 8 commits intoJul 15, 2026
Conversation
ea-rus
marked this pull request as ready for review
July 13, 2026 09:07
ZoranPandovski
approved these changes
Jul 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes ENG-626 —
recall_scratchpad_wisdom()had no size cap and injected content unconditionally into thescratchpadtool description, which is resent on every LLM call. On a heavy real user this grew to ~103K chars, becoming the largest single component of the per-call payload and burying the tool's own safety-critical timeout/progress()contract.Also fixes two related bugs found while investigating: a shared-singleton mutation bug that leaked memory content across sessions, and a duplication issue where the same procedural knowledge was billed twice per call (system prompt + tool description).
Changes
Static contract ordering (
tool_defs.py)IMPORTANTtimeout/progress()block right after theActionslist (whereexec/cellare defined) and before the capabilities paragraphs — previously it sat right before the unbounded memory injection, so growth pushed it further from view on every turn.Token budget for
recall_scratchpad_wisdom()(hippocampus.py)get_rules()(cleanEngramobjects) instead of scrapingrecall_rules()'s raw markdown, which was leaking<!-- confidence:... ts:... -->metadata comments verbatim into the injected text.high→medium/unset →low) then recency within a tier, and trimmed totoken_budget(default 2000/scope, ~4 chars/token) via a shared_filter_by_token_budget()helper (also used byget_lessons()).whenrule regardless of topic. Added the same"scratchpad" in text/topicfilter already used for lessons (_is_scratchpad_related()).Fixed
SCRATCHPAD_TOOLsingleton mutation (session.py)_build_core_tools()aliased the module-levelSCRATCHPAD_TOOLsingleton and mutated.descriptionin place (scratchpad_tool = SCRATCHPAD_TOOL). In a process serving multiple sessions/users, each session's injected wisdom compounded on top of the previous one's, forever, until process restart — a real risk of leaking one user's memory into another user's tool description within the same worker.scratchpad_tool = replace(SCRATCHPAD_TOOL)(shallow copy viadataclasses.replace)..descriptionis untouched and no cross-session leakage occurs.Removed duplication between system prompt and tool description (
cortex.py,hippocampus.py)build_memory_context()(system prompt) andget_scratchpad_context()(tool description) were both pulling from the sameget_rules()/get_lessons()data with no coordination — any scratchpad-relatedwhenrule or lesson was billed in both places on every call.exclude_scratchpad/exclude_scratchpad_whenparameters toget_lessons()/get_rules()(defaultFalse, sorecall_scratchpad_wisdom()and the existingrecall_rules()caller insession.py— an ACC dedup check that needs the full rule set — are unaffected).recall_lessons()now always excludes scratchpad-related entries;build_memory_context()callsget_rules(exclude_scratchpad_when=True)/recall_lessons(token_budget=1000)directly, keeping the LLM-based rule-relevance filtering and episodic-memory read-logging paths intact.Testing
tests/test_hippocampus.py,tests/test_cortex.py,tests/test_chat_scratchpad.py— new coverage for: budget truncation, confidence/recency ordering, the scratchpad-relatedness filter on rules, singleton-mutation regression, and system-prompt/tool-description exclusion in both directions.test_large_input_no_crash, reproduces identically onmain— unrelated to this change, confirmed viagit stash).Not in this PR
recall_skill(pull-based) instead of push-injecting into the tool description — larger change, left as follow-up.Fixes: https://linear.app/mindsdb/issue/ENG-626/memory-consolidation-inflates-scratchpad-tool-description-without