fix(memory): separate context from extraction target in extractCandidates#8
Merged
Merged
Conversation
…ates Previously, when ContextMessages was non-empty, it completely replaced Messages for extraction. This caused the LLM to extract memories from the entire context window — including old messages that were likely already processed — leading to duplicate extractions and wasted tokens. Now ContextMessages is passed as a system message (reference-only context with a "Do NOT extract" instruction), while Messages remains the sole extraction target in the user message. This leverages the LLM's native role-based separation, which is a much stronger signal than in-text markers. When ContextMessages is nil, behavior is entirely unchanged (single user message, backward compatible). Context is independently truncated to 30% of MaxConversationRunes to prevent it from crowding out the extraction target. Made-with: Cursor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
extractCandidatesreplacedMessageswithContextMessageswhen the latter was non-empty, causing the LLM to extract memories from the entire context window (including already-processed old messages), leading to duplicate extractions and wasted tokens.ContextMessagesis delivered as a system message with a "Do NOT extract" instruction, whileMessagesremains the sole extraction target in the user message. This leverages the LLM's native role-based message separation — a much stronger signal than in-text markers.MaxConversationRunesto prevent it from crowding out the extraction target.ContextMessagesis nil, behavior is entirely unchanged (backward compatible).Test plan
TestExtractWithContextMessagesto verify the two-message structure (system + user), system message contains context content and restriction, user message contains only new messages.TestExtractWithoutContextMessagesto verify backward-compatible single user message when no context is provided.Made with Cursor