feat: dynamic context window and compression improvements

[laynepenney]

Summary

• Dynamic context window: Context limit now adapts to the model being used (40% of model's context window)
  • Claude 200k → 80k threshold
  • GPT-4o 128k → 51k threshold
  • GPT-4 base 8k → 3.2k threshold
• Compression disabled by default: Was confusing models that output E1/E2 symbols instead of actual values
• Compression improvements: When enabled (--compress), only applies if it actually saves space; output is now decompressed
• RAG-enhanced compaction: Code relevance scoring, file context in summaries, semantic deduplication
• Code quality improvements: Vector utilities extracted to separate module, static imports for better performance

Test plan

• Run pnpm test - all 1324 tests pass
• Verify getContextWindow() returns correct values for known models
• Verify decompressWithBuffer() handles streaming chunks correctly
• Verify gpt-4 does not incorrectly match gpt-4o (different context windows)
• Manual test: start with Claude, verify context limit is ~80k
• Manual test: start without --compress, verify compression is disabled
• Manual test: start with --compress, verify no E1/E2 in output

Manual Test Results

Test	Result
Claude context limit	✅ /status shows Tokens: 0 / 80000 (40% of 200k)
Compression off by default	✅ Shows Compression: disabled without flag
--compress enables it	✅ Shows Compression: enabled with flag
No E1/E2 in output	✅ Decompression working correctly

Commits

1. feat: dynamic context window and compression improvements - Core context window and compression changes
2. fix: update maxContextTokens when provider changes - Model switching fix
3. feat: RAG-enhanced context compaction - RAG integration for smarter compaction
4. refactor: extract vector utilities and improve model matching - Code cleanup and model matching fix

🤖 Generated with Claude Code

[laynepenney]

PR Review

Overall this looks good. A few observations:

1. Model switching doesn't update context limit ⚠️

maxContextTokens is set once in the constructor. If a user switches models mid-session with /switch (e.g., from Claude 200k to GPT-4 base 8k), the context threshold remains at 80k which could exceed GPT-4's actual context window.

Suggestion for future: Either update maxContextTokens when model changes, or compute dynamically in compactContext().

2. Minor inefficiency in decompressWithBuffer

[...entities.keys()].some(id => id.startsWith(potentialId) && id !== potentialId)

This iterates all entity keys on every streaming chunk. Fine for typical entity counts (<50), but could be optimized with a Set or trie if needed later.

3. Missing direct tests for getModelContextWindow()

The getModelContextWindow() helper in models.ts is tested indirectly via BaseProvider.getContextWindow() tests, but doesn't have dedicated unit tests. Consider adding tests for edge cases like:

• Partial model name matches
• Case sensitivity
• Models not in registry

4. Good catches ✅

• Resetting lastCompressionEntities and compressionBuffer at the start of each iteration prevents stale state
• Size comparison includes legend overhead correctly
• Buffer flushing at end of response handles edge case of partial entity at stream end

---

Verdict: Approve with the note that issue #1 should be addressed in a follow-up if model switching is commonly used.

[laynepenney]

This is calculated once at construction time. If the user switches models with /switch, this value won't update. Consider computing dynamically in compactContext() or updating when model changes.