Skip to content

v0.1.7 — Virtual context scaling

Latest

Choose a tag to compare

@MaorBril MaorBril released this 23 Jul 17:39
ffe05b5

Virtual context scaling (#4)

Claude Code sizes its auto-compact against the ~200K window of the Claude model it believes it's talking to — routed models rarely match. Models can now declare what they really hold, and the router scales every client-facing token count so the client's context gauge and auto-compact track the real window:

models:
  qwen: {provider: local, id: qwen3-coder-30b, context_window: 32768}
  glm:  {provider: z, id: glm-4.7, context_window: 200000, effective_context: 60000}
  • effective_context is the attention knob: compact before the model leaves its coherent range, even under the nominal window.
  • Pricing, budgets, and the usage log always record true upstream usage; scaled values are stored alongside (reported_input, ctx_budget) for auditing.
  • New agentic context [session-id]: request-by-request trajectory of true vs reported tokens with compaction points marked.
  • agentic models add --context-window / --effective-context; models list shows a CTX column.
  • Verified live end-to-end: auto-compact fired mid-session on a routed model at 88% of its declared budget and the session ran to completion.

Note: Claude Code only engages its compaction machinery for model ids it recognizes — to get client-side compaction on a scaled model, alias it as a 200K claude-* id (an explicit model entry overrides the built-in passthrough). Details and research methodology: docs/context-scaling.md.

🤖 Generated with Claude Code