[Tech] How to Save Your Token / Time / Effort

[FFengIll]

1. Load Balance - run in parallel without provider limits; keep all providers working at most and best
2. Context Compact
   • general interaction: reduce token in context which is not necessary
   • auto-compact and manual compact: they cost yours, so we introduce a new solution to do compact in local without LLM dependency which works without token cost, but keep core context; furthermore, it save your session while context window break.
3. Affinity - Yes, cache hit is the best way to save tokens, affinity keep session with provider's model which help cache hit (generally upon 90%)
4. Advisor - server tool to ask high level model for help without long context to make the result clean and better

[richardxx]

Advisor is also a way for token saving.