Skip to content

MegaCapybara 1.17

Choose a tag to compare

View all tags
@perkel666 perkel666 released this 05 Oct 18:48
· 13 commits to main since this release
v1.17
52d86f2

What's new

  • Shared openings are read once. Agents that start with the same system prompt and tools read that part once; later ones get a copy in a few milliseconds and read only their own part. Prompts sent together (a dispatch of agents, or every conversation coming back after a restart) wait for the first one instead of each reading it. The engine finds the shared length by itself (an agent's system prompt and tools are often ~20K tokens), and keeps a few openings by how much reading they save, inside the RAM cache's budget. Answers are the same as reading the prompt whole.

    Shared openings: four agents with the same 20K-token opening, read four times in 1.12 and once in 1.17

  • The log says when a conversation cannot be resumed and why, and when a shared opening is kept or reused.

Fixed

  • Agents re-reading their whole conversation every turn (1.12). Under heavy agent load, 1.12's RAM cache could fill Windows' memory limit for the GPU; conversations then lost their resume point and each turn read its whole context again (on a busy session: 110 turns in 27 minutes re-read 4.8M tokens, and requests answered fell from ~500 to 24 per 10 minutes). 1.17 counts the memory the cache really holds and keeps room for every running conversation's resume point.

The "Source code" files GitHub attaches to this release hold only the README and the docs; the engine's source code will be released soon.

Full changelog · How to run it

Assets 4
Loading