Meet billion-context · 认识 billion-context

[github-actions[bot]]

This thread mirrors the blog post Meet billion-context · 认识 billion-context at creation time; later edits to the article do not rewrite this post.
本帖是博客文章的创建时镜像；文章后续更新不会改写本帖。权威来源：https://github.com/ranxianglei/billion-context/blob/master/website/blog/posts/2026-10-08_meet-bili.md

English

Why context windows fill up

Every turn of an AI coding session resends the whole conversation back to the model. Tool outputs, logs, file reads and retries pile up far faster than anything you typed. By the time a task gets interesting, a session can burn through a 200K window in an afternoon — after which quality degrades or the session dies.

What sits between your agent and the model

billion-context runs locally, loopback-only, between your coding agent and the model provider. It keeps a token-level ledger for every request and compresses the older parts of the conversation into dense, referenceable summaries while the most recent context stays byte-exact. Cadence is growth-driven (roughly 50K tokens between folds), not percentage-driven.

Two modes cover both integration styles: plugin mode, where the agent owns compression (for example the pi extension), and proxy mode, where the proxy performs it server-side and plain clients never notice.

What you get

• Up to 5x fewer cumulative input tokens
• A single session measured across a month of real development
• An attribution panel: cache hit rate, net savings, active blocks — per request, in the built-in web UI

The numbers on the front page are production data: 174,327 model calls, 18.8B cumulative input tokens, zero window violations over 4.5 months on three hosts.

Where it runs

One npm package, Node 22+, MIT license. Install with npm install -g billion-context --prefix=~/.local (on nvm / Homebrew Node or Windows, drop --prefix), start the proxy with bili, and open http://127.0.0.1:8787/__bili/. Full configuration lives at ~/.config/billion-context/billion-context.json — { } is a valid configuration. Read the docs for everything else.

Note: this is a sample post demonstrating the blog format (a markdown subset rendered to static pages by tools/build-blog.mjs). The first real story will replace it.

中文版

为什么上下文窗口会满

AI 编码会话的每一轮都会把整个对话重新发给模型。工具输出、日志、文件读取和重试的堆积速度远快于你敲下的任何字。任务刚变得有趣，一个会话就可能一下午烧完 200K 窗口——之后要么质量下滑，要么直接死掉。

它站在你的 Agent 和模型之间

billion-context 在本地以「仅回环」的代理方式运行在你的编码 Agent 与模型服务之间。它对每一次请求维护一份 token 级账本，把对话较老的部分压成紧凑、可引用的摘要，而最近的内容保持逐字节不变。节奏是增长驱动的（大约每 50K token 折叠一次），而不是按百分比触发。

两种模式覆盖两种集成方式：插件模式——由 Agent 自己执行压缩（例如 pi 扩展）；代理模式——由代理在服务端执行，普通客户端完全无感。

你能得到什么

• 累计输入 token 最多降低 5 倍
• 实测跨越一整个月真实开发的单会话
• 归因面板：缓存命中率、净节省、活跃块数——按请求看，就在内置 Web 面板里

首页那组数字是生产数据：174,327 次模型调用、187.6 亿累计输入 token、三台主机 4.5 个月零窗口越界。

它跑在哪里

一个 npm 包，Node 22+，MIT 协议。npm install -g billion-context --prefix=~/.local 安装（nvm / Homebrew Node 或 Windows 可省略 --prefix），bili 启动代理，浏览器打开 http://127.0.0.1:8787/__bili/。完整配置在 ~/.config/billion-context/billion-context.json——{ } 就是合法配置。其余一切见文档。

说明：这是一篇示例文章，用来演示博客格式（markdown 子集，由 tools/build-blog.mjs 渲染成静态页）。首篇正式文章会替换它。