What to do when your model keeps treating compressed old content as fresh user instructions?

[github-actions[bot]]

This thread mirrors the blog post What to do when your model keeps treating compressed old content as fresh user instructions? at creation time; later edits to the article do not rewrite this post.
本帖是博客文章的创建时镜像；文章后续更新不会改写本帖。权威来源：https://github.com/ranxianglei/billion-context/blob/master/website/blog/posts/2026-11-02_compress-at-fold-point.md

English

Some models, especially ones with weak attention like qwen3.8 27b, always rework things once compression kicks in. The reason: the content of the compressed summary gets treated as a user instruction. There is now a pretty good solution to this problem. After a compress call, we swap the positions of compress and the recent protected user messages, so on the next request the model treats compress as historical messages, avoiding attention drift. This parameter is an experimental feature, off by default, and can be turned on by ticking "fold replacement in place (compressAtFoldPoint)" on the web config page (or JSON compress.compressAtFoldPoint: true / env var BILI_COMPRESS_AT_FOLD_POINT=1).

中文版

一些模型尤其是注意力比如弱的模型比如qwen3.8 27b,只要一压缩,总是返工.原因是压缩摘要的内容当作用户的指令.这个问题现在有了一个比较好的解决办法.就是调用compress 后,我们把compress和最近几条用户保护消息交换位置, 这样下一次请求模型就以为compress是历史消息,从而避免注意力漂移.参数属于实验性功能,默认关闭,在 web 配置页勾选「压缩折叠是否原位置替换」(或 JSON compress.compressAtFoldPoint: true / 环境变量 BILI_COMPRESS_AT_FOLD_POINT=1)打开.