Repository navigation
[Feature] Claude OAuth 恢复上游连接复用,并提供可配置的空闲超时(IdleConnTimeout) #7483
Replies: 5 comments
|
Thanks for writing this up so thoroughly, and for the measurements from your mihomo setup. I was going to say to take this on, but I see that @krus210 already opened #7472 for it. Thank you! I will try to test it over the coming weekend against a local reproduction of the silent idle drop (a local proxy plus a middlebox that forgets idle flows without FIN/RST). If I get something working, I will report back. |
|
I tested this PR (9b7dd5c) against a local reproduction of the silent idle drop from #7357. Setup: CPA → mihomo (DIRECT outbound, Claude OAuth, streamed:
So the opt-in works as intended, and the default off keeps the v8.0.62 behavior. The last row shows that the idle timeout must stay below the idle cutoff of the path. With 300 s it hangs the same way v8.0.60 did. Burst, Claude OAuth: 3 parallel requests, then one request after 60 s, another after 60 s, then 3 parallel requests 10 s later.
On v8.0.60, the hanging request gets the second connection from the first burst. That connection was idle for about 130 s, even though the gap between requests was never longer than 60 s. This matches what @amazinghjj described in #7465. Claude API key:
The last two rows are not caused by this PR, since the same happens on the official v8.0.70. They show that the existing 300 s default for API keys can already hit #7357 today. Maybe a lower default for Anthropic upstreams makes sense, in this PR or in a follow-up. Two more points:
|
|
Thanks a lot for testing this, @mschirrmeister. The local reproduction with nftables is much better evidence than my mihomo numbers, and the 300 s row shows exactly why the idle timeout has to stay below the path's cutoff. Debug log. I added it to #7472 (commit e962dd9). With It reuses the trace helper Antigravity already had (I made it provider-agnostic), and it never logs URLs, headers or keys. A request that hangs right after a line with 300 s default for API keys. I agree it is worth changing, but I would keep it out of #7472: it changes behavior for people who already opted in per key, while #7472 stays strictly opt-in. I will open a follow-up PR that lowers the default Your note about mihomo's |
|
I tested e962dd9 with the same setup and With The follow-up plan sounds good. One idea for it: API keys with their own Small cosmetic point: Claude log lines show |
|
Thanks for testing e962dd9, and for the log excerpt — that |
Uh oh!
There was an error while loading. Please reload this page.
背景
message_stop之后把剩余的响应体读完,连接因此可以回到空闲池里复用,行为也和官方 Claude Code 客户端一致(连续多轮共用一条连接)。IdleConnTimeout。在会悄悄丢弃空闲 TCP 连接、两端都收不到 FIN/RST 的网络里,闲置几分钟后的请求会发进一条已经断掉的连接,然后卡住,直到客户端超时,详见 v8.0.60 起 Claude OAuth 流式请求复用闲置连接后卡住无响应:Claude Code transport 未设置 IdleConnTimeout #7357。MaxIdleConnsPerHost: -1。这样问题彻底解决了,但 OAuth 账号也就完全拿不到连接复用的收益,非流式请求也不再复用,比 v8.0.60 之前还严格。@mschirrmeister 你在 #7357 里提到愿意提交空闲超时的 PR。因为 #7357 已经关闭,我单独开这个 issue 来跟踪这个需求,欢迎你认领。我这边可以在透明代理环境下帮忙验证。
需求
在不重新引入 #7357 的前提下,让 Claude(特别是 OAuth 账号)重新复用上游连接,并且空闲超时可以配置。
建议方案
恢复 fix(claude): drain stream body so upstream connections are reused #7237 的 drain 逻辑(detachable context,以及在
message_stop之后最多读 64 KiB),只在开启复用时生效。用
IdleConnTimeout替代MaxIdleConnsPerHost: -1:给 Claude Code transport 设一个保守的默认空闲超时,建议 60–90 秒,和 net/httpDefaultTransport的 90 秒是同一个量级。MaxIdleConnsPerHost恢复默认的 2。提供 OAuth 账号也能用的配置项。目前
proxy-connection-pool只对 API Key 生效,OAuth 账号没法配置。可以参照现有的antigravity.connection-pool,在claude下增加一个全局配置,作用于 OAuth 账号,以及没有单独配置connection-pool的 API Key:默认值有两种选择,请维护者决定:
保留现有的自动重试:复用的连接如果在收到任何响应字节之前遇到 EOF/RST,CPA 现在会换一条连接重试。我这边看到的几次都在同一个请求内自动恢复了,这部分工作得很好。
(可选)可观测性:复用连接时,在 debug 日志里记录
httptrace.GotConnInfo的Reused和IdleTime,以后排查这类问题时,能直接看出是不是复用到了闲置很久的连接。补充:API Key
connection-pool默认 300 秒的风险connection-pool现在默认idle-conn-timeout: 300s(会作用于 Anthropic 上游的 Claude Code TLS transport),这个值是基于 ollama.com 测出来的。在我的网络里(代理使用默认设置),闲置 300 秒的连接已经会被悄悄断开,见下表。所以对 Claude 来说,300 秒作为默认值偏激进。建议 Claude 的默认值更保守,或者在文档里说明这个风险。依据(来自 #7357 的实测)
环境:国内网络,出口是旁路由上的透明代理(mihomo,fake-ip + TUN)。
同一条 keep-alive 连接,闲置后再复用访问
api.anthropic.com:keep-alive-idle: 600)keep-alive-idle: 60,interval: 15)在透明代理环境里,CPA 自己的 TCP keepalive 起不到作用:CPA 的 TCP 连接在本地代理那里就终止了,keepalive 探测包由本地代理直接应答。被悄悄断开的是“代理 → 节点”这一段,CPA 感知不到。所以只能靠应用层的
IdleConnTimeout。另外,请求间隔不等于连接空闲时间:#7357 里有一次卡住,发生在上一个请求刚结束、间隔 0 秒的时候。之前的并发请求在池里留下了一条闲置约 4 分钟的连接。按连接计算的
IdleConnTimeout能覆盖这种情况,按请求间隔判断则覆盖不了。收益估算(10-21 至 11-10 共 2659 条 Claude 请求,几乎都是流式,按距离上一个请求结束的时间统计):
claude tls: handshake upstream超时,每次都是等 5 秒后重试成功。这类超时只在新建连接时发生,复用可以把新建连接的次数降到大约七分之一。验收标准
httptrace验证)。enabled: false时,行为和 v8.0.62 一致。connection-pool的 API Key,行为不变。相关:#7326、#7237、#7357、2c5b01f4
All reactions