What happened?
On a network where pi.dev's AAAA record is a blackhole but its A record works fine, every non-model network operation in pi (/llama, /scoped-models, /model <name>, post-/login refresh, the startup availability sweep) stalls for the full ~5-minute undici headersTimeout instead of the request failing fast or falling back to IPv4.
PI_OFFLINE=1 makes it return immediately, because it disables the remote-catalog fan-out and the availability sweep — the only paths that actually open connections to pi.dev. The configured model itself (different origin) keeps working in either case.
Environment
@earendil-works/pi-coding-agent 0.83.0
- Node v24.16.0
- Linux x86_64, residential network, no HTTP proxy
Reproduction
pi.dev resolves to an IPv6 and an IPv4 address. The IPv6 path blackholes; IPv4 is healthy:
URL="https://pi.dev/api/models/providers/openai"
curl -4 -sS -m 12 -o /dev/null -w 'http=%{http_code} time=%{time_total}s ip=%{remote_ip}\n' "$URL"
# http=200 time=0.099s ip=172.67.221.13 ← IPv4 fine
curl -6 -sS -m 12 -o /dev/null -w 'http=%{http_code} time=%{time_total}s ip=%{remote_ip}\n' "$URL"
# Operation timed out after 12001ms (http=000) ← IPv6 blackholed
curl -sS -m 12 -o /dev/null -w 'http=%{http_code} time=%{time_total}s ip=%{remote_ip}\n' "$URL"
# Operation timed out after 12001ms (http=000) ← default path picks IPv6, hangs
A browser hits pi.dev instantly — it runs Happy Eyeballs (RFC 8305) and wins with IPv4 after the IPv6 attempt stalls ~250ms in. pi does not.
- On such a network, start
pi normally.
- Run
/llama (or /scoped-models, or /model <name>).
- The command sits on "Loading…" for ~5 minutes — the default
httpIdleTimeoutMs (300,000 ms).
- Repeat with
PI_OFFLINE=1: /llama returns immediately (still fetches the local llama.cpp model list over the net; only the pi.dev path is skipped).
Root cause
configureHttpDispatcher() in packages/coding-agent/src/core/http-dispatcher.ts builds the global dispatcher with only allowH2, headersTimeout, bodyTimeout, and custom clientFactory/factory:
const dispatcher = withUndiciErrorListener(new undici.EnvHttpProxyAgent({
allowH2: false,
bodyTimeout: normalizedTimeoutMs,
headersTimeout: normalizedTimeoutMs,
clientFactory: createUndiciClient,
factory: createUndiciOriginDispatcher,
}));
There is no connect: { autoSelectFamily: true } passed through to the undici Client. With autoSelectFamily unset, undici's connect resolves the hostname (AAAA first on this system) and connects over a single address family — IPv6 — which blackholes. There's no Happy-Eyeballs race to IPv4, so the request waits until headersTimeout (~300s, the default httpIdleTimeoutMs) before giving up. Because the catalog refresh's dangling promise serializes the rest of the runtime (see #7153 / #7301's mechanism), that one hung fetch cascades into every later registry entry point — explaining the "/llama stalls indefinitely" symptom.
Notably this is not a pi.dev-server-side regression like the cluster of existing stall issues (#7027, #7113, #7153, #7418): those reproduce with curl over both IPv4 and IPv6 hanging. Here curl -4 returns HTTP 200 in ~0.1s — only the AAAA path is dead. A Happy-Eyeballs-enabled dispatcher would route around it transparently.
Suggested fix
Enable Happy Eyeballs in the dispatcher's connect options so a dead AAAA (or A) record can be pre-empted by the other family in ~250ms instead of the 300s headersTimeout:
const CONNECT_DEFAULTS = {
autoSelectFamily: true,
autoSelectFamilyAttemptTimeout: 250,
};
function createUndiciClient(origin, options) {
return withUndiciErrorListener(new undici.Client(origin, {
...options,
connect: { ...CONNECT_DEFAULTS, ...(options.connect ?? {}) },
}));
}
EnvHttpProxyAgent already passes its connect block down to its clients; this fix also benefits normal agent/model API calls to any dual-stack origin (e.g. api.anthropic.com), not just the catalog.
Worth also noting: #1963 reported the opposite problem on Node 20 — autoSelectFamily was enabled by default and caused a hang because every Cloudflare candidate failed on that reporter's network; the workaround there was to disable it. Those two issues together suggest the right behavior is to enable Happy Eyeballs with a short autoSelectFamilyAttemptTimeout (250ms) and fall back gracefully when all candidates fail — not to leave the option unset as the current dispatcher does.
Workarounds
PI_OFFLINE=1 / --offline — skips the remote-catalog fan-out and availability sweep entirely (the local llama.cpp server and the configured model still work).
- Pin
pi.dev to IPv4 in /etc/hosts (e.g. 172.67.221.13 pi.dev) — removes AAAA resolution from the picture; zero pi config changes needed (Cloudflare A records may rotate).
- Set
"httpIdleTimeoutMs": 5000 in global pi settings — caps each blackholed fetch at ~5s so the catalog fan-out fails fast and falls back to the cached catalog. Doesn't fix the underlying IPv6 path, only the stall length.
Related
What happened?
On a network where
pi.dev's AAAA record is a blackhole but its A record works fine, every non-model network operation in pi (/llama,/scoped-models,/model <name>, post-/loginrefresh, the startup availability sweep) stalls for the full ~5-minute undiciheadersTimeoutinstead of the request failing fast or falling back to IPv4.PI_OFFLINE=1makes it return immediately, because it disables the remote-catalog fan-out and the availability sweep — the only paths that actually open connections topi.dev. The configured model itself (different origin) keeps working in either case.Environment
@earendil-works/pi-coding-agent0.83.0Reproduction
pi.devresolves to an IPv6 and an IPv4 address. The IPv6 path blackholes; IPv4 is healthy:A browser hits
pi.devinstantly — it runs Happy Eyeballs (RFC 8305) and wins with IPv4 after the IPv6 attempt stalls ~250ms in. pi does not.pinormally./llama(or/scoped-models, or/model <name>).httpIdleTimeoutMs(300,000 ms).PI_OFFLINE=1:/llamareturns immediately (still fetches the local llama.cpp model list over the net; only thepi.devpath is skipped).Root cause
configureHttpDispatcher()inpackages/coding-agent/src/core/http-dispatcher.tsbuilds the global dispatcher with onlyallowH2,headersTimeout,bodyTimeout, and customclientFactory/factory:There is no
connect: { autoSelectFamily: true }passed through to the undiciClient. WithautoSelectFamilyunset, undici'sconnectresolves the hostname (AAAA first on this system) and connects over a single address family — IPv6 — which blackholes. There's no Happy-Eyeballs race to IPv4, so the request waits untilheadersTimeout(~300s, the defaulthttpIdleTimeoutMs) before giving up. Because the catalog refresh's dangling promise serializes the rest of the runtime (see #7153 / #7301's mechanism), that one hungfetchcascades into every later registry entry point — explaining the "/llama stalls indefinitely" symptom.Notably this is not a pi.dev-server-side regression like the cluster of existing stall issues (#7027, #7113, #7153, #7418): those reproduce with
curlover both IPv4 and IPv6 hanging. Herecurl -4returns HTTP 200 in ~0.1s — only the AAAA path is dead. A Happy-Eyeballs-enabled dispatcher would route around it transparently.Suggested fix
Enable Happy Eyeballs in the dispatcher's
connectoptions so a dead AAAA (or A) record can be pre-empted by the other family in ~250ms instead of the 300sheadersTimeout:EnvHttpProxyAgentalready passes itsconnectblock down to its clients; this fix also benefits normal agent/model API calls to any dual-stack origin (e.g.api.anthropic.com), not just the catalog.Worth also noting: #1963 reported the opposite problem on Node 20 —
autoSelectFamilywas enabled by default and caused a hang because every Cloudflare candidate failed on that reporter's network; the workaround there was to disable it. Those two issues together suggest the right behavior is to enable Happy Eyeballs with a shortautoSelectFamilyAttemptTimeout(250ms) and fall back gracefully when all candidates fail — not to leave the option unset as the current dispatcher does.Workarounds
PI_OFFLINE=1/--offline— skips the remote-catalog fan-out and availability sweep entirely (the local llama.cpp server and the configured model still work).pi.devto IPv4 in/etc/hosts(e.g.172.67.221.13 pi.dev) — removes AAAA resolution from the picture; zero pi config changes needed (Cloudflare A records may rotate)."httpIdleTimeoutMs": 5000in global pi settings — caps each blackholed fetch at ~5s so the catalog fan-out fails fast and falls back to the cached catalog. Doesn't fix the underlying IPv6 path, only the stall length.Related
/scoped-modelsappears to do nothing for ~5 minutes while awaiting stalled catalog refresh #7153 — same/scoped-models//llama-style stall symptom, attributed by that reporter to a server-side pi.dev regression (curl hangs on both families for them)./loginTUI freeze on unreachable pi.dev./loginhangs after credential saved (no abort signal on post-login refresh).forceRefreshAvailability()chains onto the stuck promise, sogetAvailable()/refresh()never settle again even after the cause clears #7301 — a stalled availability refresh permanently wedges the runtime (the mechanism by which one hung fetch cascades to every latergetAvailable()//llamaetc.).autoSelectFamilyprecedent.