Repository navigation
agenttalk 0.101.1
In short: one fix for the paid gateway. When the model pauses for a long time in the middle
of a reply, the gateway now waits up to ten minutes for the next piece instead of two. Such a
call used to end as an "uncertain" paid attempt that stopped every gateway seat until someone
settled it by hand. The gateway also writes one log line for each reply it streams, so a
future stall shows what happened. Nothing else changes: LiteLLM, the spending ledger and the
prices are as before, and the installed skills do not need refreshing.
Fixed
- A long silent model reply no longer ends as an uncertain paid attempt (#424, part A). The
gateway's front door gave up when the model sent nothing for two minutes. That left the paid
call "uncertain" and stopped every gateway seat until someone settled it by hand. Once a reply
has started, the front door now waits up to ten minutes for the next piece (connecting and
starting the reply still wait two minutes). It also writes one line per started reply to
stream-waits.jsonlin the gateway's state folder: the longest wait for a byte, the total
time and how the call ended, so the next incident shows what really happened. A reply that
stays silent for more than ten minutes is still marked uncertain, exactly as before; the
LiteLLM version, the ledger and the prices are unchanged.
Technical detail:FrontConfig.stream_idle_timeout_seconds(STREAM_IDLE_TIMEOUT_SECONDS,
600 s) is applied to the upstream socket after the response headers arrive; the front keeps
the socket from beforegetresponse()becausehttp.clientdrops it on "Connection: close".
request_timeout_seconds(120 s) is unchanged for connecting and waiting for the headers.
GatewayFront.stream_wait_observerreceives one record per started reply, andStreamWaitLog
(bounded to 1 MiB, one rotation) writes it.