Skip to content

agenttalk 0.101.1

Choose a tag to compare

@zoolok17 zoolok17 released this 14 Oct 03:18
· 496 commits to master since this release
c3f8b52

In short: one fix for the paid gateway. When the model pauses for a long time in the middle
of a reply, the gateway now waits up to ten minutes for the next piece instead of two. Such a
call used to end as an "uncertain" paid attempt that stopped every gateway seat until someone
settled it by hand. The gateway also writes one log line for each reply it streams, so a
future stall shows what happened. Nothing else changes: LiteLLM, the spending ledger and the
prices are as before, and the installed skills do not need refreshing.

Fixed

  • A long silent model reply no longer ends as an uncertain paid attempt (#424, part A). The
    gateway's front door gave up when the model sent nothing for two minutes. That left the paid
    call "uncertain" and stopped every gateway seat until someone settled it by hand. Once a reply
    has started, the front door now waits up to ten minutes for the next piece (connecting and
    starting the reply still wait two minutes). It also writes one line per started reply to
    stream-waits.jsonl in the gateway's state folder: the longest wait for a byte, the total
    time and how the call ended, so the next incident shows what really happened. A reply that
    stays silent for more than ten minutes is still marked uncertain, exactly as before; the
    LiteLLM version, the ledger and the prices are unchanged.
    Technical detail: FrontConfig.stream_idle_timeout_seconds (STREAM_IDLE_TIMEOUT_SECONDS,
    600 s) is applied to the upstream socket after the response headers arrive; the front keeps
    the socket from before getresponse() because http.client drops it on "Connection: close".
    request_timeout_seconds (120 s) is unchanged for connecting and waiting for the headers.
    GatewayFront.stream_wait_observer receives one record per started reply, and StreamWaitLog
    (bounded to 1 MiB, one rotation) writes it.