Repository navigation
Breaking
HTTPProvider.Configuration no longer takes answerTokens: Int. Use answerBudget: .tokens(n). There is no deprecated overload. HTTPProvider.answerTokens (the 1,024 default) still exists as a static.
What changed
The four fixed limits on one HTTP answer are now configuration. Defaults match 0.5.2, so an existing caller that never set them behaves as before.
| Limit | Before | Now |
|---|---|---|
| Token budget | 1,024, fixed (0.5.3: an Int, at least 1) |
answerBudget: .tokens(n) (n at least 1) or .endpointDefault, which sends no token field so the server's own limit applies. A request without the field is never retried on a 400/422. The probe still asks for one token. |
| Timeouts | 60 s per request, 90 s overall | timeouts: Timeouts(idle:ceiling:), default 60 s idle and 90 s ceiling. The provider enforces them itself, so they now apply to an injected URLSession as well. A value over 100 years is refused at construction. |
| Reply cap | 1 MiB | responseCap, default 1 MiB. The 64 KiB cap on a refusal body stays fixed. |
| Delivery | whole reply only | delivery: .whole (default) or .streamed. A streamed request still returns one complete answer, never a partial one. The idle limit then restarts on every byte, so a slow model that keeps producing is not cut off. |
Streamed delivery builds the answer only from delta.content (reasoning deltas never reach it). A stream that ends without a finish reason is .badShape. finish_reason: "length" is still .truncated. Framing, line, event and answer caps are enforced as the bytes arrive.
AIProvider and AIProviderError are unchanged. The Go side and the fixtures are unchanged.
Also: a flaky cancellation test and several wall-clock-dependent tests in the CLI and conformance suites now wait on conditions or CPU time instead of fixed delays.
Not verified
- Streaming has only been run against a stand-in server. No live endpoint was tested, so real services' event framing, silence while reasoning, and behaviour without a token field are unmeasured.
- Memory: each streamed event's parsing is released with the event, but the parsed objects are not counted in the answer cap, and URLSession's own buffer is not bounded. Measurements of one large event disagreed between two runs (1-14 MB in a debug build versus about 50 MB);
spec/chat-completions.mdstates the range and the caveat. - The per-byte idle bookkeeping is a known cost: roughly 140 ms of a 1 MiB streamed answer in an earlier measurement; irrelevant at real answer sizes.
Checked
CI on macos-15 (Xcode 16) and macos-26 is green for this tag's commit. The full thread-sanitizer suite passed on a quiet 24-core Mac.
See spec/chat-completions.md, "Limits" and "Streamed delivery".