Skip to content

v1.8.4 — remove a retracted SSE section that reached main by a branching mistake

Choose a tag to compare

@latentharbor latentharbor released this 02 Aug 10:20
· 14 commits to main since this release

The README carried two streaming-cost sections disagreeing by about 80×: one reading 28 ms
for "20K in / 8K out" (3.5 µs per event), the other 250–310 µs per event.

The first is wrong. Its PR was closed as refuted — the "8K out" label is the artifact:
max_completion_tokens never bound with the prompt used, so that request streamed about fifty
events, not eight thousand. 28 ms was the cost of a fifty-event stream wearing an
eight-thousand-token label.

It reached main anyway. git branch -D refused while that branch was still checked out, printed
Aborting, and the next git checkout -b branched from it — so the refuted commit rode into main
inside the following PR. The claim was publicly retracted and the code carrying it shipped
regardless. Closing a PR retracts the proposal, not the working tree.

Removed with it: "turn HTTP/2 off", which did not reproduce, and "the request body is cheap at
4–6 ms", measured on the same mislabelled request.

What survives is the section measured with the API's own usage block as the token count. There is
now one.

1251 offline tests pass.