Skip to content

v0.5.3

Choose a tag to compare

@funakoshi-takehiro funakoshi-takehiro released this 11 Aug 00:49
· 19 commits to main since this release
4a33346
Show what you typed before the answer comes back

The form reaches the display now — the spinner proves it — but the first
thing Talk.stream yielded was the first non-empty fragment of the reply.
Until the model produced something there was nothing on screen but a
spinner, in a chat, where your own message is the one thing that should
never be in doubt. It now yields the conversation as soon as the message
is in it, with the spinner sitting in the assistant's bubble where a
messenger would put it.

That also makes the wait legible, which it wasn't. A spinner said the
same thing whether the model was mid-sentence or three minutes into a
1.5GB first download, so a slow load was indistinguishable from a hang —
which is how it was read. Ai.is_loaded() separates them and the waiting
bubble says which one is happening.

An answer that comes back empty used to leave an empty bubble, equally
unreadable. A thinking model that spends its whole budget inside <think>
does exactly that. It now says so and suggests raising max_tokens.

None of this explains a wait that never ends, and I cannot reproduce one
here. check_ai_colab.py --chat drives the same stream outside the form
and prints each update with its elapsed time, so the next report can say
whether it stalls in load or in generation.