Skip to content

CarWatch v0.5.1 - see the whole answer, and a truthful estimate

Latest

Choose a tag to compare

@ThinkOffApp ThinkOffApp released this 29 Aug 17:48
· 5 commits to main since this release

Both fixes came straight out of live testing in the car this afternoon, on top of the v0.5.0 model chooser.

A chat view beside Ask. The answer popup truncated long answers and then vanished after 25 seconds, which is no way to read what a 35B just told you. The new Chat button opens a full-screen scrollable conversation: question and answer bubbles, the answering model and the measured seconds printed under each answer, its own input so you can keep asking from inside it, and per-phone history of the last 50 exchanges. The Ask box and the chat share one code path, so estimates and answers stay identical whichever one you use.

Answer-time estimates now follow the running model. The dash hardcoded "~1 min at 3.5 tok/s" and the voice strip showed the 35B's median, so a freshly swapped 2B model was still being advertised at the big model's speed - the one number a chooser must not get wrong. Estimates are now per model: the measured median for that model once it has samples, a bench-derived estimate from its own prompt and generation speeds when it has just been swapped in, and only then a global fallback. The request timeout scales with the estimate instead of sitting at a fixed 120s, which is what used to cut off a slow model mid-answer.

Also in this tag: the README now has a section explaining the model chooser, with a screenshot taken in the car mid-swap.

  • expect_s is exposed in GET /api/models, so any client can show the same estimate the dash shows
  • 34 tests pass, including new coverage for per-model estimate selection