Skip to content

v0.3.172-alpha

Choose a tag to compare

@github-actions github-actions released this 11 Sep 05:42
· 547 commits to main since this release

Fixed

  • Phi models stop at the end of their turn instead of inventing the next one.
    Every Phi-3, Phi-3.5 and Phi-4 model declares one token as its end of sequence
    and then ends each turn with a different one, and nothing said so, so nothing
    stopped the reply there. Asked "say exactly: hello", a Phi model answered, then
    invented a second answer, then made up a question from the user and began
    answering that too, until it ran out of room. What you saw depended on the
    model's vocabulary: some carried a stray <|end|> marker into the visible text,
    others hid the seam entirely and simply rambled. A model's turn-ending tokens
    are now found in its own vocabulary rather than taken on trust, so both
    symptoms are gone, and models whose format uses that same token to separate
    parts of one long answer are unaffected.
  • Tool calls written in Phi-4's format are recognised. Phi-4-mini is built by
    its publisher for function calling and writes its calls in its own documented
    format, which nothing here could read — so every call it made came back as
    ordinary text with no error, and a client driving it would carry out nothing and
    have no way to tell. Both spellings it uses are now understood. Note that this
    model still cannot be served from shards at all, for a separate reason found
    while testing this release and not yet fixed, so the change is verified against
    its documented formats rather than against the model itself.
  • A model whose template mentions tools without using them is told about them
    anyway.
    One template asks for its tools in a place they were never offered,
    so its own tool section never appeared, and the written description was skipped
    as redundant — leaving the model told nothing at all about tools that were
    attached to the request. A reply built with tools that shows none of them is
    now treated as a failed attempt and the written description is used instead.
  • Tool calls from Qwen2.5-Coder are recognised. Its own prompt names two
    similar tags a few lines apart — one for the list of tools it has, one for
    writing a call — and the model reaches for the wrong one. The call it wrote was
    perfectly well formed and came back as plain text, so that model made no working
    tool call at all. Both tags are now understood.
  • A model repeating its tool list back is no longer mistaken for calling one.
    A tool's description and a call to it are nearly the same shape, and the
    description was being accepted as a call — with the tool's own parameter
    definition passed along where the argument values belong. A client would have
    received a call it never asked for, carrying nothing it could use. Zero-argument
    calls, which look similar, still work.
  • A tool call inside a code block followed by an explanation is read
    correctly.
    Reading the block required the whole reply to be nothing but that
    block, so the commonest thing a chatty model does — write the call, then
    explain it — defeated it, and a perfectly good call was handed back as text. A
    leftover code-fence marker no longer appears as the reply's visible content
    either.

Changed

  • Models are asked for a simpler shape when they have no tool format of their
    own.
    Two of the fields being requested were discarded on arrival, so asking
    for them only added nesting for a small model to miscount. Both the old and the
    new shape are still understood.
  • A request whose model has no readable chat template now says so in the log.
    That is the condition behind a whole class of confusing replies, and nothing
    previously named it.

Internal

  • Every step in the build that reaches the network is retried. A stale package
    index had already published one release incomplete; the retry added then
    covered the quick half of the work and not the half that downloads gigabytes.
  • The build cache no longer fills with entries nothing can use. Caches belonging
    to finished pull requests and past releases are reclaimed, and the cleaner no
    longer risks deleting a live cache in favour of an unusable one.
  • The build workflows are themselves checked now, which they were not. Every job
    has a time limit, no two runs of the same work race each other, and the shell
    inside each step is linted — including the retry loops the point above depends
    on.
  • A new pre-release check runs several model families and asserts that each
    produces a sane reply: that it answers the question, stops on its own, leaks no
    internal markers, and has its tool calls read back properly. Every
    model-specific fault fixed in the last four releases was reported by a user
    rather than found here, and each one would have passed the checks that existed.

Full commit list: v0.3.171-alpha...v0.3.172-alpha