Skip to content

v0.37.1

Choose a tag to compare

@github-actions github-actions released this 28 Sep 11:11
· 7 commits to master since this release

Asked whether any of the last three releases had been run on real
multilingual audio. They had not — the closest suite pairs a good
multilingual model with a specialist, another plays no audio at all, and
a third uses a one-language clip. The combination every one of those
defects actually lived in — the small multilingual model listening to a
meeting in two languages — had never been driven end to end.

Driving it found a real bug in the first run.

A second model corrected the words and the language was thrown away

Summo merges a revision into the line it replaces: the text, who spoke,
the confidence, the word timings. Not the language. So a model that
heard a line better — and therefore also knows which language it was,
which is most of why it was asked — handed that answer to a merge that
dropped it, and the file kept the first model's guess.

On real audio that looked like this, against a recording whose contents
are known: a Vietnamese sentence, correctly rescued, filed under
Russian.

It silently undid three things shipped in the last two releases: a
specialist stamping the language it speaks, forgetting a label for a
language nobody declared, and dropping a subtitle in the language a line
turned out to be in. All three read the language after the merge. All
three had passing tests, because every piece was correct on its own.

The suite that found it now runs every time

Real mixed-language audio, the small model listening, two languages
declared, a specialist for one and nothing for the other. It checks what
these releases were about rather than which words came back:

  • no line filed under a language nobody declared,
  • both declared languages present in the transcript,
  • the Vietnamese line produced by the model that can hear it,
  • the declared language with no specialist given a pinned decoder,
  • nothing repeating itself.

What it is not, stated in the suite itself: which language a small model
guesses on a given clip varies between runs, so it is a smoke test over
the real path rather than a proof of any single fix. Reverting the fix
and re-running it still passed — that run happened to guess right. The
deterministic guard is the unit test. Better to say that than to leave a
green suite that looks like more than it is.