v0.34.0

Somebody spoke Vietnamese and English into one meeting and got this back:

said  "Xin chào, tôi tên là Việt"   heard  "Sing out the dinner field"
said  "alo alo"                     heard  "Am I wrong? Am I wrong"
said  "Excuse me"                   heard  "And over the door"

This release is that report, traced to the end. Three separate faults, each measured before it was fixed, and two plausible fixes ruled out by measuring rather than by trying them.

A setting was telling the model what to hear

No language had been chosen for that meeting. models.language — set once, applying to every meeting ever recorded — was being handed to the live model as decode as this, and naming a language is exactly how a model that can detect stops detecting. A model told to expect English does not fail on Vietnamese; it produces confident English that was never said.

It also made the bilingual arrangement unreachable. The pairing that exists for precisely this meeting only runs when no language is named, and the setting named one first, every time.

A preference chooses a model now and stops there. A language named for this meeting is still honoured — "this call is in Japanese" is a fact about the call.

The language label is a coin toss on short phrases

The whole arrangement rests on the multilingual model labelling each utterance so a specialist can be handed the ones it is for. Nobody had measured whether that label is worth anything.

Whisper, no language named, FLEURS clips cut to length, 25 per cell:

	0.6 s	1.2 s	2.5 s
whisper-tiny, Vietnamese	0 %	28 %	72 %
whisper-base, Vietnamese	0 %	28 %	76 %
whisper-tiny, English	12 %	56 %	100 %
whisper-base, English	12 %	64 %	100 %

Under a second the label is worthless. alo alo is exactly that long.

A bigger base model does not fix it — whisper-base is within four points of whisper-tiny at every length. There is not enough signal in half a second to tell two languages apart, so this is a property of the audio, not of the model. That was the obvious fix and it was the wrong one.

So the label is not believed below two seconds. Shorter utterances go to the second model whatever they claim to be.

Routing on what the models said instead

If the label cannot be trusted, something else has to decide. Decoder confidence would be the principled answer and it does not exist here — no runtime populates it; only the fake decoder in the tests returns a number.

What does exist is what each model produced. gipformer-65m, 25 clips each:

Vietnamese audio   mean 146 characters    0/25 near-empty
English audio      mean  15 characters    2/25 near-empty

A specialist fed the wrong language goes quiet. It does not invent that language. Ten to one, against a label that is right zero percent of the time exactly where it matters most.

So a refinement that says a small fraction of what the live model said is treated as the wrong model for that sentence and dropped. Replacing real words with near-silence is the worst thing that path can do, and it is what the label was letting through — Excuse me arriving as "It goes me", then "Eclos me".

The bar was grading a model that was not there

Vietnamese recorded with whisper-tiny runs at 75 % word error and the status bar said nothing, because it was grading accuracy — the best model that exists — rather than serving_accuracy, the one doing the listening. A field whose own comment reads "the number you will actually get today".

Smaller

You can choose more than one spoken language during a meeting now. The answer to "what is this meeting in" has been a list for several releases and the panel opened mid-meeting had a single dropdown — so the one moment you discover your call is bilingual was the one moment you could not say so.

With two subtitle languages the panel warned that translating into the language being spoken would subtitle each line with itself, directly under a line explaining that each sentence is rendered into whichever of them it is not. Two rows of one panel contradicting each other. The warning is for one target, which is the only case it is true of.

Settings now shows what Summo is using — machine memory, what this process is holding, recognition speed — as three rows beside the switch that controls the footer readout.

Taking it out of the footer is what was asked for and is not in this release. The move was built, and the block it moves to did not render where I expected; shipping it unverified would have hidden a number that exists because of a real bug, with nothing yet proven to be pointing at. It comes back when it can be shown to work.