Skip to content

Releases: Techainer/summo-app

v0.38.1

Choose a tag to compare

@github-actions github-actions released this 28 Sep 16:03

Three faults on the meeting screen, found by taking a screenshot of it.
I had changed that screen twice this week without once looking at it.
None of the three is visible from reading the code.

The level meter drew straight through the clock

At 820 pixels wide the recording bar read |||||D0b1Q|||||. The meter's
bars each have a minimum width, so below about a thousand pixels the row
cannot give it enough and it overflows — across the elapsed time, which
is the one number on that bar that has to stay readable.

It now keeps its own row until there is genuinely space for it, and is
clipped rather than allowed to overflow: a meter that will not fit
should be cut off, never drawn over something else.

An empty box where the words should be

Your notes and the transcript split the width evenly, so a blank note
field sat beside arriving text on a wide screen — and on a narrow one,
210 pixels of blank field above it, between the recording bar and
the thing the recording produces.

The transcript takes the larger share now, and comes first when the
columns stack. A ratio rather than a collapse: typing during a meeting
is the other half of what the screen is for.

A download button that looked like a heading

In the panel over a running meeting, where every other control is small,
the "download the model" button drew as a line of large bold text alone
in a grid cell. It read as a title rather than as something to press.
That one was introduced in 0.35.0.

And a flaky test, now that the cause is known

One browser test changed the meeting's language and then waited three
seconds. Building a speech model takes about 2.7 seconds, so the wait
sat directly on top of it and failed roughly one run in four with "the
daemon did not take the new language" — a true statement about a daemon
three hundred milliseconds away from taking it. It waits for the change
instead of guessing how long it takes.

v0.38.0

Choose a tag to compare

@github-actions github-actions released this 28 Sep 14:55

Which model to use for Japanese, Korean, Chinese and Cantonese

Summo knew Chinese for all three of its multilingual speech models and
Japanese for exactly one. That blank was not neutral: Summo ranks
models on how accurate they have been measured to be on the languages
your meeting declares, so a Japanese meeting had nothing to rank on and
fell through to whatever order the catalogue happened to list. The model
page said "never measured on this language" and left you to guess.

Measured now — 100 FLEURS clips per language, character error rate,
lower is better:

whisper-tiny whisper-base sense-voice-small
Japanese 55.7 % 30.0 % 8.4 %
Korean 32.8 % 25.2 % 8.5 %
Chinese 51.8 % 44.2 % 7.4 %
Cantonese refused refused 8.4 %

Bold is new. Chinese was re-run rather than copied, and reproduced the
published figures to the decimal — which is what makes the rest of the
table worth trusting.

sense-voice-small is three to six times more accurate than the best
Whisper on every language it covers
, and it idles in less memory than
whisper-base does. For a meeting in any of those languages it is not a
preference. Those numbers are in the catalogue now, so Summo ranks on
them instead of on list order, and the model page shows them instead of
a blank.

Word error rate is deliberately absent for Japanese, Chinese and
Cantonese. Nothing separates words in those scripts, so every model
scores above 100 % and the number means nothing:
sense-voice-small returns 127 % WER at 8.4 % character error on
Japanese, which is as clear a demonstration as that gets.

Whisper cannot do Cantonese at all

Not a bad score — a refusal. Cantonese is not one of Whisper's
languages, and the Whisper entries in the catalogue claim every
language, so they are offered for a language they cannot attempt.

Nothing breaks: a decode that fails costs that line its second opinion
and the recording carries on. But a Cantonese meeting with only Whisper
installed will not produce anything usable, and until this is fixed the
catalogue will not tell you so. Install sense-voice-small for
Cantonese.

Recorded as a known limitation rather than patched over, because fixing
it properly means teaching Summo which languages each runtime refuses —
its own piece of work, and worth doing once rather than guessing at.

v0.37.2

Choose a tag to compare

@github-actions github-actions released this 28 Sep 12:35

Nothing new to use. This is the guard around the fix in 0.37.1, which
was thinner than the bug deserved.

That bug — a second model correcting a line's language and the
correction being thrown away when the line was merged — broke three
separate features for two releases. Every one of those features had
passing tests. Every one of them was correct on its own. Nothing ran the
chain they formed, so nothing noticed.

A merge can no longer drop a field in silence. The test that covers
it names every field explicitly, so adding a new one to a transcript
line stops the build until somebody decides what a merge does with it.
Auditing the rest while there: three fields are refused on purpose —
they identify the line or record when the audio began — and the other
seven are taken. Nothing else was being lost.

And a test that spans the pieces instead of checking each one, in the
order a real meeting produces them: the fast model mishears a Vietnamese
sentence and calls it Russian; a subtitle is written against that, which
is correct at the time because nothing yet knows better; the specialist
hears it properly and says what language it was; and the file must end
up with a Vietnamese line, no Vietnamese subtitle, and the English one
intact.

Both were checked the way these get checked here: with the fix reverted,
they fail.

docs/benchmarks.md now records the real two-language run — the models
the daemon chose, the transcript against a recording whose contents are
known, and what that suite does and does not prove.

v0.37.1

Choose a tag to compare

@github-actions github-actions released this 28 Sep 11:11

Asked whether any of the last three releases had been run on real
multilingual audio. They had not — the closest suite pairs a good
multilingual model with a specialist, another plays no audio at all, and
a third uses a one-language clip. The combination every one of those
defects actually lived in — the small multilingual model listening to a
meeting in two languages — had never been driven end to end.

Driving it found a real bug in the first run.

A second model corrected the words and the language was thrown away

Summo merges a revision into the line it replaces: the text, who spoke,
the confidence, the word timings. Not the language. So a model that
heard a line better — and therefore also knows which language it was,
which is most of why it was asked — handed that answer to a merge that
dropped it, and the file kept the first model's guess.

On real audio that looked like this, against a recording whose contents
are known: a Vietnamese sentence, correctly rescued, filed under
Russian.

It silently undid three things shipped in the last two releases: a
specialist stamping the language it speaks, forgetting a label for a
language nobody declared, and dropping a subtitle in the language a line
turned out to be in. All three read the language after the merge. All
three had passing tests, because every piece was correct on its own.

The suite that found it now runs every time

Real mixed-language audio, the small model listening, two languages
declared, a specialist for one and nothing for the other. It checks what
these releases were about rather than which words came back:

  • no line filed under a language nobody declared,
  • both declared languages present in the transcript,
  • the Vietnamese line produced by the model that can hear it,
  • the declared language with no specialist given a pinned decoder,
  • nothing repeating itself.

What it is not, stated in the suite itself: which language a small model
guesses on a given clip varies between runs, so it is a smoke test over
the real path rather than a proof of any single fix. Reverting the fix
and re-running it still passed — that run happened to guess right. The
deterministic guard is the unit test. Better to say that than to leave a
green suite that looks like more than it is.

v0.37.0

Choose a tag to compare

@github-actions github-actions released this 28 Sep 09:54

Both halves of the last report, which the previous release only answered
one of.

A decoder for every language you name

"Khai vi+en thì chạy whisper vi+en thôi — chạy zh he ja làm gì? Lọc ra
ok nhưng mà không đúng."

Right, and for a harder reason than 0.36 treated it as. Whisper's
language setting is not a label on the output — it is the decoder's
prompt
. Left free it detects across ninety-nine languages and then
decodes as whichever it picked, so a Vietnamese sentence it guessed was
Chinese comes back in Chinese characters. Correcting the label
afterwards cannot un-write the text. Filtering was treating the symptom.

Now every language you name has a decoder built for it. A language with a
specialist uses the specialist. A language with none — English, on the
meeting that was reported — gets the multilingual model pinned to it,
and pinned it cannot wander:

refining with models=["gipformer-1.5-68m", "whisper-tiny[en]"]

Driven entirely by what you declare. Name three languages and three
decoders appear; name one and nothing is pinned at all.

Translating sentences, not the pieces silence cut them into

"Dịch 1 câu thì dễ sai… dịch thì phải dịch đủ câu."

Summo cuts audio on silence, not on sentence ends, so what reached
the translator was very often half a thought — Hoặc là anh bảo việt anh bận tí thì. Half a thought in, half a translation out.

The obvious rule, waiting for a full stop, fails exactly where it is
needed: the Vietnamese model emits no punctuation at all. A version
that waited for one reached a test run and was thrown away — every
Vietnamese subtitle paid the full delay and then arrived in
240-character blocks.

So the boundary is the pause, which is on every line already and
needs no model to produce it. Three rules keep it free in the common
case: a line long enough to stand on its own goes immediately;
punctuation only ever sends sooner; and a change of language ends the
sentence, because two people taking turns is not one sentence.

A fragment nobody finishes waits 1.2 seconds and goes as it stands. A
late subtitle of half a sentence beats none.

What this cost, honestly

A sentence that is still being assembled delays its own subtitle until
it finishes. For the model Vietnamese meetings use, most lines are whole
clauses already and are not delayed at all. Short fragments are — that
is the trade, and it is the one that was asked for.

v0.36.0

Choose a tag to compare

@github-actions github-actions released this 28 Sep 07:10

A Vietnamese-and-English meeting came back as Chinese, Hebrew and
Japanese, with a Vietnamese line subtitled into Vietnamese. This is that
report, traced to the end.

The specialist was locked out by the guess it exists to correct

The first reading was "whisper-tiny is too poor — warn, or refuse to
record". That was wrong, and the user said so: gipformer was installed,
and several Vietnamese lines in the very same transcript are perfect.
The specialist was running. The question was why it did not run on the
lines that were broken.

Because routing asked whisper which language each sentence was, and
refused every specialist that did not claim the answer. Whisper-tiny
answered zh, he, ja, ru — nothing claims those, so the rescue
path was shut. Above two seconds the wrong answer was actively
believed.

The languages you name are now what decides. A language you declared
gets its specialist whatever the fast model guessed; the guess only
decides which specialist is tried first. Whether an answer is kept is
settled by what the model produced, not by what anything claimed — a
specialist fed a language it does not speak goes quiet, measured at 15
characters against 146, and that silence is the referee.

If you name no languages, nothing changes: there is no better signal
than the guess.

A rescued line kept the wrong label, so it was translated wrongly

A specialist reports no language — there is nothing for it to choose
between — so a Vietnamese line it had just rescued kept whisper's zh.
That label decides which way a line is translated, which is why Các cái nguồn lực came back subtitled Carregla in Vietnamese. A specialist
that has been heard now stamps the language it speaks.

And the subtitle already drawn against the wrong label is removed once
the language is known — in what you are reading and in the subtitle files
on disk.

The loop filter could not see Chinese or Japanese

今天今天今天… twenty-four times, with Vietnamese and English subtitles
of it. The repetition guard splits on spaces, and that text has none, so
the whole loop was one word and slipped under every threshold. It now
reads scripts without word spaces properly, and catches a loop that
starts partway through a sentence — a model does not usually begin stuck,
it says something and then jams.

And the screen

  • Minimised showed no words at all. The meter is sized by its
    container; as a bare row item it took the whole strip. The line is
    back, and the translation with it — which is what you shrink the window
    for.
  • The settings panel sat open over the transcript. "Đang ghi + config
    to quá không xem được transcribe." It folds now, and the control that
    opens it says what is behind it rather than reading "Đổi".
  • "Tải model (0 MB)" did nothing when pressed — a language with no
    specialist in the registry has no model to download. It says that in
    words now, without a button.
  • The status line names every model doing the listening, not just the
    first, so "why is the Vietnamese right and the English not" is
    answerable from the screen.
  • The transcript gets a reading measure and sets translations in against
    a rule, instead of three near-identical greys per line.

Still true, and worth saying

Translation with SMALL100 is sentence by sentence: that model's interface
takes one line and has no place to put context. Contextual translation
needs the LLM path, and that is the next piece of work rather than
something claimed here.

v0.35.2

Choose a tag to compare

@github-actions github-actions released this 28 Sep 03:28

A release with no new features in it, closing two things the last two
left open.

Proof that three languages really means three models

0.35.0 said Summo now pairs a specialist with every language you
name rather than only the first. It did — but nothing outside the code
could tell you so, and this is a feature that had already spent
releases quietly doing nothing.

There is now a test that names Vietnamese, English and Japanese and
counts the speech models the daemon says it is running. Both readings,
because they are different claims: what the meeting was configured
with, and what is actually loaded and costing you memory. A version
that picked three models and loaded one used to look identical from
outside; now it fails before it ships.

live:     whisper-base
resolved: gipformer-1.5-68m, parakeet-tdt-110m-en, sense-voice-small
loaded:   gipformer-1.5-68m, parakeet-tdt-110m-en, sense-voice-small

The other half is checked too: naming one language must load
exactly one second model and no more — three decoders is several
hundred megabytes each, and a meeting in one language has no use for
them.

The remaining wait, settled

0.35.1 got pressing record down to about a third of a second for anyone
who pauses a moment first, and said the last 2.9 seconds — building the
speech model — was "a different piece of work". That work is done, and
the answer is that it cannot be made smaller from inside Summo.

Timed at one, two, four, eight and sixteen threads: 2636, 2722, 2704,
2741, 2680 ms.
Flat. It is not thread pools and it is not the disk.
It is ONNX Runtime building the model's graph, and the library Summo
uses offers no way to lower the optimisation level or cache the result.

So the cost gets moved, not shrunk — which is what Summo now does, at
the earliest moment it can know somebody is there. Both tables are
written down in the repository so nobody spends an afternoon rediscovering
this.

Nothing to do when you update. Everything above is a check that runs
before a release rather than something you will see.

v0.35.1

Choose a tag to compare

@github-actions github-actions released this 28 Sep 00:11

Pressing record was racing the model load, and losing

Last release put a number on the complaint that starting a meeting is
slow — 2977 ms — and stopped there. This one takes it apart.

Where the time was going, on this machine:

pressed straight away 3489 ms — model built after the press: 2907 ms
pressed 2 s after opening 347 ms
pressed 5 s after opening 298 ms
model already prepared 245 ms

So Summo's habit of preparing the model ahead of time works — ten times
over. The problem was that it is only prepared for the first couple of
seconds after you open the app, which is precisely when somebody who
opened Summo in order to record presses record.

Two things were wrong, and neither was visible from anywhere.

Pressing during the preparation started a second one. Summo knew
what it was holding and not what was coming, so a press mid-build
found nothing and loaded its own copy beside the one already being
built — two of them competing for the same cores, and a few hundred
megabytes that existed for no reason. Now the press waits for the model
already on its way.

The preparation started too late. It began when the record card
appeared on screen. Summo can know sooner: the request for the app's own
page is the first sign somebody is here, seconds before the interface has
finished loading. A Summo nobody has opened still holds nothing.

Afterwards:

pressed straight away 3000 ms
pressed 2 s after opening 269 ms

If you take even a moment between opening Summo and pressing record —
reading the screen, picking a language — it now starts in about a third
of a second. Pressing the instant the window appears still waits for the
model, and that wait is now 2.9 seconds once instead of twice over.

Making the model itself build faster is a separate piece of work. Saying
so is better than a release note that implies the number is finished.

v0.35.0

Choose a tag to compare

@github-actions github-actions released this 27 Sep 15:51

A specialist for every language you name

Summo could pair one specialist model with the language you named
first. Name Vietnamese, English and Japanese and the Japanese half of
your meeting had no specialist at all — it was whatever the multilingual
model happened to hear. On the published figures that is 8.4 %
character error against 39.8 %: most of the accuracy of every Japanese
sentence anybody said.

Now each language you name gets its own model, and each utterance goes to
the model that names its language rather than to one that claims all of
them. A model brought in for one language no longer waits behind a
sentence in another — they are different models and there was never a
reason beyond a single counter for them to queue.

If you name a language whose model is not installed, Summo says so and
offers the download, instead of showing a chip that bought nothing.

Việc cần làm is a board you can actually use

Every way into it needed a meeting behind it, so a task nobody had
recorded could not be written down at all. And a task could be dragged
between columns and nothing else: no fixing the half-sentence a model
pulled out of a summary, and no removing a line that was never a task —
which could only be dragged to Xong, a lie in the one place you look
to find out what you finished.

  • Write one down — a task, who it is for, when it is due.
  • Reword it, without opening the Markdown file.
  • Delete it. An agent's plan goes with it.
  • A list as well as a board, because four columns are good for moving
    work along and bad for reading it in order.

Loose tasks live on an ordinary note, so they are still a list you can
open in Obsidian or anything else that reads Markdown.

Thư viện: sort, a flat list, and old recordings that play

There was no sort — filters, yes, but the order was fixed. There is one
now, newest first by default, plus the flat list that makes sorting by
title or by length mean anything at all: inside a day heading it could
only ever reorder that day.

And the detail pane has always counted your recordings — "· 2 bản
ghi" under the date — while offering no way to hear one. The player and
the route both existed. The only screen that lists old meetings could
reach neither. Now it can.

The readout left the chrome of every screen

The RAM and RTF numbers were in the footer of every page. They are behind
the performance switch now, and still on screen for anyone who wants
them, under Cài đặt → Giao diện, beside the switch that decides it.

This was attempted last release and pulled, because hiding a number is
only acceptable if it is still somewhere and that could not be shown. It
can now: one browser test asserts the number is readable where it moved
to, another asserts it has not come back to the footer.

A recording falling behind is not behind that switch. That is the
recording going wrong, not a statistic you opted into.

Measured, not fixed

Pressing record takes 2977 ms before the meeting is running. That is
printed on every test run now, and the run fails above eight seconds — so
a change that makes it worse shows up in the log of the run that made it
worse. It is slower than it should be: Summo prepares the model ahead of
time and the press is beating it. That is the next thing to fix, and
saying so is better than a release note that does not mention it.

v0.34.0

Choose a tag to compare

@github-actions github-actions released this 27 Sep 14:24

Somebody spoke Vietnamese and English into one meeting and got this back:

said  "Xin chào, tôi tên là Việt"   heard  "Sing out the dinner field"
said  "alo alo"                     heard  "Am I wrong? Am I wrong"
said  "Excuse me"                   heard  "And over the door"

This release is that report, traced to the end. Three separate faults, each measured before it was fixed, and two plausible fixes ruled out by measuring rather than by trying them.

A setting was telling the model what to hear

No language had been chosen for that meeting. models.language — set once, applying to every meeting ever recorded — was being handed to the live model as decode as this, and naming a language is exactly how a model that can detect stops detecting. A model told to expect English does not fail on Vietnamese; it produces confident English that was never said.

It also made the bilingual arrangement unreachable. The pairing that exists for precisely this meeting only runs when no language is named, and the setting named one first, every time.

A preference chooses a model now and stops there. A language named for this meeting is still honoured — "this call is in Japanese" is a fact about the call.

The language label is a coin toss on short phrases

The whole arrangement rests on the multilingual model labelling each utterance so a specialist can be handed the ones it is for. Nobody had measured whether that label is worth anything.

Whisper, no language named, FLEURS clips cut to length, 25 per cell:

0.6 s 1.2 s 2.5 s
whisper-tiny, Vietnamese 0 % 28 % 72 %
whisper-base, Vietnamese 0 % 28 % 76 %
whisper-tiny, English 12 % 56 % 100 %
whisper-base, English 12 % 64 % 100 %

Under a second the label is worthless. alo alo is exactly that long.

A bigger base model does not fix it — whisper-base is within four points of whisper-tiny at every length. There is not enough signal in half a second to tell two languages apart, so this is a property of the audio, not of the model. That was the obvious fix and it was the wrong one.

So the label is not believed below two seconds. Shorter utterances go to the second model whatever they claim to be.

Routing on what the models said instead

If the label cannot be trusted, something else has to decide. Decoder confidence would be the principled answer and it does not exist here — no runtime populates it; only the fake decoder in the tests returns a number.

What does exist is what each model produced. gipformer-65m, 25 clips each:

Vietnamese audio   mean 146 characters    0/25 near-empty
English audio      mean  15 characters    2/25 near-empty

A specialist fed the wrong language goes quiet. It does not invent that language. Ten to one, against a label that is right zero percent of the time exactly where it matters most.

So a refinement that says a small fraction of what the live model said is treated as the wrong model for that sentence and dropped. Replacing real words with near-silence is the worst thing that path can do, and it is what the label was letting through — Excuse me arriving as "It goes me", then "Eclos me".

The bar was grading a model that was not there

Vietnamese recorded with whisper-tiny runs at 75 % word error and the status bar said nothing, because it was grading accuracy — the best model that exists — rather than serving_accuracy, the one doing the listening. A field whose own comment reads "the number you will actually get today".

Smaller

You can choose more than one spoken language during a meeting now. The answer to "what is this meeting in" has been a list for several releases and the panel opened mid-meeting had a single dropdown — so the one moment you discover your call is bilingual was the one moment you could not say so.

With two subtitle languages the panel warned that translating into the language being spoken would subtitle each line with itself, directly under a line explaining that each sentence is rendered into whichever of them it is not. Two rows of one panel contradicting each other. The warning is for one target, which is the only case it is true of.

Settings now shows what Summo is using — machine memory, what this process is holding, recognition speed — as three rows beside the switch that controls the footer readout.

Taking it out of the footer is what was asked for and is not in this release. The move was built, and the block it moves to did not render where I expected; shipping it unverified would have hidden a number that exists because of a real bug, with nothing yet proven to be pointing at. It comes back when it can be shown to work.