Add Google Cloud Speech-to-Text v2 as an online speech-to-text engine

[niksedk]

Summary

Adds Google Cloud Speech-to-Text to the Speech to text engine list, using the v2 API over plain REST with word level timings.

• No new NuGet packages. The service-account JSON key becomes a bearer token via Google.Apis.Auth, which SE already ships for Google TTS. Everything else is HttpClient + System.Text.Json.
• No new language strings. The settings rows reuse "Key file", "Region", "Model", "Language Hint" and "Timeout (seconds)".
• Flow: ensure bucket → upload audio → batchRecognize → poll operation → delete object. BatchRecognize only reads from Cloud Storage; the bucket <project>-subtitle-edit-stt is created on first use with a one-day lifecycle rule (override via GoogleCloudSttBucketName in Settings.json).
• Parsing: one segment per Google result, timed by its first and last word. Words with impossible offsets are dropped (the API has been observed returning offsets far outside the audio), and a transcript that ends well before the billed duration is logged as possibly truncated.
• IOnlineSttEngine gains an optional MaxChunkSeconds; the shared chunker now also splits by duration. Google caps a file with word timings at 20 minutes, the engine uses 18.

Background: the plugin proposed in SubtitleEdit/plugins#289 ships a 40 MB self-contained app per platform. Folding the engine into SE costs a few tens of KB because the gRPC/auth stack is already present.

Setup for users

Speech-to-Text v2 rejects API keys. The user needs a Google Cloud project with billing, the Speech-to-Text API enabled, a service account with the roles Cloud Speech Client and Storage Admin, and its JSON key. Defaults: region us, model chirp_3, empty language hint = automatic detection.

Testing

• tests/UI/.../GoogleCloud/GoogleCloudSttServiceTests.cs: host selection, bucket naming, request body, response parsing incl. corrupt words and per-file errors, proto duration parsing.
• All 278 SpeechToText UI tests pass.
• Not tested against the live API (needs a funded Google Cloud project). REST shapes follow the v2 reference docs and match the plugin's working gRPC calls.

🤖 Generated with Claude Code

[muaz978]

I went through the merged code against the runs the numbers in your description come from.
The structure matches what I ended up with independently: the bucket lifecycle rule, the
finally that deletes the object, the regional host selection, recognizers/_, inline
output for a single file, and the duration chunker. I checked the chunk arithmetic on a
real case: a 8,716 s episode gives ceil(8716.98 / 1080) = 9 chunks of about 16.1 min, and
with the silence snapper moving each boundary by up to 10 s the worst case is about
16.5 min, so it stays clear of Google's 20 minute cap. That all looks right.

Four things I would raise, in order of how much they matter.

1. The engine sends 32 kbit/s MP3, and every accuracy number came from lossless FLAC

isOpenAiEngine is GetEffectiveSelectedEngine() is IOnlineSttEngine, which is true for
GoogleCloudSttEngine, so audio extraction falls through to the "mp3" branch in
SpeechToTextViewModel. GetFfmpegTranscodeFormatString describes that path as targeting
"~32 kbit/s mono at 16 kHz", and gives the reason as keeping a two hour video "well under
OpenAI's 25 MB upload limit".

That limit does not apply here. The audio goes to Cloud Storage, which has no such cap, and
the file is already split into 18 minute chunks.

This matters because all the figures quoted for this engine, the 13,175 timed words and the
54.3% speech density, were measured on 16 kHz mono FLAC, lossless. I have not measured
chirp_3 on 32 kbit/s MP3 at all, so I cannot tell you how much accuracy that costs, only
that it is a different input from the one the evidence describes. A 16 kHz mono FLAC chunk
is roughly 20 MB for 18 minutes, which is nothing for a bucket upload.

2. No processingStrategy, so every run bills at the undiscounted rate

BuildRequestBody does not set processingStrategy, so transcription bills at $0.016 per
minute instead of $0.003 with DYNAMIC_BATCHING. On my Teşkilat episode that is the
difference between $0.42 and about $2.23 for the same 139.74 minutes. Dynamic batching
trades latency for cost, so it probably wants to be a checkbox rather than a default, but
right now the cheaper path is not reachable at all.

3. Silent truncation is detected but only logged

The check in ParseResponse is right, and it is good that it is there at all. My concern is
what happens next: it writes a line to the tools log and returns the short transcript as if
it were complete.

This is the defect that cost me 11.4 minutes of dialogue out of one 18 minute chunk, with
the operation reporting success. The user gets a subtitle that looks finished, with a gap
they will only find by watching. In my own build I re-cut the tail from the last good word
and resubmitted it, which recovered the missing audio in one extra request. At minimum I
think this deserves surfacing to the user rather than only to the log.

4. The offset guard has no upper bound when totalBilledDuration is absent

if (start < 0 || end < start || (billed > 0 && end > billed + 1))

If totalBilledDuration is missing from the response, billed is 0 and the range check
degrades to "not negative and not inverted". The word I actually hit claimed 6,324 s inside
a 1,080 s chunk, and it would pass that. Bounding against the chunk duration instead is
always available and does not depend on the response.

One thing I have not verified

The default languageCodes of ["auto"]. Every one of my runs passed an explicit tr-TR,
so automatic detection on chirp_3 is untested by me.

I am setting up a project key to run this against the live API as you asked, and will
report back with results. Happy to send the fixes above as a PR.

[muaz978]

Ran it against the live API. Short version: your REST shapes are correct, the defaults
work, and the run turned up one real bug plus a measurement that I think justifies changing
the audio format.

Method: the exact request body BuildRequestBody produces, posted to
us-speech.googleapis.com/v2/projects/<id>/locations/us/recognizers/_:batchRecognize,
on 65 seconds of Turkish drama, submitted twice from identical source audio: once encoded
the way the engine encodes it today, once lossless.

What works exactly as you wrote it

• languageCodes: ["auto"] is accepted, and came back with languageCode: "tr". Good
  default, and I was wrong to flag it as risky.
• chirp_3 returns word level timings over REST, same as over gRPC. 160 words with
  startOffset and endOffset on a 65 second clip.
• The response nests exactly where your parser looks:
  response.results["gs://..."].inlineResult.transcript.results[].alternatives[0].words[],
  with durations as strings like "1.560s".
• totalBilledDuration came back as "65s", matching the audio.
• Both operations finished in about 12 seconds.

One bug: the first word of every result is being dropped

proto3 JSON omits zero-valued fields, and the REST API follows that, so a word starting at
0.000s arrives with no startOffset field at all:

{ "endOffset": "1.560s", "word": "Ay" }
{ "startOffset": "1.560s", "endOffset": "1.800s", "word": "vay" }

ParseResponse reads the absence as -1, and the range check then discards it as an
impossible timing. In my 160 word response exactly one word had no startOffset, the
first one, and it was dropped in both runs. The segment then starts at the second word's
offset instead of at zero, so the text keeps the word but the timing loses it.

Fixed in #14567 by treating an absent startOffset as 0. An absent endOffset stays
invalid, since a word ending at zero is meaningless.

Also worth knowing: resultEndOffset was not present in this response at all, so the
fallback that uses it will not always have a value.

The audio format is worth changing

Same source audio, same request, only the encoding differs:

	32 kbit/s mp3 (current)	lossless flac
Words returned	159	160
Word agreement	85.6%
Words differing	23 of 160

A sample of what actually changed:

'Amca oğlum. Deniz gözlüm.'  ->  'Amcaoğlum seni izledim.'
'vardır. Şu ceketini'        ->  'var da şu kravatını'
'çıkart'                     ->  'düzelteyim dur'
'gözünün'                    ->  'ağzının'
'Gitmeyelim'                 ->  'Hiç gitmeyelim'

I do not have ground truth for this clip, so I am not claiming flac is 14% more accurate.
What I can say is that the encoding alone changes one word in seven, including whole
content words like jacket versus tie and eye versus mouth. On 65 seconds. That is the
compression talking, and it is being applied for a reason that does not hold here, since
the audio goes to a bucket rather than through a 25 MB request body.

Measured cost of the change: an 18 minute chunk is 20.7 MB as flac against 4.1 MB as mp3.

All of this is in #14567 along with the dynamic batching option and the truncation
recovery. 283 SpeechToText tests pass.

[muaz978]

One more from the live run, and I think this one is more serious than the rest, so flagging
it separately.

Chirp returns a whole file as a single result. Measured just now against the live API:

Audio	Results returned	Words in it
65 seconds	1	160
3 minutes	1	322

ParseResponse emits one segment per result, and IngestTranscriptionResponse creates one
Paragraph per segment and never looks at segment.Words. So on an 18 minute chunk the
engine currently produces one subtitle line spanning the entire chunk.

The word timings are fetched, parsed, range checked, and then dropped on the floor at
ingestion, which is a shame because they are the whole reason for choosing this API.

Fixed in #14567 by cutting segments from the word timings with
OpenAiSttService.BuildSegmentsFromWords, the same helper the OpenAI and OpenRouter paths
already use, so cues break on real pauses and the cue shape stays consistent across engines.
A result with no usable words still falls back to resultEndOffset exactly as before.

That change made four existing assertions fail, including one of yours, because they
asserted the one-segment-per-result shape. I updated them rather than working around the
behaviour, and noted why in each. 284 SpeechToText tests pass.

For context on what the output should look like: on a 145 minute episode the same word
timings produce roughly 3,000 cues averaging about 1.2 seconds, with 313 pauses longer than
2 seconds preserved.

[muaz978]

Closing the loop on the live testing you asked for: I ran the engine end to end, not just
the parser.

The full round trip works

GoogleCloudSttService.TranscribeAsync against the live API on 3 minutes of Turkish audio:
credential load, bucket check, upload to Cloud Storage, batchRecognize, polling, parse,
cue building. 30 seconds wall clock. The tools log from that run:

progress: Uploading audio to Cloud Storage...
progress: Submitting transcription...
progress: Waiting for transcription to complete...
Google Cloud: dropped word 'Sağ' with impossible timing 162.76-160.28 s

That last line is your range check earning its keep on live data. Google returned
startOffset 162.760s, endOffset 160.320s for that word, ending 2.4 seconds before it
starts. One word in 322, which matches the 0.8% rate I saw across a full episode.

Two things worth knowing

Chirp is deterministic. Same audio, same config, two separate batchRecognize calls:
322 words each, 100% identical. So results are reproducible run to run, which is not
something I would have assumed.

The language setting materially changes the transcription. Same audio, same everything
else, only languageCodes differing:

Setting	Word agreement against the other
["auto"] versus ["tr-TR"]	92.2%

Real differences, not punctuation: Amcaoğlum. Seni çok özledim. under auto against
Amca oğlum seni özledim. under tr-TR. I have no ground truth to say which is better,
and auto correctly identified Turkish, so your default looks sound. But it is worth
knowing that a user who sets a language hint will get a different transcript from one who
leaves it empty, and neither is obviously wrong.

One caveat, stated plainly

I could not use a service account key. My account on that project has speech.client and
storage.objectAdmin but no rights to create service accounts, and the project carries a
disableServiceAccountKeyCreation org policy. I ran it with my own user credential
instead, which GoogleCredential.FromFileAsync accepts, after adding the project_id
field that ReadProjectId looks for.

So everything from the credential onwards is genuinely exercised: token exchange, bucket
probe, upload, submit, poll, parse, cue building, object cleanup. The one line I have not
run is FromFileAsync on a service_account type file, which is the library's primary
path. I have asked our cloud admin for a proper key and will report back if anything
surprises me, but I did not want to hold this report for it.

Everything above is with #14567 applied. Without it the same run produces one subtitle line
for the whole chunk, so I could not have measured any of this.