Skip to content

Accept OpenRouter transcription models that reject verbose_json - #14542

Merged
niksedk merged 1 commit into
SubtitleEdit:mainfrom
muaz978:fix/openrouter-stt-non-gpt-verbose-json-rejection
Sep 5, 2026
Merged

Accept OpenRouter transcription models that reject verbose_json#14542
niksedk merged 1 commit into
SubtitleEdit:mainfrom
muaz978:fix/openrouter-stt-non-gpt-verbose-json-rejection

Conversation

@muaz978

@muaz978 muaz978 commented Sep 5, 2026

Copy link
Copy Markdown

Subtitle Edit already talks to OpenRouter's transcription endpoint, but the request it
sends today only works for OpenAI's own models. OpenRouter also hosts Google's Chirp
models on that same endpoint, and they reject the current request in three separate
ways. None of the three can be predicted from the model name, and none of them produce
a usable error message.

This PR makes those models work. It adds no engine, no new provider, and no setting.

Note

Scope: 4 files, one behavior change (accept Chirp). Nothing is removed, and no
existing model's behavior changes.

The three rejections

# What Chirp rejects How it surfaces Fix in this PR
1 verbose_json 400, sometimes with the provider's own message, sometimes an opaque Provider returned 400 One automatic retry with plain json
2 mp3 audio The same opaque 400 Extract wav for these models up front
3 Audio longer than ~60 s The same opaque 400 again Smaller, duration aware chunks

1. verbose_json is refused

IsJsonOnlyModel only recognizes OpenAI's own gpt-*-transcribe name shape. Chirp needs
the same treatment but does not match that shape, and the failure cannot be recognized
from the error text either, because OpenRouter sometimes forwards the provider's specific
complaint and sometimes wraps it as an opaque Provider returned 400.

So rather than pattern matching on error strings, any 400 on the first verbose_json
attempt now gets exactly one retry with plain json. Any other status, or a model already
known to be json only, does not retry.

Chirp returns no word timings in either format. That is a limitation of this model over
OpenRouter rather than something this PR can fix, and the existing chunk pipeline already
handles the text only case by spanning each chunk's duration and splitting into sentences.

2. mp3 is refused

No request level retry can fix this one, because the audio was already encoded before the
service ever saw it. The caller now extracts wav for these models instead.

Confirmed by sending byte identical audio as mp3 and as wav: only wav succeeds.

3. There is a hard ~60 second cap

It is independent of file size, and it looks like the synchronous Google Speech-to-Text
API underneath. Confirmed empirically: identical audio succeeds at 60 s, then fails at
65 s and at 180 s with the same opaque 400, regardless of language or prompt.

Why the chunk size is 1 MB and not 1.4 MB

Chunk sizing has to survive silence snapping. ComputeAdjustedBoundaries moves each cut
by up to maxOffsetSeconds (10 s by default) to land on a silence, and a chunk sits
between two independently snapped cuts. So a chunk can run up to 20 s past its target,
not 10 s.

At 16 kHz mono 16 bit PCM, roughly 32 KB/s:

Chunk size Targets Worst case Under the ~60 s cap?
1.4 MB ~45 s ~65 s No
1 MB ~33 s ~53 s Yes

Testing

24 unit tests ship with this PR. Beyond those, this was exercised against the live
OpenRouter endpoint across roughly 40 hours of transcription runs on Turkish TV drama:

Observed Count
verbose_json rejections, each one triggering the new fallback 211
Automatic plain json retries fired 203
Hard failures before the fix 17
Opaque Provider returned 400 responses, where the mp3 rejection and the duration cap are indistinguishable from one another 12

The mp3 versus wav result, and the 60 s versus 65 s versus 180 s result, each come from
paired runs on byte identical source audio.


A question for you: would you want the timestamped path in core?

Worth stating plainly: even with this fix, Chirp over OpenRouter returns text only.
Cue times are then inferred by splitting text proportionally to character count, which
cannot represent silence.

Important

In the OpenRouter output I measured, zero gaps longer than 2 seconds appear
anywhere, the largest gap in the entire file is 0.85 s, and 35 of 50 consecutive cue
boundaries touch at exactly 0.001 s, and the first Speech-to-Text v2 cue on the same
source does not arrive until 110.8 s because the opening theme is genuinely empty.
That is the signature of arithmetic subdivision
rather than of measured speech.

The same model, addressed directly through Google's Speech-to-Text v2 API, does return
word level timings. Measured on a 145 minute episode:

Metric OpenRouter, text only Speech-to-Text v2
Cue timing source character count measured word offsets
Pauses longer than 2 s 0 334
Largest gap 0.85 s 138.4 s, correctly empty across the opening theme
Words carrying real timings none 13,175 across 2,978 cues
Speech density 97.4% 54.3%, which is what TV drama actually looks like

To be straight about the comparison: the OpenRouter figures are measured over the sample
I have from it rather than a full episode, so read them as the shape of the timing rather
than as a coverage result. That shape does not improve with length, because character
proportional timing cannot represent silence at any duration.

On the documentation: Google's chirp_3 page lists "Word-level timestamps" under a
heading of features the model does not support, yet the same page's own row text, its
BatchRecognize duration note, and its shipped code sample all describe word timestamps
as something you enable. The table row looks mis-filed rather than the behavior being
undocumented. Either way, I received word timings on every run.

I would rather ask than assume: would you be open to that path living in Subtitle Edit
itself?
It is a materially more accurate transcription engine, but it is not free, and
the cost sits mostly in onboarding rather than in code. Honestly, here is everything it
needs:

Requirement What it involves
Authentication OAuth2 service account JSON. API keys are refused outright: 401, API keys are not supported by this API. Expected OAuth2 access token. Every existing online engine in SE uses a single API key textbox, so this needs a JSON file picker and a token exchange. This is the biggest change, and it is a product decision as much as a code one.
Endpoint The regional endpoint us-speech.googleapis.com, with recognizer path projects/<id>/locations/us/recognizers/_. The global endpoint does not serve chirp_3.
Dependency Google.Cloud.Speech.V2 pulls in gRPC and protobuf, which noticeably increases bundle size. A hand rolled REST client against the v2 API avoids that, at the cost of more code to maintain. Your call which you would prefer.
Transport, sync Recognize maps 1:1 onto the existing ISttTranscriber.TranscribeAsync: inline audio, no bucket, word timings returned directly. It carries the same ~60 s cap as above, so it needs duration aware chunking, which this PR already builds.
Transport, batch Handles hours long audio (a 140 minute episode in 8 chunks), but requires a user supplied Cloud Storage bucket, uses an async job model that does not fit the current engine contract, accepts at most 5 files per request, and returns inline results only for single file requests.
Reliability guards Non negotiable for correctness. I hit silent truncation, where one 18 minute chunk stopped transcribing 6.6 minutes in and discarded the remaining 11.4 minutes while reporting success with no error at all. Coverage has to be validated against expected duration, with short chunks resubmitted. I also saw corrupt word offsets on 79 of 9,432 words (0.8%), one of them claiming 6,324 s inside a 1,080 s chunk, so offsets need per chunk range checking.
Cost $0.016 per minute at standard rate, or $0.003 per minute with dynamic batching enabled, roughly an 81% discount. A 2.5 hour episode is $2.32 or $0.44 respectively.

If that is more than you want to carry in core, particularly the service account auth and
the Cloud Storage dependency, then I completely understand, and my company will ship it as
a separate SE5 plugin instead, so the users who want it can opt in without everyone paying
for it. I just did not want to make that decision on your behalf.

Either way, this PR is only the OpenRouter fix, and it stands on its own.


One more small thing, whichever way you answer

There is a related snag I ran into. RunPlugin returns early when the subtitle is empty:

if (IsEmpty)
{
    ShowSubtitleNotLoadedMessage();
    return;
}

That is exactly the right guard for a plugin that transforms existing lines, which is
what plugins do today. But it makes a transcription plugin impossible to start from the
state users are actually in, which is a video open and no subtitle yet. As it stands they
would have to type a placeholder line first, then run the plugin, then delete it.

Would you accept a small opt in for this? Something like a manifest field
"allowsEmptySubtitle": true, which a plugin sets when it produces a subtitle rather
than transforming one, leaving the current behavior untouched for every existing plugin.
Happy to send it as its own focused PR if you like the idea.

OpenRouter hosts transcription models from providers other than OpenAI,
and Google's Chirp models reject the request Subtitle Edit sends today in
three separate ways. None of them is discoverable from the model name.

- verbose_json is rejected. IsJsonOnlyModel only recognizes OpenAI's own
  gpt-*-transcribe name shape, and the rejection cannot be recognized from
  the error text either: OpenRouter sometimes forwards the provider's
  specific complaint and sometimes wraps it as an opaque "Provider
  returned 400". So any 400 on the first verbose_json attempt now gets one
  automatic retry with plain json before failing. Chirp returns no word
  timings, and the existing chunk pipeline already spans each chunk's
  duration and splits into sentences when only text comes back.

- mp3 is rejected outright, with that same opaque 400. No request-level
  retry can fix it because the audio was already encoded before the
  service saw it, so the caller now extracts wav for these models.

- The synchronous Google Speech-to-Text API underneath has a hard ~60
  second audio cap, independent of file size. Confirmed empirically:
  identical audio succeeds at 60s and fails at 65s and 180s. Chunk sizing
  has to survive silence snapping, which can move each of a chunk's two
  cuts by up to 10s, so a chunk can run 20s past its target - 1 MB of
  16 kHz mono PCM targets ~33s with a ~53s worst case.
@muaz978
muaz978 force-pushed the fix/openrouter-stt-non-gpt-verbose-json-rejection branch from dc3e07d to 73c56ac Compare September 5, 2026 03:24
@niksedk
niksedk merged commit 2d017ab into SubtitleEdit:main Sep 5, 2026
1 check passed
@muaz978

muaz978 commented Sep 5, 2026

Copy link
Copy Markdown
Author

Would you mind dear @niksedk answering the 2 important questions as they are critical and both are on you to decide? They are ready with already tested samples whenever you are ok with them

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants