anyway to force language in case of wrong whisper autodetection?

[ndx1905-github]

Hi

Thanks for the great work and very useful tool (I'm amazed that it works on my 2018 NAS)

Faster whisper sometimes picks the wrong language..
Is there a way to force the language?

Here is an example, it's French audio wrongly detected as English (score of .56)

I should pass the --language fr to faster-whisper. One way could probably be to add "FR" somewhere at the end of the file name right before the file extension?
openai/whisper#529

[McCloudS]

There is a way, but I haven’t exposed it. I can look at that. Whisper only uses the first 30 seconds to detect audio. So if there is little audio or it’s mixed with English, it can guess wrong.

…

On Thu, Feb 8, 2024 at 2:51 AM ndx1905-github ***@***.***> wrote: Hi Thanks for the great work and very useful tool (I'm amazed that it works on my 2018 NAS) Faster whisper sometimes picks the wrong language.. Is there a way to force the language? Here is an example, it's French audio wrongly detected as English (score of .56) image.png (view on web) <https://github.com/McCloudS/subgen/assets/64833823/d72e3b1d-010c-4529-962c-4be5b97a4acd> — Reply to this email directly, view it on GitHub <#47>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APJACQKEPES3KCPUOMMSXI3YSSN2FAVCNFSM6AAAAABC7NLD56VHI2DSMVQWIX3LMV43ASLTON2WKOZSGEZDINZUHA3DANA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[McCloudS]

Take a look at the readme @ FORCE_DETECTED_LANGUAGE_TO and try to re-pull the image. Let me know if it works!

[ndx1905-github]

Hey.. it didn't change anything. Here is the screenshot after adding the FORCE_DETECTED_LANGUAGE_TO variable and rebooting the docker.

However... I watched the video and subtitles still worked. I'm not sure what this "detected language" notification does in practice since the output was correct. Log says "English" detected, but in practice I still have the right "French to English" srt that is generated.

(EDIT: to be more precise, subtitles were correct even before the patch, and despite the wrong language detection)

[McCloudS]

Good to know that even with the misdetected language, it produces something passable. Did you actually pull the new docker image instead of just restarting it?

[ndx1905-github]

Yes I redownloaded the image. Just did it again, same result. Still detects English.
But does the --language option change the language that is detected, or does it still detect English, writes that in the console and then the --language option forces faster-whisper to use FR even though EN was detected ?

[McCloudS]

It's passed the way that stable-ts and faster-whisper document. And per https://github.com/SYSTRAN/faster-whisper/blob/f144e4c83d54f3c3304b6a75a3f563e5f84de6cf/faster_whisper/transcribe.py#L344 it shows that if a language is manually defined it shouldn't try to detect the language unless it's an English only model. Any chance you're using the .en models? Like medium.en instead of medium.

[ndx1905-github]

yes I'm sure I've selected medium, not medium.en
Also medium.en would probably produce garbage? but my output is still good

[McCloudS]

I'm a little puzzled as to why it's ignoring the language. I'll play with it later if I have time, but it's comforting to know that it may not actually matter?

[McCloudS]

This is now fixed and I confirmed it will show as the correct forced language. You can also navigate to http://subgenip:8090/docs in a browser and use the Batch option.