Standalone Faster-Whisper-XXL features #231
Replies: 17 comments 67 replies
|
any hope of doing something similar for mac in the future? |
|
Probably there won't be macOS builds for this. |
|
I guess no chance of a python package either? |
Yeap, it's out of the scope of this repo. |
|
Sorry, saw a comment here earlier about 'why post' this here but now its gone -- I was trying to keep it in the thread of this 'XXL' version. For example, if there can be an XXL version why not as if there will be a '-MAC' version in the future... who knows - just asking :) Separately, for others who may find this thread and dismayed at no mac support there is a lot of work being done with 'whisperkit' which also does great work. https://github.com/argmaxinc/WhisperKit The future of high-quality transcriptions is very bright. |
|
WhisperKit has no plans to add VAD though. I plan to do it using a flutter port I found of silero |
|
A little annoyance with I'm running Faster-Whisper-XXL in a Nextcloud folder (with a cronjob checking if new audio files have been synchronized, then running faster-whisper-xxl). So far, this worked fine, but in r192.3.3 with MDX filtering enabled, it seems first the *_mdx.wav file is created and then it's moved to a temp folder (?). This move fails because Nextcloud already tries to sync the mdx file, and this leads to whisper-faster-xxl just quitting with an error that the *_mdx.wav file is already in use. I now set the Nextcloud rules to just ignore *_mdx.wav files, but would it be possible to create them in a temp folder from the start? |
It's intermediate file, first it's written to the same folder as the original input file then it's deleted. No "movements" of it.
Should be possible. |
|
I'm confused: If the mdx file is deleted before transcription starts, then what file is whisper running on? |
The intermediate file is loaded into variables further and it's not needed in the physical form anymore.
VAD's "intermediate" is created only in memory. |
|
Ah, I get it, thanks for clarifying and for implementing the patch. :) |
|
Implemented in r192.3.4 |
|
Do I need to use some kind of tag to make the recognition against a little noise or soft music better? |
|
You're right, I thought the memory error was at the point of vocal isolation. |
How can I make the memory be used from the CPU as before? |
|
Use |
Try using CUDA with int8: for Russian: -prompt default
or -prompt None
or -prompt auto
|
It's taking me a long time. I've never been able to wait. |
|
Such a great tool. Especially for those who aren't very saavy in Python or command line! Thanks for creating! Is it possible to perform speaker diarization with this standalone version? |
|
I would throw in $50 |
|
You can count on $20 from me as well. |
|
A new thread would be better place for this -> #281
If you don't trust me with your email "address" then how you can trust me running the program?! 😉 |
|
I'll do the rest of it to make the 100£ ;) Let's continue on the other thread. #281 |
|
@dgoryeo - only 10£ will be needed in the end |
|
Hey @Purfview , I was wondering if you have (or willing to run) any benchmarks that compare |
|
Sorry for late answer, probably you are aware now that silero_v5 is not good, worst from all Silero versions. |
|
In my testing with many audio (english or japanese) samples, silero v5 is the worst one. My final test result, [--vad_method default (silero_v4_fw)], gives better results. |
|
@mjamil85 , did you get to test silero_v3 as well? |
|
Already test, not good either. |
|
Hi @Purfview, I did a test with the --ff_mdx_kim2 feature and it took a long time to complete, about 45min for a 10min video. Is the voice extraction feature processed using the GPU, or CPU? |
|
If you have an NVIDIA graphics card with CUDA support and you didn't specify --mdx_device cpu - CUDA should be used. On my NVIDIA 3060 12GB, processing ff_mdx_kim2 for a 90 minute video takes no more than 3-5 minutes. Please note ff_mdx_kim2 is quite demanding on video memory, my video memory usage is over 8GB during ff_mdx_kim2. |
|
Yep, I forgot about it, my bad. |
That's too slow for GPU. Try
How much VRAM?
It can use both. |
|
Is there a series of parameters that work best to capture very short audio clips? My clips with just "Yes" or "Let's go" produce a blank transcription. I've adjusted --vad_min_speech_duration_ms and others, but nothing catches these short clips. |
|
Which vad method do you use? I believe Auditok would perform best for your scenario. |
|
I fixed this with --vad_filter=false Now it perfectly transcribes every file down to about 1 or 2 seconds long. Thank you! |
|
Is there any way to make auto dialogs to work? instead of Thanks! |
|
Since this faster Whisper model has been modified from the original version, could you please upload the source code so the community can contribute and add new features or im i missing something? Thanks! |
|
@Purfview How are you compiling the python project into an .exe? |
|
@Purfview Totally a fair ask, i think, im not looking to contribute directly, i think just it d be helpful to see the source since this standalone has cool tweaks like VAD and --sentence for movies, wich is not implemented in the faster whisper or whisper-ctranslate2 repo. Having it open will make it easier for us to understand and maybe build on it. |
https://github.com/pyinstaller/pyinstaller
If you want something implemented somewhere you go post there in corresponding repos. And leave programming things to the programmers they know what to do and where to look for sources, no need to fantasize about things. |
|
@Purfview |
I end up logging my query and output, and if i run it several times, the log shows all the runs. Maybe a --logging option but .log not .txt or lyrics will get overwritten |
|
Are |
|
Hi, first I want to thank you for the great solution. Makes the life much easier. Maybe I am missing out one of the myriads of call arguments. Is there any option to keep the python backend running between invocations to shorten startup times? I want to use the solution for a voice chat bot and short turnaround times are key here. |
|
Currently such feature is not implemented. |
|
That would be a great feature - I am willing to donate for that. |
|
I looked at it, it's possible to do, but as much as I love to do feature creep , such implementation I think is way out of scope for this project. |
|
I’ve been thinking about this issue during my music-collection-transcription-project (which has grown to 6000+ lines of script & code outside of WhisperXXL) ... I’ve cranked out 7,200 SRT files now.
Really, just need a separate agent that keeps the model in memory which is called upon by the transcriber.
The agent would just take the filename and/or type/alias of the model and store it in a way that can be called via the same filename and/or type/alias. So we could ```load_model_into_memory.exe whisper``` or ```load_model_into_memory.exe c:\whatever\model_folder```
Which I'm sure is much easier said than done.
But at this point 50% of my energy bill is going to reloading the model 500+ times a day, I hit a lifetime-record electric bill of $620 this month, and I’ve only done 10% of my collection, with ≥50% to go. We’re talking 3–5 more months of hugely-increased electric bills and the environmental impact that comes with it.
I’ve had people call me a monster for not pausing my music when I leave the house. It was quite over-the-top, but my point is some people care about this more than others and for those people it’s a way to mitigate the ethical concerns of the environmental impact of AI
My January electric bill is usually $370-$470 and this January it was $620, in part due to the extra-cold January, and us setting the heat a bit higher now that we’re older ....... but no doubt the lifetime record was partially achieved due to me running my GPU non-stop and I expect to see a huge bill at the end of February too.
And I have months to go. I’ve gotten my transcription compliance for my 60,000-song collection from an initial starting point of 29% now up to 45%. But that’s not even halfway through... I’ve done 14% myself with the 7,200 SRTs i’ve generated.
All of this would finish so much faster without that model load time. It would halve the the impact. And the cost. We’re talking several hundred dollars being thrown out the window to further warm our planet. Nobody has an agent like this and I’m sure it would be rapidly picked up by other coders as a way of speeding up workflows and mitigating some of the ethical concerns with AI usage. I almost feel like it would be award-worthy for introducing a concept that, if caught on, could greatly mitigate various ethical and environmental concerns. Concerns i don’t car about too much personally... I care about my electric bill, llol
-𝓒𝓵𝓪𝓲𝓻𝓮
p.s. [alas, batch_dir is not useful for me as I have a lot of per-file processing that occurs outside of whisper, to the point of using 6 different alternative data stream tags to manage my music files and their status while passing through this workflow]
…On Fri, Feb 14, 2025 at 4:48 AM Purfview ***@***.***> wrote:
Currently such feature is not implemented.
I'm not sure how such would work, I guess subsequent commands could be
passed with pipe.
—
Reply to this email directly, view it on GitHub
<#231 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAGAYVYVJAWLSUIXKZMOS432PW3WJAVCNFSM6AAAAABF2RK6GKVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEMJZHA3DKNI>
.
You are receiving this because you are subscribed to this thread.Message
ID:
<Purfview/whisper-standalone-win/repo-discussions/231/comments/12198655@
github.com>
|
|
I'm just amazed by the sheer size of your (vintage) music collection, @ClaireCJS. Hats off! |
It’s actually mostly post-1975 punk rock, thrash metal, industrial metal, 80’s pop, cartoon & movie soundtracks & chiptune covers/cover albums. I think the vintage is only about 4,000 songs 🤣
|
|
Impressive! May I ask what parameter settings/values do you use for transcribing songs? Would vocal extraxtion be needed? |
Hey! What I ended up with was: 👂 20250205151625: prompt v17: Faster-Whisper-XXL.exe --model=large-v2 --language=en --output_dir "C:\mp3\They Might Be Giants\2015 - Why_" --output_format srt --vad_filter True --max_line_count 1 --max_line_width 20 --ff_mdx_kim2 --highlight_words False --beep_off --check_files --sentence --verbose True --vad_filter=True --vad_threshold=0.1 --vad_min_speech_duration_ms=150 --vad_min_silence_duration_ms=200 --vad_max_speech_duration_s 5 --vad_speech_pad_ms=198 --vad_dump --best_of 5 --max_comma_cent 70 --max_gap 2.0 --initial_prompt "[Robin Goldwasser] Don't accidentally do things you'll later regret. Oh, no, you did. Some things can't be taken back. Don't do them. Too late, you did. Don't say these words to your dad. That he's gross or that he smells bad. Don't spill ink on the rug. Don't hammer the wall. Why won't you take directions. Or listen to instructions. Now you're informed. And you've been warned. So don't. Oh, wait, you did. [John and John] Sorry, sorry. Don't be angry. I didn't mean to do it. I swear I'll never do that again. Don't. Oh, you did. Really, you should not go anywhere high up and. Like the Pyramids of Gizza. Or the Leaning Tower of Pizza. You know what. Too late you did. How did the carton of milk get filled with shaved off puppy hairs? What's that river of water doing pouring down the stairs? Did you just eat a napkin. To find out what would hapkin? I thought I told you not to but you had to and you did. Blah blah blah da blee. Blah blee blah blah. Blah blah blah blee blah. Blah blah blah da blah. Blah blah blee blah. Blah blah blee blah blah. Hmm hmm hmm hmm hmm hmm hmm. Blah blah blah blah. Blah blah blah blah da blah. Blah blah blah blah blah." "c:\mp3\They Might Be Giants\2015 - Why_\01_Oh You Did (feat Robin Goldwasser).mp3" Part of the system is downloading lyrics in order to set up a prompt for more accurate transcription. In fact I’d say that’s 75% of the work. My documentation file mainly for my own use: Unfortunately it’s mostly written in BAT for an obscure command line. I have my reasons, including self-challenge and that the system will remain intact longer. I’ll probably turn it into a virtual machine someday for kicks. The periods in the lyric prompting are VERY important for proper segmentation. I then remove them afterward with a postprocessor. |
|
Great approach. FWIW, and may be an idea, your approach reminds me this work which is for a different use case, but it impleneted a GUI on top of the core: https://github.com/machinewrapped/gpt-subtrans/wiki/GUI#gui-subtrans Although this repo is for translation, it uses a systemic way to propmpt each defined section with a relevant input. The GUI on top, makes it very engaging. |
|
This is probably a remarkable naive question, but I can't find any method to put the command line options into a text file and run like this: |
|
What you are looking for is a BAT file. But you’ll have a way to go before it gets super useful. Pick a command line... If you have $100 or a pirate hat for TCC it’s worth it even if hardly anyone uses it. I’d avoid power shell. Probably just stick to CMD.EXE as lame as it is, for most compatibility. Ask ChatGPT to help you write BAT files to do what you want. Look up the commands you don’t understand. Bootstrap that knowledge. You’ll get something useful for way less effort in the end. I’m using ... a couple 2500 line ones to do this at this point as I’ve been adding to them for ~6 months Basically: save what you have as "runme.bat" in the same folder You can go further tho and make one to do all the files in a folder: you may need to fill in the full location to that EXE tho... |
|
Thank you, @ClaireCJS. Another seemingly obvious question. I am using "--without_timestamps true" but I'm still getting output with timestamps. Maybe "true" is the wrong parameter. |
Maybe use |
|
Still getting timestamps with "false" |
|
I found it: SYSTRAN/faster-whisper#509 |
|
Hello |
|
Hey @Purfview - first just wanted to say thank you for the brilliant tool. I am running into an issue trying to run this (R245.4) on my 5080 laptop and encountering the following error: RuntimeError: cuBLAS failed with status CUBLAS_STATUS_NOT_SUPPORTED It seems the build may not yet support Blackwell architecture? Could you advise if a Blackwell-compatible XXL build is planned or if there’s a workaround (e.g., recompiling)? Again, thank you for your time and the amazing work! |
|
add -ct float16 |
|
This worked, thank you. |
Yes. Blackwell is not supported by r245.4. But it's already supported in Pro r3.256.1
The issue is already fixed at its core [not in any release yet]: OpenNMT/CTranslate2#1982 |
and others
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
EDIT1: Don't post your questions here, it's already littered with random posts.
Includes all Standalone Faster-Whisper features +the additional ones mentioned below.
Includes all needed libs.
Vocal extraction model:
--ff_mdx_kim2: Preprocess audio with MDX23 Kim vocal v2 model (thanks to Kimberley Jensen). [Better than HT Demucs v4 FT]Alternative VAD (Voice activity detection) methods:
--vad_methodchoices:silero_v3- Generally less accurate than v4, but doesn't have some quirks of v4.silero_v4- Same assilero_v4_fw. Runs original Silero's code instead of adapted one.silero_v5- Same assilero_v5_fw. Runs original Silero's code instead of adapted one.silero_v4_fw- Default model. Most accurate Silero version, has some non-fatal quirks.silero_v5_fw- Bad accuracy. Not a VAD, it's Random Detector of Some Speech :), has various fatal quirks. Avoid!pyannote_v3- The best accuracy, supports CUDA.pyannote_onnx_v3- Lite version ofpyannote_v3. Similar accuracy to Silero v4, maybe a bit better, supports CUDA.webrtc- Low accuracy, outdated VAD. Takes only 'vad_min_speech_duration_ms' & 'vad_speech_pad_ms'.auditok- Actually it's not VAD, it's AAD - Audio Activity Detection.Speaker Diarization:
--diarizechoices:pyannote_v3.0- Fastest for CPUpyannote_v3.1- Same as v3.0 but should be faster with CUDAreverb_v1- Allegedly better than pyannote v3reverb_v2- The slowest, allegedly the bestFor more read and post there -> Speaker Diarization
Legal notice: Reverb models are only for personal non-profit use.
Latest CTranslate2:
Up to ~26% faster on CPU with the int8 quantizations.
Flash attention support, that's CUDA, but the benchmarks shows no effect on the performance.
All reactions