Making whisper.cpp use streamed audio from stdin

[luarvique]

Hello!

I am wondering if there are existing command line options to make whisper.cpp accept raw streamed audio from the stdin. I know that a .wav file can be fed to it, but .wav files are inherently finite.

If there are no such options at the moment, could someone add them? Would essentially come down to specifying sample rate and the data format (e.g. S16 or F32).

[KitaitiMakoto]

Currently it seems supported by specifying "-" as input file though I don't try it:
#3050

[luarvique]

I have tried it, and it does let the user stream a .WAV file via stdin, but whisper.cpp still requires that audio to be in the .WAV format. I am currently looking for a way to feed it in raw format, as it is coming from an audio source.

[KitaitiMakoto]

Ah, I see. Maybe we need wait for someone to add the feature...

If Ruby code is acceptable (because I'm a Rubyist), we can do it in Ruby:

# transcribe-from-stdin.rb

require "whisper"

# Suppress logs by no-operation callback
Whisper.log_set proc {}, nil

def main
  input = $stdin.read
  samples = input.unpack("e*")

  whisper = Whisper::Context.new("base.en")
  # Transcribe
  whisper.full(Whisper::Params.new, samples)
  puts whisper.to_webvtt
end

main

% sox ./samples/jfk.wav -t f32 -r 16000 -c 1 ./jfk.raw
% cat jfk.raw | ruby ./transcribe-from-stdin.rb
WEBVTT

1
00:00:00.000 --> 00:00:11.000
 And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country.

Ruby library is available from bindings/ruby directory (you need build and install it yourself) or gem install whispercpp.

[KitaitiMakoto]

It might help to use whisper-stream command instead of whisper-cli:

% ./build/bin/whisper-stream -m ./models/ggml-base.en.bin -t 8 --step 500 --length 5000