whisper

[Manubi]

Hi,
is it possible to do the use case microphone -> whisper -> text via streaming?
meaning can i record voiced via microphone and get the real time stream of the text back?
thanks!

[satoshi02198]

I implemented whisper on this template.
using 'react-audio-voice-recorder' https://www.npmjs.com/package/react-audio-voice-recorder

const sentToChatGPT = async (audioBlob: Blob) => {
    const formData = new FormData();
    formData.append('file', audioBlob, 'audio.webm');
    formData.append('model', 'whisper-1');
    formData.append('token', previewToken);
    formData.append('language', inputLanguage);

    try {
      const res = await fetch('/api/whisper', {
        method: 'POST',
        body: formData,
      });

      const { text, error } = await res.json();
      if (text) {
        const args: Message = { id: id as string, content: text, role: 'user' };
        await append(args);
        if (pathname === '/') {
          router.push(`/chat/${id}`);
        }
      } else {
        console.error('Error from whisper response', error);
      }
    } catch (error: any) {
      console.error('In voice-recorder component', error.message);
    }
  };

I call this function on onRecordingComplete prop

 <AudioRecorder
        recorderControls={recorderControls}
        onRecordingComplete={(audioBlob: Blob) => sentToChatGPT(audioBlob)}
      />
   ```

[Manubi]

Does the streaming work with vercel edge functions? meaning will it be fine to translate a 30min+ audiostream? thanks!

[satoshi02198]

I don't use edge, I use nodejs for runtime because I get error when deployment. and I don't know whisper allow 30min+ audiostream as I don't upload that much size. But In OpenAi API doc, written this 'File uploads are currently limited to 25 MB and the following input file types are supported: mp3, mp4, mpeg, mpga, m4a, wav, and webm.' hope It helps you.
https://platform.openai.com/docs/guides/speech-to-text