Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Browser audio recording is not supported #147

Open
manosec opened this issue Dec 5, 2023 · 2 comments
Open

Browser audio recording is not supported #147

manosec opened this issue Dec 5, 2023 · 2 comments

Comments

@manosec
Copy link

manosec commented Dec 5, 2023

Hi there,

I was working with react-mic to stream the audio buffers to backend socket event handlers in NodeJS, my goal is to transcribe the audio to text.

I'm able to record the audio and stream it to the socket events but when I deploy the app, I'm unable to record the audio. A pop-up says browser does not support audio recording. I have attached the reference screenshots below.

  1. When deployed in the server
    audio_notsupport

  2. In my local machine
    audio_succ

@manosec
Copy link
Author

manosec commented Dec 5, 2023

I have attached my react component code here,

import { ReactMic } from 'react-mic';
import React, { useState, useEffect, useRef } from 'react';
import io from 'socket.io-client';


const Example = () => {
  const [record, setRecord] = useState(false);
  const socketRef = useRef(null);
  const [result, setResult] = useState("")

  useEffect(() => {
    
    return () => {
      if (socketRef.current) {
        socketRef.current.disconnect();
      }
    };
  }, []);

  const downsampleBuffer = function (buffer, sampleRate, outSampleRate) {
    if (outSampleRate == sampleRate) {
      return buffer;
    }
    if (outSampleRate > sampleRate) {
      throw 'downsampling rate show be smaller than original sample rate';
    }
    var sampleRateRatio = sampleRate / outSampleRate;
    var newLength = Math.round(buffer.length / sampleRateRatio);
    var result = new Int16Array(newLength);
    var offsetResult = 0;
    var offsetBuffer = 0;
    while (offsetResult < result.length) {
      var nextOffsetBuffer = Math.round((offsetResult + 1) * sampleRateRatio);
      var accum = 0,
        count = 0;
      for (var i = offsetBuffer; i < nextOffsetBuffer && i < buffer.length; i++) {
        accum += buffer[i];
        count++;
      }
  
      result[offsetResult] = Math.min(1, accum / count) * 0x7fff;
      offsetResult++;
      offsetBuffer = nextOffsetBuffer;
    }
    return result.buffer;
  };
  

  const startRecording = () => {
    setRecord(true);
    const socket = io('http://localhost:3000');
    socket.on('connect', () => {
      console.log('Socket connected:', socket.connected);
      socketRef.current = socket;
    });

    socket.on("result", data=>{
      setResult(data)
    })
  };

  const stopRecording = () => {
    setRecord(false);
    if (socketRef.current) {
      socketRef.current.disconnect();
    }
  };

  const onData = async (recordedBlob) => {
    if (socketRef.current && socketRef.current.connected) {
      // const data = await downsampleBuffer(recordedBlob, 44100, 16000)
      // console.log(data)
      // console.log(recordedBlob)
      socketRef.current.emit('binaryData', recordedBlob);
    } else {
      console.log('Socket not connected');
    }
  };

  const onStop = (recordedBlob) => {
    console.log('recordedBlob is: ', recordedBlob);
  };

  return (
    <div>
      <ReactMic
        record={record}
        className="sound-wave"
        onStop={onStop}
        onData={onData}
        strokeColor="#000000"
        backgroundColor="#FF4081"
      />
      <button onClick={startRecording} type="button">
        Start
      </button>
      <button onClick={stopRecording} type="button">
        Stop
      </button>
    </div>
  );
};

export default Example;

@manosec
Copy link
Author

manosec commented Dec 5, 2023

Additionally, how can I modify the audio buffer here, I tried to downsample the buffer to make it compatible with Speech to Text service I'm using. I have attached the format below that I wanted the audio buffer convert to. Please share the relevant resources on this.

const {  samplesPerSecond, bitsPerSample, channels } = {
    "samplesPerSecond": 16000,
    "bitsPerSample": 16,
    "channels": 1
    };

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant