speech-command playRawAudio plays blank audio

[adotnusiyan]

System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow.js): No
• OS Platform and Distribution: Windows10
• TensorFlow.js installed from: npm
• TensorFlow.js version: 3.2.0
• speech-command version: 0.5.2
• Browser version: Version 88.0.4324.192 (Official Build) (x86_64)

Describe the current behavior
I'm trying to use playRawAudio method by providing RawAudioData object. Float32Array as data and sampleRateHz:44100.

There's an audio playing with the same duration of data, but the audio is blank (no audio)

I provided Float32Array array from SpeechCommandRecognizerResult.spectrogram.data in listen function, then concatenated the array using concatenateFloat32Arrays util function

Describe the expected behavior
Expected to play the correct audio, not a blank audio.

Standalone code to reproduce the issue
Example in Codesandbox

[zipweight]

@adotnusiyan
did you find a solution?
i also need this and got same issue

[adotnusiyan]

Not yet.
Hopefully waiting @lina128

[lina128]

Hi @adotnusiyan and @zipweight , I'm investigating this, I'm seeing a sampleRate mismatch error in my browser, not sure if it is related, just fyi: tensorflow/tfjs-models#612

[lina128]

Hi @adotnusiyan, I looked at the code you provided, it seems you provide the spectrum data to the playRawAudio function. You need to provide raw audio sound, not the spectrum data.

[adotnusiyan]

@lina128

How to play the spectrum data as audio?

Or how can I get raw audio data from listen function?

Thanks

[lina128]

If you have a transferRecognizer with collectExample({includeRawAudio = true}). And then get the rawAudio from tranferRecognizer.getExamples()[0].example.rawAudio.

If you just want a generic solution, you need to implement your own.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No

[adotnusiyan]

@lina128 I tried your suggestion with basic code from the example in README and the audio doesn't sound right. You can hear some distortion/glitching in the audio

You can try it yourself in this example