how could I save audio to wavfile?

[Jackiexiao]

I want to use streamlit-webrtc to record wav and save to disk (like temp.wav'), but I have some problem, after I click stopbutton, audio_buffer becomeNone` instead of audio from microphone

my code

    webrtc_ctx = webrtc_streamer(
        key="sendonly-audio",
        mode=WebRtcMode.SENDONLY,
        audio_receiver_size=256,
        client_settings=WEBRTC_CLIENT_SETTINGS,
    )

    sound_window_len = 1  # 1ms

    audio_buffer = pydub.AudioSegment.silent(
        duration=sound_window_len
    )
    status_indicator = st.empty()

    while True:
        if webrtc_ctx.audio_receiver:
            try:
                audio_frames = webrtc_ctx.audio_receiver.get_frames(timeout=1)
            except queue.Empty:
                status_indicator.write("No frame arrived.")
                continue

            status_indicator.write("Running. Say something!")

            sound_chunk = pydub.AudioSegment.empty()
            for audio_frame in audio_frames:
                st.info('get audio frame')
                sound = pydub.AudioSegment(
                    data=audio_frame.to_ndarray().tobytes(),
                    sample_width=audio_frame.format.bytes,
                    frame_rate=audio_frame.sample_rate,
                    channels=len(audio_frame.layout.channels),
                )
                sound_chunk += sound

            if len(sound_chunk) > 0:
                audio_buffer += sound_chunk
        else:
            status_indicator.write("AudioReciver is not set. Abort.")
            break

    st.info("Writing wav to disk")
    audio_buffer.export('temp.wav', format='wav')

[whitphx]

Hi, please try below.

The reason is that your app script is re-executed every time component state (e.g. playing or not) changes then audio_buffer object differs in the execution where the audio recording loop is running and the next execution where .export() is called.
This re-execution is Streamlit's fundamental execution model (See https://docs.streamlit.io/en/stable/main_concepts.html#app-model),
and it provides some ways to handle problems like this case, and the example below uses st.session_state to keep the same sound object over executions.

import queue

import pydub

import streamlit as st
from streamlit_webrtc import webrtc_streamer, WebRtcMode, ClientSettings


def main():
    webrtc_ctx = webrtc_streamer(
        key="sendonly-audio",
        mode=WebRtcMode.SENDONLY,
        audio_receiver_size=256,
        client_settings=ClientSettings(
            rtc_configuration={
                "iceServers": [{"urls": ["stun:stun.l.google.com:19302"]}]
            },
            media_stream_constraints={
                "audio": True,
            },
        ),
    )

    if "audio_buffer" not in st.session_state:
        st.session_state["audio_buffer"] = pydub.AudioSegment.empty()

    status_indicator = st.empty()

    while True:
        if webrtc_ctx.audio_receiver:
            try:
                audio_frames = webrtc_ctx.audio_receiver.get_frames(timeout=1)
            except queue.Empty:
                status_indicator.write("No frame arrived.")
                continue

            status_indicator.write("Running. Say something!")

            sound_chunk = pydub.AudioSegment.empty()
            for audio_frame in audio_frames:
                sound = pydub.AudioSegment(
                    data=audio_frame.to_ndarray().tobytes(),
                    sample_width=audio_frame.format.bytes,
                    frame_rate=audio_frame.sample_rate,
                    channels=len(audio_frame.layout.channels),
                )
                sound_chunk += sound

            if len(sound_chunk) > 0:
                st.session_state["audio_buffer"] += sound_chunk
        else:
            status_indicator.write("AudioReciver is not set. Abort.")
            break

    audio_buffer = st.session_state["audio_buffer"]

    if not webrtc_ctx.state.playing and len(audio_buffer) > 0:
        st.info("Writing wav to disk")
        audio_buffer.export("temp.wav", format="wav")

        # Reset
        st.session_state["audio_buffer"] = pydub.AudioSegment.empty()


if __name__ == "__main__":
    main()

[whitphx]

If your purpose is only to record the sound to a file without any processing with pydub, recorder option is the simplest way:

from streamlit_webrtc import webrtc_streamer, WebRtcMode, ClientSettings
from aiortc.contrib.media import MediaRecorder


def main():
    def recorder_factory():
        return MediaRecorder("record.wav")

    webrtc_streamer(
        key="sendonly-audio",
        mode=WebRtcMode.SENDONLY,
        in_recorder_factory=recorder_factory,
        client_settings=ClientSettings(
            rtc_configuration={
                "iceServers": [{"urls": ["stun:stun.l.google.com:19302"]}]
            },
            media_stream_constraints={
                "audio": True,
                "video": False,
            },
        ),
    )


if __name__ == "__main__":
    main()

[Jackiexiao]

thx very much for your help, I will dive into webRTC and streamlit for details, thxs

[Jackiexiao]

If your purpose is only to record the sound to a file without any processing with pydub, recorder option is the simplest way:

from streamlit_webrtc import webrtc_streamer, WebRtcMode, ClientSettings
from aiortc.contrib.media import MediaRecorder


def main():
    def recorder_factory():
        return MediaRecorder("record.wav")

    webrtc_streamer(
        key="sendonly-audio",
        mode=WebRtcMode.SENDONLY,
        in_recorder_factory=recorder_factory,
        client_settings=ClientSettings(
            rtc_configuration={
                "iceServers": [{"urls": ["stun:stun.l.google.com:19302"]}]
            },
            media_stream_constraints={
                "audio": True,
                "video": False,
            },
        ),
    )


if __name__ == "__main__":
    main()

I use sample code as above, but it doesn't work as expected if you play it (seems it missing half of audio frame), it used to work before.

[Jackiexiao]

if you put wavfile in audition, you get ( by the way, record wav using webrtc_ctx.audio_receiver.get_frames(timeout=1) works fine

@whitphx

[whitphx]

mm, while I'm not familiar with sound processing and cannot understand what the figures mean,
I wonder if setting some options to the MediaRecorder works.
MediaRecorder passes the options parameter to FFmpeg (via the underlying PyAV library). So, what if you set options like options={"sample_rate": ..., "channels": ...}?

[Jackiexiao]

set options doesn't work, but if I change mode to mode=WebRtcMode.SENDRECV instead of SENDONLY, MediaRecorder works fine

I write a gist here for reproduce

[hihunjin]

@whitphx @Jackiexiao I just did in your @Jackiexiao's gist. I played it right after I recorded it, and it seems the speed of the .wav file is two times faster. How could I solve this problem?

[Jackiexiao]

@hihunjin use this function to record https://gist.github.com/Jackiexiao/9c45d5ad9caf524471b5ed966c57b5b9#file-streamlit_recorder-py-L131

[whitphx]

@Jackiexiao thank you for your investigation and problem-solving.
The fact that SENDRECV instead SENDONLY works fine is interesting hint. Thanks!

[JordiBustos]

Hi! Im using the code that you post there https://gist.github.com/Jackiexiao/9c45d5ad9caf524471b5ed966c57b5b9#file-streamlit_recorder-py-L131. When i click the START button it works just fine but when i stop it the audio file doesn't save it anywhere. I don't know if a need to change some parameter or something like that. Sorry im new coding with streamlit :)

[JordiBustos]

Actually im getting this log #552
Any idea on how to solve it :/?

[casiopa]

Hi @whitphx and @Jackiexiao, this code is very helpful. Thanks a lot!
I wonder how to, by default, turn off the speaker while recording. The goal is to prevent audio feedback when not wearing headphones.

[whitphx]

You can control the HTML attributes of the <audio /> element through the audio_html_attrs of webrtc_streamer() component as below.

    webrtc_ctx: WebRtcStreamerContext = webrtc_streamer(
        key="sendonly-audio",
        mode=WebRtcMode.SENDRECV,
        in_recorder_factory=recorder_factory,
        media_stream_constraints=MEDIA_STREAM_CONSTRAINTS,
        audio_html_attrs={"muted": True} # THIS!
    )

When you set only muted=true, the audio component is completely hidden. So if you want to show the audio control and mute the audio at the same time, set controls=true too as below.

        audio_html_attrs={"muted": True, "controls": True}

You can find other available attributes: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/audio