Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AudioRecorder control #2494

Merged
merged 21 commits into from
Feb 3, 2024
Merged

AudioRecorder control #2494

merged 21 commits into from
Feb 3, 2024

Conversation

ndonkoHenri
Copy link
Collaborator

@ndonkoHenri ndonkoHenri commented Jan 27, 2024

Tested on:

  • Desktop
  • Web
  • Mobile

Notes:

  • To test on mobile (ios simulator for example), modify the path used in the start_recording case to the one below:
import 'package:path_provider/path_provider.dart';
String? fullPath = '${(await getApplicationDocumentsDirectory()).path}/test-audio-file.wav';

Test code:

import flet as ft


async def main(page: ft.Page):
    page.theme_mode = "light"
    page.window_always_on_top = True
    page.horizontal_alignment = ft.CrossAxisAlignment.CENTER
    path = "test-audio-file.wav"

    async def handle_start_recording(e):
        print(f"StartRecording: {path}")
        await audio_rec.start_recording_async(path)

    async def handle_stop_recording(e):
        output_path = await audio_rec.stop_recording_async()
        print(f"StopRecording: {output_path}")
        if page.web and output_path is not None:
            await page.launch_url_async(output_path)

    async def handle_list_devices(e):
        devices = await audio_rec.get_input_devices_async()
        print(devices)

    async def handle_has_permission(e):
        try:
            print(f"HasPermission: {await audio_rec.has_permission_async()}")
        except Exception as e:
            print(e)

    async def handle_pause(e):
        print(f"isRecording: {await audio_rec.is_recording_async()}")
        if await audio_rec.is_recording_async():
            await audio_rec.pause_recording_async()

    async def handle_resume(e):
        print(f"isPaused: {await audio_rec.is_paused_async()}")
        if await audio_rec.is_paused_async():
            await audio_rec.resume_recording_async()

    async def handle_audio_encoding_test(e):
        for i in list(ft.AudioEncoder):
            print(f"{i}: {await audio_rec.is_supported_encoder_async(i)}")

    page.appbar = ft.AppBar(
        title=ft.Text("Audio Recorder"),
        adaptive=True,
        bgcolor=ft.colors.PRIMARY_CONTAINER,
    )

    audio_rec = ft.AudioRecorder(
        audio_encoder=ft.AudioEncoder.WAV,
    )
    page.overlay.append(audio_rec)
    await page.update_async()

    await page.add_async(
        ft.ElevatedButton(
            "Start Audio Recorder",
            on_click=handle_start_recording,
        ),
        ft.ElevatedButton(
            "Stop Audio Recorder",
            on_click=handle_stop_recording,
        ),
        ft.ElevatedButton(
            "List Devices",
            on_click=handle_list_devices,
        ),
        ft.ElevatedButton(
            "Pause Recording",
            on_click=handle_pause,
        ),
        ft.ElevatedButton(
            "Resume Recording",
            on_click=handle_resume,
        ),
        ft.ElevatedButton(
            "Test AudioEncodings",
            on_click=handle_audio_encoding_test,
        ),
        ft.ElevatedButton(
            "Has Permission",
            on_click=handle_has_permission,
        ),
    )


ft.app(target=main)

@ndonkoHenri ndonkoHenri linked an issue Jan 27, 2024 that may be closed by this pull request
@FeodorFitsner
Copy link
Contributor

Do we need stream support for audio recording? Are there any use-cases/examples on Python side that could benefit from a real-time streaming?

@ndonkoHenri
Copy link
Collaborator Author

No idea tbh.

@rkleivel
Copy link

Do we need stream support for audio recording? Are there any use-cases/examples on Python side that could benefit from a real-time streaming?

Hi! As a software developer with a musical background, I see real potential in adding stream support for audio recording. If it's feasible to integrate without significant overhead, I believe it would be a valuable addition

@exactstat
Copy link

Do we need stream support for audio recording? Are there any use-cases/examples on Python side that could benefit from a real-time streaming?

Hi! Speech recognition requires streaming raw PCM data chunks to be used by the AI model. Chunks can be formed based on fixed time intervals or based on auto pause/resume by preconfigured dB levels + some time gap.
I can imagine these scenarios:

  1. manual start record + each X milliseconds: push data to the URL or a WebSocket, or execute some passed function on the data gathered.
  2. manual "start" record + auto reaction. dB levels and time gaps should be configured for the first start, total end, pause, and resume. Each chunk can be passed to the URL / ws / function

@FeodorFitsner FeodorFitsner merged commit 1a2d73b into main Feb 3, 2024
1 of 2 checks passed
@FeodorFitsner FeodorFitsner deleted the microphone branch February 4, 2024 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

AudioRecorder control
4 participants