Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add speech-to-text cooldown for local wake word #108806

Merged
merged 14 commits into from Feb 27, 2024

Conversation

synesthesiam
Copy link
Contributor

@synesthesiam synesthesiam commented Jan 24, 2024

Breaking change

Proposed change

With streaming wake word detection, HA stops multiple satellites waking up at the same time by enforcing a "cooldown" period (default: 5 seconds). During this period, any pipeline with the same wake word (and wake word system) cannot run.

When satellites use local wake word detection, they begin their pipeline runs at the speech-to-text (STT) stage instead. This PR adds a "cooldown" to STT, and includes a new parameter for identifying the wake word phrase that was detected by the satellite (e.g., "ok nabu"). Now, if two satellites report the same wake word and attempt to begin their STT stages within the cooldown period (5 seconds), one will succeed and the other will receive a duplicate_wake_up_detected error.

Additionally, both local and streaming wake word detection now share cooldown periods for the same "wake word phrase". For example, if one satellite uses streaming wake word detection with an ok nabu phrase and a different satellite uses local wake word detection with an ok nabu phrase, they cannot both be woken up at the same time.

The wake word phrase must be supplied by the wake word provider or satellite. For Wyoming-based services, this is now included in the info message.

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Deprecation (breaking change to happen in the future)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

  • This PR fixes or closes issue: fixes #
  • This PR is related to issue:
  • Link to documentation pull request:

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • I have followed the perfect PR recommendations
  • The code has been formatted using Ruff (ruff format homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
  • Untested files have been added to .coveragerc.

To help with the load of incoming pull requests:

@home-assistant
Copy link

Hey there @balloob, mind taking a look at this pull request as it has been labeled with an integration (assist_pipeline) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of assist_pipeline can trigger bot actions by commenting:

  • @home-assistant close Closes the pull request.
  • @home-assistant rename Awesome new title Renames the pull request.
  • @home-assistant reopen Reopen the pull request.
  • @home-assistant unassign assist_pipeline Removes the current integration label and assignees on the pull request, add the integration domain after the command.
  • @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the pull request.
  • @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the pull request.

@synesthesiam synesthesiam marked this pull request as ready for review January 25, 2024 17:34
@synesthesiam synesthesiam requested a review from a team as a code owner January 30, 2024 22:48
@synesthesiam synesthesiam marked this pull request as draft January 31, 2024 02:46
@synesthesiam synesthesiam force-pushed the synesthesiam-20240124-stt-cooldown branch from 50a6a30 to 872229a Compare February 26, 2024 20:21
@synesthesiam synesthesiam marked this pull request as ready for review February 26, 2024 20:26
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
@balloob balloob merged commit f6622ea into dev Feb 27, 2024
53 checks passed
@balloob balloob deleted the synesthesiam-20240124-stt-cooldown branch February 27, 2024 01:35
hass_ws_client: WebSocketGenerator,
snapshot: SnapshotAssertion,
) -> None:
"""Test that two speech-to-text pipelines can run within the cooldown period if they have the different wake words."""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please limit docstrings to max 72 characters per line in accordance with PEP8.

SatelliteAsyncTcpClient(events),
) as mock_client,
patch(
"homeassistant.components.wyoming.satellite.assist_pipeline.async_pipeline_from_audio_stream",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider breaking out the patch to a pytest fixture that yields the mock. Then set a side effect on the mock in the test as needed instead. We can avoid repeating all these patches in the tests then.

@github-actions github-actions bot locked and limited conversation to collaborators Feb 28, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants