Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat/listening_modes #150

Merged
merged 20 commits into from Aug 31, 2022
Merged

Conversation

NeonJarbas
Copy link

@NeonJarbas NeonJarbas commented Jun 9, 2022

adds ListenMode, this is globally set in mycroft.conf and can be one of the following

  • WAKE_WORD - the classic mode, listen for wakewords and trigger actions based on them
  • CONTINUOUS - use VAD to detect speech and always transcribe it, ignore STT failures
    • ⚠️ offline streaming STT highly recommended! ⚠️
  • HYBRID - after a wake word detection switch to continuous mode for X seconds (45 by default)
    • this allows follow up question without wake word, but good AEC is needed or mycroft will hear its own TTS

adds ListeningState, this is a runtime state and can be one of the following

  • WAKE_WORD - currently waiting for a wake word
  • CONTINUOUS - currently using VAD to detect speech
  • RECORDING - save all audio until told to stop, then reset to default mode
    • think "captain's log, stardate 184876, today the warp core exploded, bla bla bla"

other notes:

  • bus events can be used to query and change the listening mode / state
  • new class of hotwords, "stopwords", only used while in RECORDING state to stop recording
  • wakeword/utterance save paths now respect XDG
  • added filename to message.data in "recognizer_loop:record_end"

@JarbasAl JarbasAl requested a review from NeonDaniel June 9, 2022 09:26
@JarbasAl JarbasAl added the enhancement New feature or request label Jun 9, 2022
@codecov
Copy link

codecov bot commented Jun 9, 2022

Codecov Report

Merging #150 (1f5fb2c) into dev (6ceb058) will increase coverage by 3.24%.
The diff coverage is 36.58%.

@@            Coverage Diff             @@
##              dev     #150      +/-   ##
==========================================
+ Coverage   50.35%   53.59%   +3.24%     
==========================================
  Files         119      152      +33     
  Lines       10077     9725     -352     
==========================================
+ Hits         5074     5212     +138     
+ Misses       5003     4513     -490     
Impacted Files Coverage Δ
mycroft/audio/__main__.py 0.00% <0.00%> (ø)
mycroft/client/enclosure/__main__.py 0.00% <0.00%> (ø)
mycroft/client/enclosure/mark1/arduino.py 0.00% <0.00%> (ø)
mycroft/client/enclosure/mark1/eyes.py 0.00% <0.00%> (ø)
mycroft/client/enclosure/mark1/mouth.py 0.00% <0.00%> (ø)
mycroft/client/speech/__main__.py 0.00% <0.00%> (ø)
mycroft/client/speech/hotword_factory.py 0.00% <0.00%> (-88.89%) ⬇️
mycroft/client/speech/service.py 0.00% <0.00%> (ø)
mycroft/client/speech/silence.py 0.00% <0.00%> (-42.86%) ⬇️
mycroft/client/text/__init__.py 0.00% <0.00%> (ø)
... and 138 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@NeonJarbas NeonJarbas marked this pull request as ready for review June 9, 2022 10:32
mycroft/listener/__init__.py Show resolved Hide resolved
mycroft/listener/__init__.py Outdated Show resolved Hide resolved
mycroft/listener/__init__.py Outdated Show resolved Hide resolved
Copy link
Member

@NeonDaniel NeonDaniel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any idea why codecov is annotating unmodified lines? Makes the PR hard to read...

mycroft/listener/mic.py Outdated Show resolved Hide resolved
Copy link
Member

@NeonDaniel NeonDaniel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/OpenVoiceOS/ovos-core/pull/150/files#diff-0f7600a790272b32edf283b7755335cab5e9458d57eae200a4aa6648d43ed272L483

self._listen_triggered should be set to False here to avoid an infinite listening loop

@NeonDaniel
Copy link
Member

Tested in NeonGeckoCom/neon_speech#96

@JarbasAl JarbasAl merged commit 002838d into OpenVoiceOS:dev Aug 31, 2022
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants