wake word only triggering once and not showing up on home assistant

[joshmsamuels]

Hey, I tried setting up wyoming satellite on a Raspberry Pi 3 with an Anker Powerconf as the Mic & Speaker.

I followed the raspberry pi tutorial, so I made sure the mic and speaker are working with the raspberry pi.

I finished the installation successfully, however, when I go to use my wake word "Ok Nabu" I hear the awake sound play once ... and then nothing. I don't hear the finish sound and when I say "Ok Nabu" again I don't hear the awake sound unless I restart wyoming-satellite, in which case I hear it one more time.

I ran wyoming-satellite and wyoming-openwakeword in debug mode which leads me to believe that wyoming-satellite hears my wake word but never stops listening to process the command.

Here are the logs for:
wyoming-satellite

pi@raspberrypi:~/wyoming-satellite $ /home/pi/wyoming-satellite/script/run --name 'Bedroom Satellite' --uri 'tcp://0.0.0.0:10700' --mic-command 'arecord -D plughw:CARD=S330,DEV=0 -r 16000 -c 1 -f S16_LE -t raw' --snd-command 'aplay -D plughw:CARD=S330,DEV=0 -r 220 -r 22050 -c 1 -f S16_LE -t raw' --mic-auto-gain 5 --mic-noise-suppression 2 --wake-uri 'tcp://127.0.0.1:10400' --wake-word-name 'ok_nabu' --awake-wav sounds/awake.wav --done-wav sounds/done.wav --debug
DEBUG:root:Namespace(mic_uri=None, mic_command='arecord -D plughw:CARD=S330,DEV=0 -r 16000 -c 1 -f S16_LE -t raw', mic_command_rate=16000, mic_command_width=2, mic_command_channels=1, mic_command_samples_per_chunk=1024, mic_volume_multiplier=1.0, mic_noise_suppression=2, mic_auto_gain=5, mic_seconds_to_mute_after_awake_wav=0.5, mic_no_mute_during_awake_wav=False, mic_channel_index=None, snd_uri=None, snd_command='aplay -D plughw:CARD=S330,DEV=0 -r 220 -r 22050 -c 1 -f S16_LE -t raw', snd_command_rate=22050, snd_command_width=2, snd_command_channels=1, snd_volume_multiplier=1.0, wake_uri='tcp://127.0.0.1:10400', wake_word_name=[['ok_nabu']], wake_command=None, wake_command_rate=16000, wake_command_width=2, wake_command_channels=1, wake_refractory_seconds=5.0, vad=False, vad_threshold=0.5, vad_trigger_level=1, vad_buffer_seconds=2, vad_wake_word_timeout=5.0, event_uri=None, startup_command=None, detect_command=None, detection_command=None, transcript_command=None, stt_start_command=None, stt_stop_command=None, synthesize_command=None, tts_start_command=None, tts_stop_command=None, tts_played_command=None, streaming_start_command=None, streaming_stop_command=None, error_command=None, connected_command=None, disconnected_command=None, timer_started_command=None, timer_updated_command=None, timer_cancelled_command=None, timer_finished_command=None, awake_wav='sounds/awake.wav', done_wav='sounds/done.wav', timer_finished_wav=None, timer_finished_wav_repeat=(1, 0), uri='tcp://0.0.0.0:10700', name='Bedroom Satellite', area=None, no_zeroconf=False, zeroconf_name=None, zeroconf_host=None, debug_recording_dir=None, debug=True, log_format='%(levelname)s:%(name)s:%(message)s')
INFO:root:Ready
DEBUG:root:Detected IP: 192.168.2.17
DEBUG:root:Zeroconf discovery enabled (name=b827eb6281eb, host=None)
DEBUG:root:Connecting to mic service: ['arecord', '-D', 'plughw:CARD=S330,DEV=0', '-r', '16000', '-c', '1', '-f', 'S16_LE', '-t', 'raw']
DEBUG:root:Connecting to snd service: ['aplay', '-D', 'plughw:CARD=S330,DEV=0', '-r', '220', '-r', '22050', '-c', '1', '-f', 'S16_LE', '-t', 'raw']
DEBUG:root:Connecting to wake service: tcp://127.0.0.1:10400
INFO:root:Connected to services
DEBUG:root:Using webrtc audio enhancements
Recording raw data 'stdin' : Signed 16 bit Little Endian, Rate 16000 Hz, Mono
DEBUG:root:Connected to mic service
DEBUG:root:Connected to wake service
DEBUG:root:Server set: 10580093688460
INFO:root:Connected to server
INFO:root:Waiting for wake word
DEBUG:root:Ping enabled
DEBUG:root:Detection(name='ok_nabu_v0.1', timestamp=10627807351039, speaker=None)
DEBUG:root:Streaming audio
DEBUG:root:Event(type='run-pipeline', data={'start_stage': 'asr', 'end_stage': 'tts', 'restart_on_end': False, 'snd_format': {'rate': 22050, 'width': 2, 'channels': 1}}, payload=None)
DEBUG:root:Muting microphone for 0.8995918367346939 second(s)
DEBUG:root:Connected to snd service
Playing raw data 'stdin' : Signed 16 bit Little Endian, Rate 22050 Hz, Mono
DEBUG:root:Unmuted microphone

wyoming-openwakeword (you can see me disconnecting and reconnecting wyoming satellite a few times)

pi@raspberrypi:~/wyoming-openwakeword $ /home/pi/wyoming-openwakeword/script/run --uri 'tcp://127.0.0.1:10400' --debug
DEBUG:root:Namespace(uri='tcp://127.0.0.1:10400', models_dir=PosixPath('/home/pi/wyoming-openwakeword/wyoming_openwakeword/models'), custom_model_dir=[], preload_model=[], threshold=0.5, trigger_level=1, output_dir=None, debug=True, log_format='%(levelname)s:%(name)s:%(message)s', debug_probability=False, version=False, model=[])
DEBUG:root:Loading /home/pi/wyoming-openwakeword/wyoming_openwakeword/models/melspectrogram.tflite
DEBUG:root:Loading /home/pi/wyoming-openwakeword/wyoming_openwakeword/models/embedding_model.tflite
INFO:root:Ready
DEBUG:wyoming_openwakeword.handler:Client connected: 9202134323062
DEBUG:root:Loading ok_nabu_v0.1 from /home/pi/wyoming-openwakeword/wyoming_openwakeword/models/ok_nabu_v0.1.tflite
DEBUG:wyoming_openwakeword.handler:Started thread for ok_nabu_v0.1
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
DEBUG:wyoming_openwakeword.handler:Sent info to client: 9202134323062
DEBUG:root:Loading ok_nabu_v0.1 from /home/pi/wyoming-openwakeword/wyoming_openwakeword/models/ok_nabu_v0.1.tflite
DEBUG:wyoming_openwakeword.handler:Started thread for ok_nabu_v0.1
DEBUG:wyoming_openwakeword.handler:Sent info to client: 9202134323062
DEBUG:root:Triggered ok_nabu_v0.1 (client=9202134323062)
DEBUG:wyoming_openwakeword.handler:Client disconnected: 9202134323062
DEBUG:wyoming_openwakeword.handler:Client connected: 9753518198084
DEBUG:root:Loading ok_nabu_v0.1 from /home/pi/wyoming-openwakeword/wyoming_openwakeword/models/ok_nabu_v0.1.tflite
DEBUG:wyoming_openwakeword.handler:Started thread for ok_nabu_v0.1
DEBUG:wyoming_openwakeword.handler:Sent info to client: 9753518198084
DEBUG:root:Loading ok_nabu_v0.1 from /home/pi/wyoming-openwakeword/wyoming_openwakeword/models/ok_nabu_v0.1.tflite
DEBUG:wyoming_openwakeword.handler:Started thread for ok_nabu_v0.1
DEBUG:wyoming_openwakeword.handler:Sent info to client: 9753518198084
DEBUG:root:Triggered ok_nabu_v0.1 (client=9753518198084)
DEBUG:root:Triggered ok_nabu_v0.1 (client=9753518198084)
DEBUG:wyoming_openwakeword.handler:Client disconnected: 9753518198084
DEBUG:wyoming_openwakeword.handler:Client connected: 10483911934835
DEBUG:root:Loading ok_nabu_v0.1 from /home/pi/wyoming-openwakeword/wyoming_openwakeword/models/ok_nabu_v0.1.tflite
DEBUG:wyoming_openwakeword.handler:Started thread for ok_nabu_v0.1
DEBUG:root:Loading ok_nabu_v0.1 from /home/pi/wyoming-openwakeword/wyoming_openwakeword/models/ok_nabu_v0.1.tflite
DEBUG:wyoming_openwakeword.handler:Started thread for ok_nabu_v0.1
DEBUG:wyoming_openwakeword.handler:Sent info to client: 10483911934835
DEBUG:wyoming_openwakeword.handler:Sent info to client: 10483911934835
DEBUG:root:Triggered ok_nabu_v0.1 (client=10483911934835)
DEBUG:root:Triggered ok_nabu_v0.1 (client=10483911934835)
DEBUG:wyoming_openwakeword.handler:Client disconnected: 10483911934835
DEBUG:wyoming_openwakeword.handler:Client connected: 10577285254060
DEBUG:root:Loading ok_nabu_v0.1 from /home/pi/wyoming-openwakeword/wyoming_openwakeword/models/ok_nabu_v0.1.tflite
DEBUG:wyoming_openwakeword.handler:Started thread for ok_nabu_v0.1
DEBUG:root:Loading ok_nabu_v0.1 from /home/pi/wyoming-openwakeword/wyoming_openwakeword/models/ok_nabu_v0.1.tflite
DEBUG:wyoming_openwakeword.handler:Started thread for ok_nabu_v0.1
DEBUG:wyoming_openwakeword.handler:Sent info to client: 10577285254060
DEBUG:root:Triggered ok_nabu_v0.1 (client=10577285254060)
DEBUG:wyoming_openwakeword.handler:Sent info to client: 10577285254060

[joshmsamuels]

My issue looked similar to #113 so I checked my HA logs and found many instances of this one error. It sounds like it could be related but I am not sure if it's the cause of my issue

Log details (ERROR)
Logger: homeassistant
Source: components/assist_pipeline/pipeline.py:1447
First occurred: 9:28:49 AM (16 occurrences)
Last logged: 1:15:49 PM

Error doing job: Task exception was never retrieved (None)
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/assist_pipeline/__init__.py", line 123, in async_pipeline_from_audio_stream
    await pipeline_input.validate()
  File "/usr/src/homeassistant/homeassistant/components/assist_pipeline/pipeline.py", line 1447, in validate
    raise PipelineRunValidationError(
homeassistant.components.assist_pipeline.pipeline.PipelineRunValidationError: the pipeline does not support speech-to-text

[joshmsamuels]

I modified my setup a little bit, following this "wyoming-enhancements" tutorial, which lets me use my Wyoming satellite to also play music. I am still seeing the same core issue, however, for completeness will paste the new logs

enhanced-wyoming-satellite.service

-- Journal begins at Tue 2024-03-12 01:13:58 GMT. --
Jul 01 03:21:13 raspberrypi run[2973]: INFO:root:Ready
Jul 01 03:21:14 raspberrypi run[2973]: DEBUG:root:Detected IP: 192.168.2.17
Jul 01 03:21:14 raspberrypi run[2973]: DEBUG:root:Zeroconf discovery enabled (name=b827eb6281eb, host=None)
Jul 01 03:21:14 raspberrypi run[2973]: DEBUG:root:Connecting to mic service: ['parecord', '--property=media.role=phone', '--rate=16000', '--channels=1', '--format=s16le', '--raw', '--latency-msec', '10']
Jul 01 03:21:14 raspberrypi run[2973]: DEBUG:root:Connecting to snd service: ['paplay', '--property=media.role=announce', '--rate=44100', '--channels=1', '--format=s16le', '--raw', '--latency-msec', '10']
Jul 01 03:21:14 raspberrypi run[2973]: DEBUG:root:Connecting to wake service: tcp://127.0.0.1:10400
Jul 01 03:21:14 raspberrypi run[2973]: INFO:root:Connected to services
Jul 01 03:21:14 raspberrypi run[2973]: DEBUG:root:Using webrtc audio enhancements
Jul 01 03:21:14 raspberrypi run[2973]: DEBUG:root:Connected to mic service
Jul 01 03:21:14 raspberrypi run[2973]: DEBUG:root:Connected to wake service
Jul 01 03:21:21 raspberrypi run[2973]: DEBUG:root:Server set: 2396338320539
Jul 01 03:21:21 raspberrypi run[2973]: INFO:root:Connected to server
Jul 01 03:21:21 raspberrypi run[2973]: INFO:root:Waiting for wake word
Jul 01 03:21:23 raspberrypi run[2973]: DEBUG:root:Ping enabled
Jul 01 03:22:20 raspberrypi run[2973]: DEBUG:root:Detection(name='ok_nabu_v0.1', timestamp=2455442727772, speaker=None)
Jul 01 03:22:20 raspberrypi run[2973]: DEBUG:root:Streaming audio
Jul 01 03:22:20 raspberrypi run[2973]: DEBUG:root:Event(type='run-pipeline', data={'start_stage': 'asr', 'end_stage': 'tts', 'restart_on_end': False, 'snd_format': {'rate': 44100, 'width': 2, 'channels': 1}}, payload=None)
Jul 01 03:22:20 raspberrypi run[2973]: DEBUG:root:Running ['/home/pi/wyoming-enhancements/snapcast/scripts/awake.sh']
Jul 01 03:22:20 raspberrypi run[2983]: Mon 1 Jul 03:22:20 BST 2024 [awake.sh] ...Starting awake.sh script
Jul 01 03:22:20 raspberrypi run[2983]: Mon 1 Jul 03:22:20 BST 2024 [awake.sh] ...Starting silence
Jul 01 03:22:20 raspberrypi run[2973]: DEBUG:root:Muting microphone for 0.8995918367346939 second(s)
Jul 01 03:22:20 raspberrypi run[2973]: DEBUG:root:Connected to snd service
Jul 01 03:22:21 raspberrypi run[2973]: DEBUG:root:Unmuted microphone
^C
pi@raspberrypi:~ $ date # Added date here to see how long I let it sit at the `Unmuted microphone` log line
Mon  1 Jul 03:27:07 BST 2024

[joshmsamuels]

I also realized I forgot to share my service config, so here is the enhanced-wyoming-satellite.service file

[Unit]
Description=Wyoming Satellite
Wants=network-online.target
After=network-online.target
Requires=wyoming-openwakeword.service
Requires=pulseaudio.service

[Service]
Type=simple
ExecStart=/home/pi/wyoming-satellite/script/run \
--name 'my Satellite' \
--uri 'tcp://0.0.0.0:10700' \
--mic-command 'parecord --property=media.role=phone --rate=16000 --channels=1 --format=s16le --raw --latency-msec 10' --snd-command 'paplay --property=media.role=announce --rate=44100 --channels=1 --format=s16le --raw --latency-msec 10' \
--snd-volume-multiplier 0.1 \
--mic-auto-gain 7 \
--mic-noise-suppression 3 \
--wake-uri 'tcp://127.0.0.1:10400' \
--wake-word-name 'ok_nabu' \
--awake-wav sounds/awake.wav \
--done-wav sounds/done.wav \
--snd-command-rate 44100 \
--debug --detection-command '/home/pi/wyoming-enhancements/snapcast/scripts/awake.sh' --tts-stop-command '/home/pi/wyoming-enhancements/snapcast/scripts/done.sh'
WorkingDirectory=/home/pi/wyoming-satellite
Restart=always
RestartSec=1

[Install]
WantedBy=default.target

[betodaviola]

I am facing a similar issue, but I don't have the sounds configured yet (I know it is happening by watching the logs). I will try it tomorrow with the sounds. In the meantime, it seems like we are not alone if you look at this thread:
#154

[N0C1]

It must be related to OpenWakeWord. When I switch to Porcupine everything is working as intended.

[hamiller]

are you sure? Just tried it also with porcupine and it's also only triggered once and never returned...

[kirel]

I currently have two satellites on a pi3 using different porcupine wakewords and it works quite okay. Feel free to check https://github.com/kirel/pi/tree/master/roles/mic-satellite for other differences you can spot. 🤔

[hamiller]

i've got it running with both systems: here was a good hint: missing configurations for whole "assist" pipeline in home assistant. After adding the conversation agent, as well as speech-to-text, it worked.

[Graimalkin]

Are you running VAD in this scenario? In the config without VAD, WakeStreamingSatellite event_from_mic() was getting stuck for me.