New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Wake word & Satellite audio stream using seemingly random UDP high (> 1024) port #105106
Comments
Hey there @balloob, @synesthesiam, mind taking a look at this issue as it has been labeled with an integration ( Code owner commandsCode owners of
(message by CodeOwnersMention) wyoming documentation |
Seems like I ran face first into this as well This will still likely break for some use cases without specifying the listen IP address as well, since the UDP connection to HA won't be an established connection so docker networking/nat/etc can all break it |
me too. i refuse to run containers as root and/or in host mode so it's impossible for me right now to establish the needed udp connection Edit: So i think i a fixed port/small port-range would be awesome Probably relevant: #95654 |
Configurable port ranges would be fine. But it's poor practice to open the network (or Docker config) for "all ports > 1024”. |
Folks, I know this is annoying for some or most of us, but please don't comment on this just to add that you're affected, too. That does provide any additional input towards a solution. Give the issue a thumbs up instead. That helps visibility more and keeps it easy to read for everyone. Thanks! |
I tried to monkey fix it myself. Maybe you guys can give it a try: https://gist.github.com/koriwi/617cf90107c7d413b53ca2c2c6fdf1e6 |
@koriwi great idea. I did a very similar local patch and it works great for me now. |
I tried this in kurbenetes and was unable to get it working. Running a k3s cluster at home, nodeport service and tried using 3100-3107 with those 8 ports configured from your code. I tested the ports opening when a request goes through that makes the port open and the port does open, but it ends up timing out on whisper. I setup the ports to open under home assistant container, I assume that would be correct since the being modified is under home assistant, or should it be under whisper? |
Yeah. That is correct. But do you get any other log messages? Like audio empty or something? I needed to switch the mirophone channel(in esphome) because it was outputting on the wrong channel |
I am not using wake work and making the esphome device requiring holding the button down to activate for testing. Home Assistant: Whisper:
under debug assistant I get
|
@koriwi, how did you change the channel and what channel did you set it to? Looks like the board version for my device is v1.1 and G23 shown here http://docs.m5stack.com/en/atom/atomecho appears to be what the official firmware uses. |
I wonder if anyone has tried or considered making hass send a first UDP packet to punch a hole in the NAT gatewa/firewall, as an alternative to restrict the port range. The way this could work would be as follows:
The advantage of doing this is that firewalls and NAT gateways will see Homeassistant as the client, or the peer who initiated the connection, and thus create a path for the packets data the esphome device sends to flow back to hass. This would not require any configuration or privileged access, and it should work behind firewalls and NAT gateways, including Docker and Kubernetes. I'm not very strong at Python but I think it is simple enough for someone to give it a shot. WDYT? |
I'm on my phone right now, but you will find it in the i2c microphone docu of esphome Also I'm not using an m5 echo |
I managed to get this working, but on the latest docker stable tag, it looks like the file had some heavy modifications and tts stopped working. I tried to reapply the port range modification, and unfortunately it still errors, though it does play tts for a split second. Stable version the image is currently pulling from
|
I am facing the same problem, would be very good to be able to configure a range of ports to be able to support segmented and firewalled networks. |
Looks like this is in the works: esphome/esphome#6471 |
Looks very promising, thanks mate |
ESPHome 2024.4.0 has been released. The changelog notes:
I will verify this when Home Assistant 2024.5.0 is released and will update everyone here. |
In the case it could help, in my case it looks UDP port is still used even if I uploaded a firmware with ESPHome 2024.4.2 and Home Assistant 2024.5.1. 2024-05-03 18:27:01.590 WARNING (MainThread) [homeassistant.components.esphome.manager] Voice assistant UDP server was not stopped I don't really understand why I still have message about UDP port if both side recognize native API can be used instead of UDP or I probably have missed something. |
Closing this issue as it works now with latest HA and ESPHome version. If you still encounter issues, please open a new issue and attach logs. |
The problem
The wake word pipeline as described in Year of the Voice - Chapter 4 connects from the satellite to Home Assistant using a seemingly random "high" (= number > 1024) port.
This leads to pipeline timeouts as described in e.g.:
The current workaround for all systems is "open the firewall on all UDP ports for Home Assistant".
This does not apply for Home Assistant OS since it seems to not firewall these ports/open these for that purpose.
However, this is not possible without massively compromising on security on some setups, e.g. the one I run with the docker container on Kubernetes. In that setup, I would need to run Home Assistant in "host mode", meaning that it effectively runs in the network space of the host.
Proposal
To enable the use of wake words in all supported Home Assistant installation methods, the UDP port used to transmit the voice stream should be fixed.
What version of Home Assistant Core has the issue?
core-2023.11.3
What was the last working version of Home Assistant Core?
never worked
What type of installation are you running?
Home Assistant Container
Integration causing the issue
wyoming
Link to integration documentation on our website
https://www.home-assistant.io/integrations/wyoming/
Diagnostics information
Check these tcpdump lines to see that an M5 Atom Echo configured according to the 13$ Voice Assistant tutorial connects on seemingly random UDP high ports.
The M5 Atom Echo is currently configured to not use the Wake Word, but the button.
Pressing the button for the first time leads to the light turning blue (= listening) and the connections in the log on port 60252:
Pressing it again deactivates the microphone. Then pressing a third time leads to it listening again and connections on port
55427
:Full log in the logs section below.
Example YAML snippet
No response
Anything in the logs that might be useful for us?
Additional information
I am seriously impressed with all the progress voice integration in Home Assistant is making. It's amazing to see it coming along and I'm looking forward to the future with this.
I decided to open this issue to get a discussion started rather sooner than later.
If I missed another open issue or documentation for this - I couldn't find any - please let me know.
The text was updated successfully, but these errors were encountered: