Some devices not responding across subnets.

[cnrd]

Hi

I'm trying to change around a couple of things in my HASS setup, which seems to have surfaced a bug in the communication with chuangmi_ir devices.

I can ping the device just fine:

ping 10.0.3.18
PING 10.0.3.18 (10.0.3.18) 56(84) bytes of data.
64 bytes from 10.0.3.18: icmp_seq=1 ttl=254 time=5.14 ms
64 bytes from 10.0.3.18: icmp_seq=2 ttl=254 time=6.86 ms
64 bytes from 10.0.3.18: icmp_seq=3 ttl=254 time=3.88 ms
64 bytes from 10.0.3.18: icmp_seq=4 ttl=254 time=2.10 ms
--- 10.0.3.18 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3005ms
rtt min/avg/max/mdev = 2.107/4.502/6.866/1.742 ms

but if i try to get info:

mirobo --ip 10.0.3.18 --token [TOKEN] info
ERROR:miio.device:Unable to discover a device at address 10.0.3.18
Error: Unable to discover the device 10.0.3.18

doing the same thing with my Roborock Vacuum works fine:

mirobo --ip 10.0.3.17 --token [TOKEN]
State: Charging
Battery: 100 %
Fanspeed: 100 %
Cleaning since: 0:02:34
Cleaned area: 2.765 m

It does work fine if I have an IP in the same subnet:

mirobo --ip 10.0.3.18 --token [TOKEN] info
chuangmi.ir.v2 v1.2.4_38 ([MAC]) @ 10.0.3.18 - token: [TOKEN]

I have tested on 2 different physical machines in different subnets, so it shouldn't be a problem with my network configuration.

[ivancwc]

yes this is definitely a bug. i bought two chuangmi_ir devices and encountered same issue

[cnrd]

@syssi any chance you know what is going on here? :-)

[syssi]

I own the device and will try to reproduce the issue. I don't know what's going on here. The code is untouched for month. I've no clue why it's broken now.

[cnrd]

I just don't think we have catched this bug before, is it only manifests when trying to communicate across subnets (in my case routed VLANs)

In my case the IR device is in 10.0.3.0/24
If I try to access it from another device in 10.0.3.0/24 then everything is fine.
If I try to access it from eg. 10.0.0.0/24 or 10.0.1.0/24 (both of which have access through the firewall) to the device, then it does not work.

One idea I had is just that the device does not like to communicate with IPs that are not in the same subnets as itself, in which case I don't think there is anything we can do.

[ivancwc]

I just don't think we have catched this bug before, is it only manifests when trying to communicate across subnets (in my case routed VLANs)

In my case the IR device is in 10.0.3.0/24
If I try to access it from another device in 10.0.3.0/24 then everything is fine.
If I try to access it from eg. 10.0.0.0/24 or 10.0.1.0/24 (both of which have access through the firewall) to the device, then it does not work.

One idea I had is just that the device does not like to communicate with IPs that are not in the same subnets as itself, in which case I don't think there is anything we can do.

for my case, my hassbian and the remote is in the same subnet, sadly i still get the issue

[domibarton]

I've the same issue with my humidifiers!

My test script:

#!/usr/bin/env python

from miio.airhumidifier import AirHumidifier

x = AirHumidifier(ip='10.9.10.23', token='…')
print(x.status())

My error:

Unable to discover a device at address 10.9.10.23
Traceback (most recent call last):
  File "./test.py", line 6, in <module>
    print(x.status())
  File "/opt/homeassistant/homeassistant/lib/python3.5/site-packages/miio/airhumidifier.py", line 280, in status
    self.device_info = self.info()
  File "/opt/homeassistant/homeassistant/lib/python3.5/site-packages/miio/device.py", line 319, in info
    return DeviceInfo(self.send("miIO.info"))
  File "/opt/homeassistant/homeassistant/lib/python3.5/site-packages/miio/device.py", line 224, in send
    self.do_discover()
  File "/opt/homeassistant/homeassistant/lib/python3.5/site-packages/miio/device.py", line 162, in do_discover
    raise DeviceException("Unable to discover the device %s" % self.ip)
miio.exceptions.DeviceException: Unable to discover the device 10.9.10.23

I've the same issue with the v1 and ca1 humidifiers, but not the RoboRock. My HomeAssistant (i.e. where the script run) is in 10.1.10.0/24, while all devices (humidifiers & vacuum) are in 10.9.10.0/24.

I'm sure the tokens and IP address are correct, as I can ping the IP address and the tokens are fetched from the latest iOS backup. The devices work flawlessly in the Mi Home app.

[syssi]

@domibarton Could you capture some network traffic? There must be a difference between the MiHome app (android <-> device) and the python-miio traffic (python-miio <-> device).

[domibarton]

@syssi I didn't capture anything, but I can try if you want. The communication isn't encrypted?

I'm using the iOS client. When starting the humidifier via, isn't the traffic then routed via Xiaomi's cloud service/server?

[domibarton]

So my setup looks like this:

                    +-------------+ 
                    |HomeAssistant| 
                    +------+------+ 
  Trusted VLAN             | 10.1.10.2
  10.1.10.0/24             |              10.1.10.1 +-----------+
  -------------------------+------------------------|           |
                                                    |  Gateway  +--> Internet Of Shit
  -------------------------+------------------------|           |
  Untrusted VLAN           |              10.9.10.1 +-----------+
  10.9.10.0/24             | 10.9.10.23   
                      +----+-----+
                      |Humidifier|
                      +----------+

The firewall is configured as following:

• 10.1.10.0/24 outbound: ACCEPT
• 10.1.10.0/24 inbound: DROP except for related/established connections
• 10.9.10.0/24 outbound: DROP except for DHCP/DNS (local) & HTTP/HTTPS (WAN)
• 10.9.10.0/24 inbound: DROP except for related/established connections & source 10.1.10.0/24

I also tried to ACCEPT everything, to make sure the firewall isn't the issue here. No success either!

I'm now capturing traffic the following ways:

• On the routing 10.9.10.1 interface, filtering for 10.9.10.23 host
  • Clicking on the humidifier in the Mi Home app
  • Starting the humidifier in the Mi Home app
  • Works as expected
  • Captured TCP packets from/to the Xiaomi Server (port 80 but not really HTTP)
• On the routing 10.9.10.1 interface, filtering for 10.9.10.23 host
  • Running the test.py above
  • Doesn't work
  • Captured 1 UDP packet IP 10.1.10.2.49908 > 10.9.10.23.54321: UDP, length 32

There's no reply to the UDP packet, thus the timeout before the Unable to discover the device… error.

[syssi]

The MiHome app talks the same protocol as python-miio as long the device is available directly. If your are not at home the instructions are delivered (TCP?) via the xiaomi cloud. Your capture proved: The MiHome app is unable to communicate directly, too.

[cnrd]

My guess is still that it is a "security" feature where the device will not respond to messages not coming from the same subnet. @domibarton any chance you are able to replay that first UDP package but change the IP header such that it looks to be coming from the same subnet?

[domibarton]

@syssi OC the app isn't communicating directly, as I'm connected to a different subnet. I assume the app wouldn't "guess" that the device is probably in another routed subnet and just give it a try. I think the app will check the direct connection only when the device IP is in the same subnet / SSID or alike?

@cnrd Mhm good point, I can try to SNAT that UDP package and see if I get a response from the device. Or I could configure the Untrusted VLAN tag on the RPi, so that I've a direct connection.

Nvmd of it, the RoboRock and the humidifiers apparently work differently. The RoboRock is in the same subnet and responds to the UDP packages, while the humidifers stay silent. I'll test the SNAT-thingy this evening (Switzerland UTC+1 ^^) and let you know!

[domibarton]

@cnrd Sorry for the delay, but I was quite busy the last three days. But I've good news:

YAY IT WORKS! I've masqueraded the traffic on the outbound interface of the VLAN/subnet 10.9.10.0/24 on the gateway. Now the device answers properly :)

[cnrd]

Thank you for confirming my suspicion :-) @syssi is there anything we can do about this or do we just have to accept that it only respond if the devices are in the same subnet? (Also I'm not sure if it happens for other devices, but we should probably add something to the documentation, if it can't be fixed).

[domibarton]

Problem

As @cnrd described above, discovering and querying devices across subnets doesn't work. @cnrd experienced this issue with chuangmi_ir devices, while I discovered the same issue with my humidifers!

Please read the comments above for more insights.
Long story, short: I don't know if this is a bug or a feature, but I guess it's a "security feature". In the end it doesn't matter, as this is hardcoded on the Xiaomi devices. Communication happens over UDP and if a packet source doesn't match the subnet of the Xiaomi device, there's simply no response.

Workarounds

Move into the same subnet

The most obvious solution: Move yourself into the same subnet as the Xiaomi devices or vice-versa.

Dual-homed node

Instead of moving your whole HomeAssistant (or whatever hub) into a new subnet, you can also dual-home your node (e.g. server, Raspberry Pi). You can do this either physically by connecting two ethernet cables (requires 2 ethernet ports on your server), or virtually by using VLAN's.

I'd probably prefer VLAN's in this case ;)
Just remember one thing: If your server was connected to a "trusted subnet" before and you're going to dual-home it, think about security. It was probably OK to have no local firewall when your server was connected to the trusted subnet, as you've a central firewall (and no port forwardings) in place. However, now with 2 legs in 2 subnets - one of them probably in a more "insecure" net - you might want to think about configuring a local firewall to prohibit access to private services.

Masquerading

If you've multiple VLAN's/subnets and you're in control over the router in between, then I'd setup masquerading for the outgoing routing interface of the VLAN/subnet where the Xiaomi devices reside. This basically means changing the source address in the UDP packet headers to the IP address of routing interface. If you want to know more about this, just inform yourself about packet masquerading and/or SNAT.

[domibarton]

I don't think we can do anything about that in the miio Python lib. That's basically a "networking issue" and you've to be in control of the router in between to "fake" the source.

At least if we're staying at this communication protocol & payload. There's probably another way to communicate with the Xiaomi devices, but I didn't look into that. Xiaomi's cloud servers also communicate with the devices. However, the devices are initiating the connection there, so that's most likely a dead path too ;)

Nevermind of it, I'd recommend changing the title of this issue to something more generic, as it doesn't only affect IR devices. What I can say right now is that it's also affecting both Xiaomi humidifer types, but not the RoboRock S50. Mentioning this issue in the docs would also be helpful for new users :)

[domibarton]

Btw thanks for your awesome work guys ;)

[rytilahti]

@domibarton considering you have been exploring this issue for a while, it would be really awesome if you could create a PR to add your discoveries to the documentation. A new troubleshooting section would be very helpful (there are also other well-known issues, e.g., the token encryption, that are currently only documented in various github issues which are really hard to locate aftwards).

[domibarton]

@rytilahti I'll have a look into it, as I love Sphinx as a Python dev… ;)

[domibarton]

@cnrd What was your device? I'm currently writing the docs, and I'd like to mention that in it!

[Ming-A]

so im having similar error '[homeassistant.components.xiaomi_miio.remote] Device unavailable or token incorrect: Unable to discover the device 10.69.20.19'

I did this in opnsense and seems like it still doesn't work. i double checked the IP and token and its correct

wherever you have 10.69.20.19 should be the IP of the home assistant

Gave it a try again, didn't seem to work for me for some reason, and i also only have the main 'Remote: Learn Command' service instead of the xiaomi miot learn command.

Home assistant: 10.69.40.4
wifi network: 10.69.20.0

[L4rryFisherman]

Spent quite a while today replicating this solution in openwrt. This worked, for anyone finding it later:

HA LAN: 192.168.31.60 IoT subnet: 192.168.32.x HA SNAT: 192.168.32.60

Firewall rules to let HA talk with IoT (these might be looser than necessary):

NAT rule to solve the subnet issue (masquerading instead of SNAT also works):

[Loic691]

I tried on mikrotik router without success.
SRC-NAT from HA IP to S7 with action MASQUERADE
Is there a workaround for mikrotik ?

[al13nus]

For me it also doesn't work with OPNsense. Someone had more luck? Can someone post a screen from inside the rule?

Edit: yeah, working now. Wrong configuration of the rule.

[xanox1]

I tried on mikrotik router without success. SRC-NAT from HA IP to S7 with action MASQUERADE Is there a workaround for mikrotik ?

Yes sir, i have this working.

/ip firewall add action=masquerade chain=srcnat dst-address=10.0.20.213 out-interface=BR_IOT src-address=10.0.30.101

10.0.20.213 = L10
10.0.30.101 = HA

BR_IOT is my IOT vlan interface.

You could limit the port to UDP/54321 i suppose, but i couldn't be bothered.

[rwjack]

I run pfsense and even though my HassOS has multiple interfaces, it also has one in the IOT Subnet, and I'm still getting these connectivity blips.

I don't see how NAT is the solution to this problem. As far as I can tell everyone here has HASS and their IOT devices on the same home network, be it in the same subnet, or a separate one. Pfsense being a router, should route the packets to the destination subnet, no NAT required, nothing is leaving the home network.

[hoppel118]

I can confirm that setting up masquerading between vlans on dest port 54321 did indeed work and all miio-based libraries started working for me, so big thanks to @domibarton!

@fuomag9 can you give an example of the rule you implemented to do MASQUERADE with a specific port? Currently I have:
iptables -t nat -A POSTROUTING -s 192.168.1.0/24 -d 192.168.20.0/24 -j MASQUERADE
is adding --dport 54321 enough before the -j?

I tried that, but it didn't work. How to pass the port 54321?

Hi,

the complete iptables command (for the udm-pro) is the following:

iptables -t nat -A POSTROUTING -s YOUR_HOMEASSISTANT_IP/24 -d YOUR_DEVICE_IP/24 -p UDP -j MASQUERADE --to-ports 54321

This works here for two Xiaomi Smart Fan models:

• Smartmi DC Pedestal Fan
• Smartmi Standing Fan 2S

A long standing „issue“ is solved for me. Thank you so much, guys. :D

Regards Hoppel

Hi,

I migrated from FHEM to Home Assistant and now my masquerading doesn't work with home assistant.

This is how it worked with FHEM:

iptables -t nat -A POSTROUTING -s YOUR_FHEM_IP/32 -d YOUR_DEVICE_IP/32 -p UDP -j MASQUERADE --to-ports 54321

and this is how it works with Home Assistant:

iptables -t nat -A POSTROUTING -s YOUR_HOMEASSISTANT_IP/32 -d YOUR_DEVICE_IP/32 -j MASQUERADE

But why? Which ports are needed by Home Assistant, that FHEM doesn't need?

Thanks Hoppel

[rwjack]

I wouldn't use masquerade or NAT-ing. You want HA to talk to another LOCAL device. If that device in another subnet then your router should do the talking for you.

[xanox1]

I wouldn't use masquerade or NAT-ing. You want HA to talk to another LOCAL device. If that device in another subnet then your router should do the talking for you.

The router still does the talking. The problem is that the robots wont accept traffic from addresses outside of their own subnet range.
Hence the masquerade to pretend to be coming from the same subnet.

[rwjack]

Ah, I didn't fully understand the issue then.

The problem I'm having is just connectivity blips:

[hoppel118]

I wouldn't use masquerade or NAT-ing. You want HA to talk to another LOCAL device. If that device in another subnet then your router should do the talking for you.

The router still does the talking. The problem is that the robots wont accept traffic from addresses outside of their own subnet range. Hence the masquerade to pretend to be coming from the same subnet.

@rwjack Yes, the iptables rules are configured on my router (Unifi Dream Machine SE).

Your last post triggered me to investigate some time into this again.

After the migration from FHEM to Home Assistant, my iptables rules didn’t work any more. I described that in my last post.

So I configured a trunk port to my home assistant vm and connected the iot vlan to HA. Now, my Xiaomi devices were reachable, but I also had the „connectivity blips“. The devices were switching between available/unavailable constantly.

Now with the working iptables rules, the connectivity blips seem to be solved in my case. At least there was no unavailable log for the last 5 hours. Hurrah… 😃

EDIT: Did you see the „Note“ in the HA documentation under „Prerequisites“?

Note
For more complex network setups (e.g. VLANs), reference the following documentation for additional information.

[csi-lk]

So, for the UDM pro folks if you never figured this out

@gespo89 this saved me a significant amount of time, thank you so much for writing this out, would suggest adding this to the documentation

[hoppel118]

Is here anybody who can help making iptables nat masquerading rules permanent on a Unifi Dram Machine (firmware 3.x)?

https://community.ui.com/questions/UDM-SE-Permanently-remove-change-iptables-rules-for-NAT-masquerading/243e20c5-8642-4929-8163-b075d5f4cc02

EDIT: Issue solved. You can find the solution in the link before.

Thanks and regards Hoppel

[JefferiesTube]

Any chance to get this done with a simple FritzBox (I don't have any nice pfSense oder DM setup yet)?