Huawei LTE fails during setup

[larsvinc]

Home Assistant release with the issue:

0.102.2

Last working Home Assistant release (if known):

Operating environment (Hass.io/Docker/Windows/etc.):

Debian 10 Buster
Home assistant in virtual environment per the home assistant guide.

Integration:

https://www.home-assistant.io/integrations/huawei_lte/

Description of problem:
Huawei B535 setup fails. I have tested huawei-lte-api manually and it works with the B535. I am running the B535 in bridge mode, so it cannot see any devices. I want to use the integration to send notifiations and monitor the LTE

Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):

It is configured via the GUI, with the proper username and password. It also fails if I run it in non-authenticated mode.

Traceback (if applicable):

Error setting up entry B535-232 for huawei_lte
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/config_entries.py", line 192, in async_setup
    hass, self
  File "/etc/homeassistant/custom_components/huawei_lte/__init__.py", line 313, in async_setup_entry
    **device_data,
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/helpers/device_registry.py", line 123, in async_get_or_create
    for key, value in connections
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/helpers/device_registry.py", line 123, in <setcomp>
    for key, value in connections
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/helpers/device_registry.py", line 56, in format_mac
    if len(to_test) == 17 and to_test.count(":") == 5:
TypeError: object of type 'NoneType' has no len()

Additional information:

[probot-home-assistant]

Hey there @scop, mind taking a look at this issue as its been labeled with a integration (huawei_lte) you are listed as a codeowner for? Thanks!

[larsvinc]

@scop let me know if I can assist in further debugging.

[nepozs]

It looks like platform independent problem (I'm using hass.io) - after update from 0.101.3 to 0.102.2 this integration is broken.

I've read Breaking Changes section before update, but I already have configuration like in example
- url: http://192.168.22.1/ username: !secret huawei_lte_user password: !secret huawei_lte_pass and it was working till update.
I've also tried GUI configuration, but also without success.

[scop]

@larsvinc the problem originates from that we're unable to grab mac address for your router. What does the URL you're using look like? If you're using an IP address in the URL, can you try what this outputs?

python -c 'import getmac;print(getmac.get_mac_address(ip="192.168.100.1"))' # replace your ip address

...or if you're using a hostname in it:

python -c 'import getmac;print(getmac.get_mac_address(hostname="your-hostname-here"))' # replace your hostname

@larsvinc have you got the integration to work with earlier Home Assistant versions, or is this the first one you tried with? IIRC we have been using mac address identifiers since HA 0.97, so I believe something could have failed already with those versions.

@nepozs please confirm that the traceback you're seeing is essentially the same as in the original comment. Also please specify if you're using the router in bridged mode or not.

[larsvinc]

I have not gotten this to work with earlier versions.

(homeassistant) homeassistant@kraftkar:/srv/homeassistant$ python3 -c 'import getmac;print(getmac.get_mac_address(ip="192.168.8.1"))' # replace your ip address
None

See the output above. Can this be caused by the router sitting outside the firewall? It is kind of interesting because when I run it on devices that does not exist in the local network it returns all zeros:

getmac.get_mac_address(ip="192.168.100.109")
'00:00:00:00:00:00'

Looks like the fix is to handle the returned "None"?

[scop]

Maybe, but I'd first like to understand when does it end up returning None and to see if this is something that can be addressed in getmac. We'd really like a usable mac, even if it would be the one of the firewall.

Could you make the getmac module within your HA to produce debug output in your HA log and see what's there in case it gives some hints? There are two parts to this, first the logging config in configuration.yaml:

logger:
  default: info
  logs:
    getmac: debug

...as well as a dirty code hack in some module, for example near the rest of the imports in homeassistant/components/huawei_lte/__init__.py, add these lines:

import getmac
getmac.getmac.DEBUG = 9

It should start producing lines containing [getmac].

[larsvinc]

First I changed the router address to 10.0.0.1 to make absolutely sure that there was no issue in the 192.168-ranges. The problem persisted. Then I ran the full debug per your request. Output:
2019-12-03 21:31:26 DEBUG (SyncWorker_19) [getmac] Trying: 'arp -a 10.0.0.1'
2019-12-03 21:31:26 DEBUG (SyncWorker_19) [getmac] Running: '/usr/sbin/arp -a 10.0.0.1'
2019-12-03 21:31:27 DEBUG (SyncWorker_19) [getmac] Output from '/usr/sbin/arp' command: b'arp: in 5 entries no match found.\n'
2019-12-03 21:31:27 DEBUG (SyncWorker_19) [getmac] Result: None
2019-12-03 21:31:27 DEBUG (SyncWorker_19) [getmac] Trying: '_uuid_ip' (to_find: '10.0.0.1')
2019-12-03 21:31:27 DEBUG (SyncWorker_19) [getmac] Result: None
2019-12-03 21:31:27 DEBUG (SyncWorker_19) [getmac] Raw MAC found: None

I tried the arp command manually with sudo, (sudo arp -a 10.0.0.1), and it seems like the problem is that the router is not in the local arp cache.

[scop]

Thanks for testing. Can you reach the device via arping? I.e. arping -f -c 1 10.0.0.1? I submited that to getmac upstream, let's see what they think. GhostofGoes/getmac#52

Anyway I suppose we need to handle the case where None is returned nevertheless. #29455 hopefully accomplishes that.

[larsvinc]

:~$ sudo arping -F -c 1 10.0.0.1
ARPING 10.0.0.1
Timeout

--- 10.0.0.1 statistics ---
1 packets transmitted, 0 packets received, 100% unanswered (0 extra)

I also tested arping on a different valid IP and there it worked.

[nepozs]

@scop my traceback (looks nearly identical)
`

2019-12-01 01:16:30 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry B525s-23a for huawei_lte
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/config_entries.py", line 192, in async_setup
hass, self
File "/usr/src/homeassistant/homeassistant/components/huawei_lte/init.py", line 287, in async_setup_entry
await hass.async_add_executor_job(router.update)
File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/src/homeassistant/homeassistant/components/huawei_lte/init.py", line 176, in update
get_data(KEY_WLAN_HOST_LIST, self.client.wlan.host_list)
File "/usr/src/homeassistant/homeassistant/components/huawei_lte/init.py", line 152, in get_data
self.data[key] = func()
File "/usr/local/lib/python3.7/site-packages/huawei_lte_api/api/WLan.py", line 69, in host_list
host = hosts.get('Hosts', {}).get('Host')
AttributeError: 'NoneType' object has no attribute 'get'
`

My Huawei LTE works as gateway, but is in in more complicated configuration: 2 routers in "cascade" connection - Huawei has LAN IP 192.168.22.1 in network 192.168.22.0/24 but hassio works behind secondary router (also in gateway mode) with LAN 192.168.20.0/24 so no arp is possible (router after router), but before upgrade to 0.102.x this integration had worked properly.

[scop]

@nepozs your backtrace indicates that it's a different issue from this one, it's almost certainly #28922 which has been fixed and I believe will appear in 0.102.4 or 0.103, whichever comes first.

[scop]

@larsvinc oh well, arping doesn't rescue all cases it seems. But BTW, which arping is the one you use? The one I have installed on Ubuntu 18.04 (from iputils-arping package) and hass.io doesn't have the -F argument, they both have just -f ("quit on first arp reply"). Does yours understand the lowercase one?

Anyway I hope the fix from #29455 covers the None case.

[scop]

(Forgot to note that in case #29455 doesn't, please let me know.)

[scop]

Nevermind the arping version, my patch was accepted in getmac and there's some discussion about that there, see GhostofGoes/getmac#52 (comment)

[larsvinc]

@scop - I am using ARPing 2.19 (running debian buster amd64). Glad to hear that there is a patch in getmac.