unavailable state for all DECO devices most of the time

[freddieleeman]

I'm experiencing that all DECO devices report an 'unavailable' status most of the time. Sometimes it works for a few minutes, but not for long. I have 4x TP-Link Deco M4 and around 70 connected devices. I tried to increase the interval, but that doesn't appear to change anything. Is anyone seeing the same thing?

Logger: custom_components.tplink_deco
Source: custom_components/tplink_deco/api.py:134
Integration: TP-Link Deco (documentation, issues)
First occurred: 1:38:08 PM (3 occurrences)
Last logged: 1:41:19 PM

Unexpected error fetching tplink_deco data: List devices error None
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 187, in _async_refresh
    self.data = await self._async_update_data()
  File "/config/custom_components/tplink_deco/coordinator.py", line 90, in _async_update_data
    new_clients = await self._api.async_list_clients()
  File "/config/custom_components/tplink_deco/api.py", line 134, in async_list_clients
    raise Exception(f"List devices error {error_code}")
Exception: List devices error None

Logger: custom_components.tplink_deco
Source: custom_components/tplink_deco/api.py:152
Integration: TP-Link Deco (documentation, issues)
First occurred: 1:29:03 PM (6 occurrences)
Last logged: 1:40:19 PM

Unexpected error fetching tplink_deco data: List clients error None
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 187, in _async_refresh
    self.data = await self._async_update_data()
  File "/config/custom_components/tplink_deco/coordinator.py", line 90, in _async_update_data
    new_clients = await self._api.async_list_clients()
  File "/config/custom_components/tplink_deco/api.py", line 152, in async_list_clients
    raise Exception(f"List clients error {error_code}")
Exception: List clients error None

[amosyuen]

Looks like for some reason the response json does not have an "error_code" attribute. I can try adding more logging to help see what response we are getting from the deco.

[amosyuen]

I added more logging in https://github.com/amosyuen/ha-tplink-deco/releases/tag/v1.1.1, which should log the response if you set the logging level to debug. Please copy and past that log.

[MickW69]

For what its worth, I am getting the same unavailable status as mentioned above, have just updated HA to 2022.3.3 and updated Deco integration. For the beginners amongst us, "Put code: logging: deco = dubug" into config etc, would be helpful.

[Gyosa3]

Hi, I have a setup of 6 P9 decos, some of which are quite unstable so I reboot all of them every night and I wish to monitor their disconnection.
I am using this integration and had trouble setting it up. I got "unavailable" permanently.
I've had this error in the log:

Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 187, in _async_refresh self.data = await self._async_update_data() File "/config/custom_components/tplink_deco/coordinator.py", line 236, in _async_update_data deco_client_responses = await asyncio.gather( File "/config/custom_components/tplink_deco/coordinator.py", line 39, in async_call_with_retry return await func(*args) File "/config/custom_components/tplink_deco/api.py", line 160, in async_list_clients data=self._encode_payload(client_payload), File "/config/custom_components/tplink_deco/api.py", line 330, in _encode_payload sign = self._encode_sign(len(data)) File "/config/custom_components/tplink_deco/api.py", line 336, in _encode_sign seq_with_data_len = self._seq + data_len TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'

I reloaded the integration, and forced a reconfigure with a polling time of 60 secs, then first the decos themselves were "home" when they're connected, while all the client devices are permanently "unavailable".

Then I had this error:

Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 187, in _async_refresh self.data = await self._async_update_data() File "/config/custom_components/tplink_deco/coordinator.py", line 236, in _async_update_data deco_client_responses = await asyncio.gather( File "/config/custom_components/tplink_deco/coordinator.py", line 39, in async_call_with_retry return await func(*args) File "/config/custom_components/tplink_deco/api.py", line 163, in async_list_clients data = self._decrypt_data(context, response_json["data"]) File "/config/custom_components/tplink_deco/api.py", line 363, in _decrypt_data raise Exception("Need to re-login") Exception: Need to re-login

Then the integration did a new attempt and all clients were "home". Then 15 minutes later they are all "unavailable" due to the first error coming again after a "forbidden" connection error:

`Logger: custom_components.tplink_deco
Source: custom_components/tplink_deco/api.py:311
Integration: TP-Link Deco (documentation, issues)
First occurred: 11:36:53 (6 occurrences)
Last logged: 11:37:07

List Clients 00-5F-67-A3-A3-8A client response error: 403, message='Forbidden', url=URL('http://192.168.68.1/cgi-bin/luci/;stok=69c7b1de8a06147b50043bb05b8a41b3/admin/client?form=client_list')
List Clients 00-5F-67-A3-91-18 client response error: 403, message='Forbidden', url=URL('http://192.168.68.1/cgi-bin/luci/;stok=69c7b1de8a06147b50043bb05b8a41b3/admin/client?form=client_list')
List Clients 00-5F-67-A3-8F-2C client response error: 403, message='Forbidden', url=URL('http://192.168.68.1/cgi-bin/luci/;stok=69c7b1de8a06147b50043bb05b8a41b3/admin/client?form=client_list')
Login client response error: 403, message='Forbidden', url=URL('http://192.168.68.1/cgi-bin/luci/;stok=/login?form=login')
List Clients 00-31-92-23-55-D8 client response error: 403, message='Forbidden', url=URL('http://192.168.68.1/cgi-bin/luci/;stok=69c7b1de8a06147b50043bb05b8a41b3/admin/client?form=client_list')`

then

Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 187, in _async_refresh self.data = await self._async_update_data() File "/config/custom_components/tplink_deco/coordinator.py", line 159, in _async_update_data new_decos = await async_call_with_retry(self._api.async_list_devices) File "/config/custom_components/tplink_deco/coordinator.py", line 39, in async_call_with_retry return await func(*args) File "/config/custom_components/tplink_deco/api.py", line 130, in async_list_devices data=self._encode_payload(device_list_payload), File "/config/custom_components/tplink_deco/api.py", line 330, in _encode_payload sign = self._encode_sign(len(data)) File "/config/custom_components/tplink_deco/api.py", line 336, in _encode_sign seq_with_data_len = self._seq + data_len TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'

I am running the uPnP integration in parallel on the main deco and it is reachable and delivers the requested data

[freddieleeman]

Here is my most recent log entry:

Logger: custom_components.tplink_deco
Source: custom_components/tplink_deco/api.py:167
Integration: TP-Link Deco (documentation, issues)
First occurred: March 14, 2022, 1:01:29 AM (17 occurrences)
Last logged: March 14, 2022, 4:03:40 AM

Unexpected error fetching tplink_deco-clients data: List Clients D8-47-32-A4-E2-40 error None
Unexpected error fetching tplink_deco-clients data: List Clients D8-47-32-A4-E1-80 error None
Unexpected error fetching tplink_deco-clients data: List Clients 28-EE-52-66-C2-F8 error None
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 187, in _async_refresh
    self.data = await self._async_update_data()
  File "/config/custom_components/tplink_deco/coordinator.py", line 236, in _async_update_data
    deco_client_responses = await asyncio.gather(
  File "/config/custom_components/tplink_deco/coordinator.py", line 39, in async_call_with_retry
    return await func(*args)
  File "/config/custom_components/tplink_deco/api.py", line 167, in async_list_clients
    raise Exception(f"{context} error {error_code}")
Exception: List Clients D8-47-32-A4-E2-40 error None

I haven't had time to enable debug mode yet. I will do that today and see if I can get more details on the issue.

[freddieleeman]

Here is the debug data from the log. It looks like the calls have timeout issues.

2022-03-15 12:01:46 DEBUG (MainThread) [custom_components.tplink_deco] Finished fetching tplink_deco-decos data in 10.638 seconds (success: True)
2022-03-15 12:02:07 DEBUG (MainThread) [custom_components.tplink_deco] Finished fetching tplink_deco-clients data in 21.005 seconds (success: True)
2022-03-15 12:03:57 DEBUG (MainThread) [custom_components.tplink_deco] Finished fetching tplink_deco-clients data in 20.935 seconds (success: True)
2022-03-15 12:05:49 DEBUG (MainThread) [custom_components.tplink_deco] Finished fetching tplink_deco-clients data in 22.786 seconds (success: True)
2022-03-15 12:07:37 DEBUG (MainThread) [custom_components.tplink_deco] Finished fetching tplink_deco-clients data in 18.840 seconds (success: True)
2022-03-15 12:09:18 DEBUG (MainThread) [custom_components.tplink_deco] List Clients 28-EE-52-66-C2-F8 error: data={'success': False, 'errorcode': 'timeout'}
2022-03-15 12:09:18 ERROR (MainThread) [custom_components.tplink_deco] Unexpected error fetching tplink_deco-clients data: List Clients 28-EE-52-66-C2-F8 error None
2022-03-15 12:09:18 DEBUG (MainThread) [custom_components.tplink_deco] Finished fetching tplink_deco-clients data in 11.466 seconds (success: False)
2022-03-15 12:11:00 DEBUG (MainThread) [custom_components.tplink_deco] List Clients 28-EE-52-66-C2-F8 error: data={'success': False, 'errorcode': 'timeout'}
2022-03-15 12:11:00 ERROR (MainThread) [custom_components.tplink_deco] Unexpected error fetching tplink_deco-clients data: List Clients 28-EE-52-66-C2-F8 error None
2022-03-15 12:11:00 DEBUG (MainThread) [custom_components.tplink_deco] Finished fetching tplink_deco-clients data in 12.546 seconds (success: False)
2022-03-15 12:12:48 INFO (MainThread) [custom_components.tplink_deco] Fetching tplink_deco-clients data recovered
2022-03-15 12:12:48 DEBUG (MainThread) [custom_components.tplink_deco] Finished fetching tplink_deco-clients data in 18.244 seconds (success: True)
2022-03-15 12:14:37 DEBUG (MainThread) [custom_components.tplink_deco] Finished fetching tplink_deco-clients data in 19.258 seconds (success: True)
2022-03-15 12:16:19 DEBUG (MainThread) [custom_components.tplink_deco] List Clients 28-EE-52-66-C2-F8 error: data={'success': False, 'errorcode': 'timeout'}
2022-03-15 12:16:19 ERROR (MainThread) [custom_components.tplink_deco] Unexpected error fetching tplink_deco-clients data: List Clients 28-EE-52-66-C2-F8 error None
2022-03-15 12:16:19 DEBUG (MainThread) [custom_components.tplink_deco] Finished fetching tplink_deco-clients data in 12.664 seconds (success: False)
2022-03-15 12:17:49 DEBUG (MainThread) [custom_components.tplink_deco] List Clients D8-47-32-A4-E1-80 error: data={'success': False, 'errorcode': 'timeout'}
2022-03-15 12:17:49 ERROR (MainThread) [custom_components.tplink_deco] Unexpected error fetching tplink_deco-clients data: List Clients D8-47-32-A4-E1-80 error None
2022-03-15 12:17:49 DEBUG (MainThread) [custom_components.tplink_deco] Finished fetching tplink_deco-clients data in 0.723 seconds (success: False)
2022-03-15 12:17:49 DEBUG (MainThread) [custom_components.tplink_deco] List Clients D8-47-32-A4-E1-64 error: data={'success': False, 'errorcode': 'timeout'}
2022-03-15 12:17:49 DEBUG (MainThread) [custom_components.tplink_deco] List Clients D8-47-32-A4-E2-40 error: data={'success': False, 'errorcode': 'timeout'}
2022-03-15 12:17:50 DEBUG (MainThread) [custom_components.tplink_deco] List Clients 28-EE-52-66-C2-F8 error: data={'success': False, 'errorcode': 'timeout'}

[Gyosa3]

Not sure if it helps, here is the history of the availability of the clients. Two bars on top are 2 deco boxes, two bars below are 2 cameras permanently connected in wifi to the Deco network.

refresh time is 90 sec

[amosyuen]

403 errors usually mean you are logging in with the admin credentials (in the app or the web portal) while the integration is running. I'm guessing you're logging in with the admin credentials to reset the routers? If so the 403s are expected errors.

The clients are showing up as unavailable because the http call times out. Please try these steps:

1. Disable the integration temporarily
2. Log in to the web portal for the router (not the app)
3. Verify that the list clients works in the web portal
4. Open the browser developer console, go to the network tab, and check how long the list clients call is taking.

We might just need to increase the timeout value. So I can increase it based on how long yours is taking.

[freddieleeman]

Here are the results:

Average: 5.18 seconds
Highest: 7.61 seconds
Lowest: 3.66 seconds

What is the current timeout setting?

[amosyuen]

Current timeout is 30s, on closer look seems the timeout error is actually a response from the deco, so that seems to indicate it's a timeout on the deco side. You had no problems with listing clients on the web portal?

[freddieleeman]

No, none.

[amosyuen]

Not clear to me why the deco is returning timeout with such a short timeout, but since that's on their side not much I can do. I added code to retry if that case happens. Hopefully that helps mitigate the problem a little.

[freddieleeman]

It looks like the timeouts are gone now. I do see these now:

2022-03-28 14:59:25 ERROR (MainThread) [custom_components.tplink_deco] Unexpected error fetching tplink_deco-clients data: 'result'
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 187, in _async_refresh
    self.data = await self._async_update_data()
  File "/config/custom_components/tplink_deco/coordinator.py", line 236, in _async_update_data
    deco_client_responses = await asyncio.gather(
  File "/config/custom_components/tplink_deco/coordinator.py", line 39, in async_call_with_retry
    return await func(*args)
  File "/config/custom_components/tplink_deco/api.py", line 192, in async_list_clients
    client_list = data["result"]["client_list"]
KeyError: 'result'

[christoros]

The latest update fixed this problem for me