Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AVM FRITZ!Box Tools - device tracker entities for the Fritz Box and all repeaters are not defined/created since 2022.2.0 #65541

Closed
browetd opened this issue Feb 3, 2022 · 77 comments
Assignees

Comments

@browetd
Copy link

browetd commented Feb 3, 2022

The problem

I am tracking the availability of all my network devices using device_tracker entities. Since I upgraded to version 2022.2.0 all the device trackers for the fritz box and all fritz repeaters are "unavailable" while other entities (button, switch, binary_sensors) for those equipments exist. All other devices I am tracking have been correctly created... No error message seems to be existent for that integration but I have the following error at startup (but this one was already there in 2021.12.x without creating any issues):

2022-02-03 10:14:19 ERROR (SyncWorker_8) [homeassistant.components.fritz.common] Connection Error: Please check the device is properly configured for remote login
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/fritz/common.py", line 537, in _service_call_action
    result: dict = self.connection.call_action(
  File "/usr/local/lib/python3.9/site-packages/fritzconnection/core/fritzconnection.py", line 284, in call_action
    return self.soaper.execute(service, action_name, arguments)
  File "/usr/local/lib/python3.9/site-packages/fritzconnection/core/soaper.py", line 259, in execute
    return handle_response(response)
  File "/usr/local/lib/python3.9/site-packages/fritzconnection/core/soaper.py", line 241, in handle_response
    raise_fritzconnection_error(response)
  File "/usr/local/lib/python3.9/site-packages/fritzconnection/core/soaper.py", line 164, in raise_fritzconnection_error
    raise exception(message)
fritzconnection.core.exceptions.FritzArgumentError: UPnPError: 
errorCode: 402
errorDescription: Invalid Args

I have also the following error (a couple times at startup) but not sure if this is linked with Fritz Box integration:

2022-02-03 10:19:45 WARNING (MainThread) [homeassistant.helpers.frame] Detected code that uses str (config) for entity category. This is deprecated and will stop working in Home Assistant 2022.4, it should be updated to use EntityCategory instead. Please report this issue.
Stack (most recent call last):
  File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 203, in <module>
    sys.exit(main())
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 191, in main
    exit_code = runner.run(runtime_conf)
  File "/usr/src/homeassistant/homeassistant/runner.py", line 119, in run
    return loop.run_until_complete(setup_and_run_hass(runtime_config))
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 629, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 596, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1890, in _run_once
    handle._run()
  File "/usr/local/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 510, in _async_add_entity
    entry = entity_registry.async_get_or_create(
  File "/usr/src/homeassistant/homeassistant/helpers/entity_registry.py", line 401, in async_get_or_create
    entity_category=_convert_to_entity_category(entity_category),
  File "/usr/src/homeassistant/homeassistant/helpers/entity_registry.py", line 103, in _convert_to_entity_category
    return convert_to_entity_category(value, raise_report=raise_report)
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 214, in convert_to_entity_category
    report(
  File "/usr/src/homeassistant/homeassistant/helpers/frame.py", line 74, in report
    _LOGGER.warning(msg, stack_info=True)
Stack (most recent call last):
  File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 203, in <module>
    sys.exit(main())
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 191, in main
    exit_code = runner.run(runtime_conf)
  File "/usr/src/homeassistant/homeassistant/runner.py", line 119, in run
    return loop.run_until_complete(setup_and_run_hass(runtime_config))
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 629, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 596, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1890, in _run_once
    handle._run()
  File "/usr/local/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 510, in _async_add_entity
    entry = entity_registry.async_get_or_create(
  File "/usr/src/homeassistant/homeassistant/helpers/entity_registry.py", line 401, in async_get_or_create
    entity_category=_convert_to_entity_category(entity_category),
  File "/usr/src/homeassistant/homeassistant/helpers/entity_registry.py", line 103, in _convert_to_entity_category
    return convert_to_entity_category(value, raise_report=raise_report)
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 214, in convert_to_entity_category
    report(
  File "/usr/src/homeassistant/homeassistant/helpers/frame.py", line 74, in report
    _LOGGER.warning(msg, stack_info=True)

What version of Home Assistant Core has the issue?

core-2022.2.0

What was the last working version of Home Assistant Core?

core-2021.12.x

What type of installation are you running?

Home Assistant OS

Integration causing the issue

AVM FRITZ!Box Tools

Link to integration documentation on our website

https://www.home-assistant.io/integrations/fritz/

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

@probot-home-assistant
Copy link

fritz documentation
fritz source
(message by IssueLinks)

@probot-home-assistant
Copy link

Hey there @mammuth, @AaronDavidSchneider, @chemelli74, @mib1185, mind taking a look at this issue as it has been labeled with an integration (fritz) you are listed as a code owner for? Thanks!
(message by CodeOwnersMention)

@mib1185
Copy link
Contributor

mib1185 commented Feb 3, 2022

Hi @browetd
please enable debug logging, restart HA and provide the homeassistant.log after the issue occurs.
To do so add the following to your configuration.yaml:

logger:
  default: info
  logs:
    homeassistant.components.fritz: debug
    fritzconnection: debug

Further please download and provide the diagnostics data for this integration if possible.

Note: it is better to drag the log into the comment (which will add it as an attachment) and not copy paste as it is hard to read logs in GitHub.

@chemelli74
Copy link
Contributor

I'm digging into the same issue; so far I discovered that:

  • cleanup doesn't fix the issue
  • removing the integration -> restart HA -> wait for autodetect -> configure integration at least remove some of the above errors

Simone

@LSDetlev
Copy link

LSDetlev commented Feb 3, 2022

I have the same Issuse.
Router ist FRITZ!Box 7582, FRITZ!OS: 07.16

logfile
home-assistant.log

@browetd
Copy link
Author

browetd commented Feb 3, 2022

@mib1185 Here is the logfile.

My network:
192.168.2.1: Fritz Box 7590
192.168.2.40: Fritz Repeater 3000 connected to the fritzbox (via cable)
192.168;2.41: Fritz Repeater 2400 connected to 192.168.2.40 (via wi-fi)
192.168.2.42: Fritz Repeater 3000 connected to the fritzbox (via wi-fi)
192.168.2.43: Fritz Powerline 540E

logfile
Nouveau document texte.txt

@mib1185
Copy link
Contributor

mib1185 commented Feb 3, 2022

@browetd @LSDetlev
may I ask you to test the linked PR if this solves the issue for you?

@LSDetlev
Copy link

LSDetlev commented Feb 3, 2022

I am very sorry but I have no Idea how to test the link, I am kinda lost
edit: managed to test it, didn't work for me tho sorry :(

@WeVolutionNet
Copy link

I'm totaly new to this integration and I'm pretty sure I'm getting it wrong...
However... While reading your issue this came to my mind?!
https://www.home-assistant.io/blog/2022/02/02/release-20222/#improved-handling-of-device-tracker-entities

Maybe you have to check this part now? " device tracker entities that match up with an existing - known by Home Assistant - device"

Hope this helps... Please ignore if not ;-)

@kongo09
Copy link

kongo09 commented Feb 3, 2022

I have the same problem. A lot of devices are detected correctly. But some are not. It feels like there is a system in int:

  • all mobile phones are missing
  • the Google devices (Nest, Chromecast) are missing
  • the Dell printer is missing
  • the Unifi access points are missing

but on the other hand

  • all Shellies are present
  • the Nukis are present
  • the roborock vacuum is present
  • the Philips air purifiers are present

It feels weird that it is consistent groups being present or not. There are no error logs at all. I've completely removed the integration and started from scratch - same result.

@kongo09
Copy link

kongo09 commented Feb 3, 2022

device tracker entities that match up with an existing - known by Home Assistant - device

That feels like something different. And something also broken. In my case, two raspis are detected. They were not previously knwon to HA. But the mobile devices are not detected, while they are all known through the companion app.

@felix63
Copy link

felix63 commented Feb 3, 2022

I have the same issue lost all my device trackers from the fritzbox integration.
FRITZ!Box Fon WLAN 7360
by AVM
Connected via meterkast
Firmware: 124.06.87

@Oxaluz
Copy link

Oxaluz commented Feb 3, 2022

Same on my system

@felix63
Copy link

felix63 commented Feb 4, 2022

2022.2.1 does not seem to solve this

@mib1185
Copy link
Contributor

mib1185 commented Feb 4, 2022

I have the same problem. A lot of devices are detected correctly. But some are not. It feels like there is a system in int:

  • all mobile phones are missing
  • the Google devices (Nest, Chromecast) are missing
  • the Dell printer is missing
  • the Unifi access points are missing

but on the other hand

  • all Shellies are present
  • the Nukis are present
  • the roborock vacuum is present
  • the Philips air purifiers are present

It feels weird that it is consistent groups being present or not. There are no error logs at all. I've completely removed the integration and started from scratch - same result.

This sounds to be related to the new device tracker handling 🤔

@Smandurlo
Copy link

Smandurlo commented Feb 4, 2022

Totally broken. Again. Every release has a problem or has a braking change and we have to adjust things again and again and again. I really miss the old integration. It was rock solid, never never never had a problem nor a glitch. It simply worked as expected. Now I am not sure I had at least a couple of months troubles free.

I lost all my mesh routers and repeaters, they are not created. Remove, reboot, add again from scratch the integration do not solve the problem. Fritz 7490.
Another home assistant with a fritz 4040 without mesh: mobile phones and a Google mini without device tracking.

@browetd
Copy link
Author

browetd commented Feb 5, 2022

Problem still exist in 2022.2.2

@miljw002
Copy link

miljw002 commented Feb 5, 2022

I updated this morning and have the same or similar issue. All the device tracker entities I had are now unavailable. Tried reboots or HA and router and hasn’t helped.

@browetd
Copy link
Author

browetd commented Feb 5, 2022

In my case, I have all the device trackers correctly created except the Fritz boxes and Fritz repeaters,
All shellys, meross plugs, tuya devices, Doorbird, Alarm Myfox, Smappee, Mobile Alert, Hormann Gateway Iroomba, Evohome system, Air Conditioning devices, Camera's, Access Points other than Fritz and other equipments... are all correctly defined...

@chemelli74
Copy link
Contributor

@Smandurlo and all the others, we are working hard to achive 2 goals:

  • all the requested features, scenarios covered
  • improve performances, means less API call to the Fritz

I think that we are now very close: firmware update, mesh management, details of each device tracker, set guest password, are all implemented. We just wait for QR code for Guest WIFI .
The Team doesn't have anything left on the todo after that.

About the second, unfortunately some API from AVM are broken on some devices, and we need to find different ways to achive the same result.
Luckily often there is more than 1 way to do it, so we are actively looking into all issues to find a stable solution for you all.

We are also looking for an updated fritzconnection to get the last requested performance improvement; with that we will do 1 single call for all devices instead of 1 for each device!

Please share an updated log and diagnostics from 2022.02.2 or later to help this process.

Simone

@Oxaluz
Copy link

Oxaluz commented Feb 5, 2022 via email

@chemelli74
Copy link
Contributor

@Oxaluz, I miss the log ;-)

Please don't send it via email, but drag and drop in a reply via web.

Simone

@browetd
Copy link
Author

browetd commented Feb 5, 2022

@chemelli74 Here is my last log... HA 2022.2.2...

Logfile.txt

@chemelli74
Copy link
Contributor

@chemelli74 Here is my last log... HA 2022.2.2...

Logfile.txt

Your device says that WANCommonInterfaceConfig exist as a service, but once invoked it throws an error:

2022-02-05 17:42:34 DEBUG (SyncWorker_6) [fritzconnection] b'<?xml version="1.0" encoding="utf-8"?><s:Envelope s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"><s:Body><u:GetAddonInfos xmlns:u="urn:schemas-upnp-org:service:WANCommonInterfaceConfig:1"></u:GetAddonInfos></s:Body></s:Envelope>'
2022-02-05 17:42:34 DEBUG (SyncWorker_8) [fritzconnection] response status: 500
2022-02-05 17:42:34 DEBUG (SyncWorker_8) [fritzconnection] <?xml version="1.0"?>
 <s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<s:Body>
<s:Fault>
<faultcode>s:Client</faultcode>
<faultstring>UPnPError</faultstring>
<detail>
<UPnPError xmlns="urn:dslforum-org:control-1-0">
<errorCode>402</errorCode>
<errorDescription>Invalid Args</errorDescription>
</UPnPError>
</detail

Is your router coming from a ISP ?

Simone

@chemelli74 chemelli74 changed the title AVM FRITZ!Box Tools - device tracker entities for the Fritz Box and all repeaters are not defined/created since 2022.2.0 AVM FRITZ!Box Tools - WANCommonInterfaceConfig service throws error when called Feb 6, 2022
@browetd
Copy link
Author

browetd commented Feb 9, 2022

Just to add to the discussion that the integration (AVM Fritz!Box Tools) discovered my 4 repeaters... For each repeater, I have 4 button entities (cleanup, firmware update, reboot, reconnect), one sensor (device_uptime) and one binary_sensor(firmware update) but no device_tracker neither... Each repeater has an IP address...

@Smandurlo
Copy link

Smandurlo commented Feb 9, 2022 via email

@chemelli74
Copy link
Contributor

Just to add to the discussion that the integration (AVM Fritz!Box Tools) discovered my 4 repeaters... For each repeater, I have 4 button entities (cleanup, firmware update, reboot, reconnect), one sensor (device_uptime) and one binary_sensor(firmware update) but no device_tracker neither... Each repeater has an IP address...

IF the repeater is part of a mesh network, then we add teh device trackers only to the master.
If it's not, then the issue can be there: is wrongly detected as slave.
Please open a new issue and share log + diagnostics.

Simone

@browetd
Copy link
Author

browetd commented Feb 10, 2022

@chemelli74 My repeaters are part of a mesh... so as you mentionned, the trackers should then be created by the master...

@Smandurlo
Copy link

Smandurlo commented Feb 10, 2022 via email

@browetd
Copy link
Author

browetd commented Feb 10, 2022

@Smandurlo I understood that the trackers for mesh repeaters will be created at the master level so in my case at the level of my FritzBox 7590... not at the level of each repeater integration...

@Smandurlo
Copy link

Smandurlo commented Feb 11, 2022

Update: I had to switch off the electricity on a section of my house where there is a mesh repeater connected with ethernet. After some minutes, I got the notification of my offline devices as expected, PLUS my fritz mesh repeater that was not discovered anymore. Funny thing is, it works in reverse mode! HA lists it as home. When I switched back on the eletricity, the mesh repeater went not_home.

Out of curiosity, I switched off again, and it went "home". Switched on electricity and "not_home". The other device trackers work as supposed to be.

@erkr
Copy link

erkr commented Feb 12, 2022

My two cents: I only have a FritzBox 5490 as a router, no mesh setup.
I only use the FritzBox device trackers for tracking mobile phones.
What I observed is simple: all devices that are not connected to the network after a HA restart, are reported unavailable, until their state changes! So once the mobile re-connects to the network, the state changes from unavailable to home. After leaving the network, the state changes to not_home.
I think the integration should initialize the device trackers that are not connected according Fritz with the not_home state after each reboot.

@Smandurlo
Copy link

I think the integration should initialize the device trackers that are not connected according Fritz with the not_home state after each reboot.

They do it as far as I know. At least, it was this way. Now I see they are not created at all even if the Fritz has the ip reserved (second problem) and new devices create a device_tracker that is active despite the option not to track new device discovered (third problem). Are you sure they are unavailable? Because it will be another problem and in my opinion now there are too much problems to fix and it is better to revert back.

@chemelli74
Copy link
Contributor

My two cents: I only have a FritzBox 5490 as a router, no mesh setup. I only use the FritzBox device trackers for tracking mobile phones. What I observed is simple: all devices that are not connected to the network after a HA restart, are reported unavailable, until their state changes! So once the mobile re-connects to the network, the state changes from unavailable to home. After leaving the network, the state changes to not_home. I think the integration should initialize the device trackers that are not connected according Fritz with the not_home state after each reboot.

Please don't mix up issues. You are reporting something different than the topic.
Anyway, recheck with latest HA, we fixed your problem. If not, open a new issue.

Simone

@erkr
Copy link

erkr commented Feb 12, 2022

...
Anyway, recheck with latest HA, we fixed your problem. If not, open a new issue.

Simone

I checked against the todays release 2022.2.6 and the problem still persists.
I can't judge that it's a different issue. So I indeed opened a new one:
#66402 (comment)

@bcutter
Copy link

bcutter commented Feb 13, 2022

Still tracker entities for Fritz!Box, Fritz!Repeaters and devices not being connected recently are shown with entity state unavailable.

  • The ones not being connected recently should have not_home
  • The ones currently being connected (Fritz!Box and Fritz!Repeater devices) should have home

@chemelli74
Copy link
Contributor

#66402 (comment)

@chemelli74
Copy link
Contributor

The beta of 2022.3.0 (currently 2022.3.0b2) has a new integration system option:

image

Please enable it and report back if issue is fixed.

Simone

@bcutter
Copy link

bcutter commented Feb 25, 2022

Thank you. Does "old integration option" mean
A) ignoring new mesh approach
B) using the very old known_devices.yaml approach
C) ...?

@NullEnt1ty
Copy link

Please enable it and report back if issue is fixed.

@chemelli74 I can confirm that all of my devices showed up after using "Enable old discovery method" 🎉

Thanks for your work, Simone!

@chemelli74
Copy link
Contributor

Thank you. Does "old integration option" mean A) ignoring new mesh approach B) using the very old known_devices.yaml approach C) ...?

Option A, back to the same code used for 2021.12.x
known_devices.yaml are deprecated in HomeAssistant.

Simone

@chemelli74
Copy link
Contributor

@browetd, can you please test latest release 2022.3.0 ?
Please remember the "old discovery option" in case you need it.

Simone

@Smandurlo
Copy link

my (last) test: #67543

@kongo09
Copy link

kongo09 commented Mar 3, 2022

I just tried it. Obviously, I deleted the "test" code from custom_components first, but HA now keeps insisting that this is a custom component, as indicated by the little logo on the top right:

image

The good news is: all entities come back once the switch for the old discovery option is enabled.

The bad news is: the devices still have cumbersome names with the term internet access added to each and every device. But I guess that's part of #65902 which for me is not fixed

@Smandurlo
Copy link

I tried to test the new version as well. I removed the files inside the custom_component dir, removed the yaml code and restarted HA before configuring the gui integration. It didn't complain of custom_component, but it is still plenty of bugs: #67543 (but @chemelli74 closed it without a solution). Unfortunately I cannot continue to test the integration in a production environment.

@kongo09
Copy link

kongo09 commented Mar 3, 2022

HA now keeps insisting that this is a custom component,

This might actually be related to something else on my setup. I'll take a deeper look into this and will report back here. Might actually not be a fritz problem at all.

@kongo09
Copy link

kongo09 commented Mar 3, 2022

Ok, now the integration loads properly. Using the "old" discovery option, everything is detected. The cumbersome naming is gone as well.

thanks!

@Smandurlo
Copy link

do you have working switches and binary_sensors as well?

@kongo09
Copy link

kongo09 commented Mar 3, 2022

do you have working switches and binary_sensors as well?

I've now spent quite a bit of time with the integration. Yes, the binary sensors and the switches seem to work as expected. All is well. Actually, @chemelli74 was very helpful with some debugging so we worked together on this via Discord. I first thought the integration was still bugged, but it turns out that I misconfigured something else (my shellies) which had the side effect of HA not starting properly, which again didn't load the new fritz integration properly.

From my perspective, this can be closed now. Thanks to all those who worked on this integration!

@browetd
Copy link
Author

browetd commented Mar 4, 2022

I am closing this issue ... If you have still specific problems please open a new issue...

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests