Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate Aurora_ABB_Powerone to DataUpdateCoordinator #72363

Merged
merged 25 commits into from Nov 22, 2023

Conversation

davet2001
Copy link
Contributor

@davet2001 davet2001 commented May 23, 2022

Breaking change

Should be no breaking changes.

Proposed change

The existing integration makes separate calls to the serial port for each sensor on demand. This PR moves those to a DataUpdateCoordinator in line with other integrations. This should improve responsiveness of the integration.

Update 13/5/2023:
It seems that problem #92985 (comms failure) beginning in 2023.5.x can be resolved by this change.

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • The code has been formatted using Black (black --fast homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
  • Untested files have been added to .coveragerc.

The integration reached or maintains the following Integration Quality Scale:

  • No score or internal
  • 🥈 Silver
  • 🥇 Gold
  • 🏆 Platinum

To help with the load of incoming pull requests:

Copy link
Contributor

@austinmroczek austinmroczek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good. Request a few quick changes.

Dev automation moved this from By Code Owner to Review in progress Oct 9, 2022

except AuroraTimeoutError:
self.data = {}
self.available = False
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you migrate to using CoordinatorEntity as a base class you don't need to worry about available anymore.

raise UpdateFailed if something goes wrong and everything will automatically go unavailable

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have done some experiments with this, and I'm not happy with the result when raising UpdateFailed.

The problem is that UpdateFailed logs an error every time, but going offline during the night is not an error. I don't want to fill the user's logs with errors that are impossible to prevent. By explicitly detecting online/offline I can manage what generates an error and also only show log messages when the device comes back online, even though it is trying 100s of times in the night.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it changed in all those months, but I just checked the code of the DataUpdateCoordinator and there is a mechanism for this:

        except UpdateFailed as err:
            self.last_exception = err
            if self.last_update_success:
                if log_failures:
                    self.logger.error("Error fetching %s data: %s", self.name, err)
                self.last_update_success = False

Which is doing exactly what you're doing right now

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get what you are saying, but I'm trying to do something very specific, which is to catch the case of a timeout error, then display a single warning message on the first instance.

I think the automatic route only gives me the possibility to suppress all failure logging which isn't what I want.

In addition, this particular device is sometimes unreliable particularly in low light, so I'm planning a follow up PR that does some auto-retrying, see davet2001@f745523 and #82983

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not talking about the log_failures but about the last_update_success. If you trigger an UpdateFailed, it will log an error, and then it will set last_update_success to False, which is checked before logging.

So in practice, if UpdateFailed is raised 5 times, 1 error is written. It won't log it again, until it actually does an update without a raised error (so it succeeded).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, they are doing very similar things. I did a quick experiment, changing the code to raise UpdateFailed on timeout. It works as you describe. But here is the log output:

Existing code:

2023-08-25 11:30:04.467 WARNING (SyncWorker_2) [homeassistant.components.aurora_abb_powerone] Communication with aurora_abb_powerone lost
2023-08-25 11:35:06.608 INFO (SyncWorker_4) [homeassistant.components.aurora_abb_powerone] Communication with aurora_abb_powerone back online

Using UpdateFailed to manage offline/online messages:

2023-08-25 11:26:35.351 ERROR (MainThread) [homeassistant.components.aurora_abb_powerone] Error fetching aurora_abb_powerone data: No response after 3 tries
2023-08-25 11:28:13.427 INFO (MainThread) [homeassistant.components.aurora_abb_powerone] Fetching aurora_abb_powerone data recovered

They are almost identical, but the key point: going offline due to sunset is not an error. So I want a way of this occurring without raising anything red in the logs (only info/warning). I can't see any other way of doing this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay you got my point, now I know we're on the same wavelength.

Is there a way in the code you can differentiate on if it's because of a failure or if it's because of sunset?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately none that I know of, unplugging the cable behaves the same as sunset (both cause a timeout).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, I still think this is a difficult situation. imo, posting something in the logs doesn't have to be seen as something bad, just an FYI. I think we can just use the normal way like every integration does, but just have to teach the user a bit more about how it works via the documentation or the error raised. You could edit the raised error like, "Lost connection to inverter, could it be dark?". We have way more integrations that just log every time the service is responds with a 500 while the next call just succeeds.

I think it would be good to have a third opinion on this matter. Like I get where you come from, but I am just wondering what's the right solution here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@joostlek I don't agree. If there are warning or error prints users will create issues and complain about it. In this case, what could be considered is to log with info level since debug prints are usually very spammy.

I think the PR is good to go as is though, adjusting to info level can be done in a follow-up.

@davet2001 davet2001 force-pushed the aurora_dataupdatecoordinator branch from d1ede77 to 35119c9 Compare April 6, 2023 16:23
@bdraco bdraco self-requested a review April 7, 2023 21:30
@davet2001 davet2001 force-pushed the aurora_dataupdatecoordinator branch 2 times, most recently from b438de0 to 32cbb19 Compare May 8, 2023 09:08
@davet2001 davet2001 marked this pull request as ready for review August 25, 2023 09:22
Copy link
Member

@joostlek joostlek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

imo, let's take a look at the DataUpdateCoordinator, since I think that the stuff you implemented here is already in there, which could make this way less complicated. But furthermore I think it's almost there :)

homeassistant/components/aurora_abb_powerone/sensor.py Outdated Show resolved Hide resolved
@home-assistant home-assistant bot marked this pull request as draft August 25, 2023 09:39
Co-authored-by: Joost Lekkerkerker <joostlek@outlook.com>
@davet2001 davet2001 marked this pull request as ready for review August 25, 2023 10:01
homeassistant/components/aurora_abb_powerone/sensor.py Outdated Show resolved Hide resolved
homeassistant/components/aurora_abb_powerone/sensor.py Outdated Show resolved Hide resolved
homeassistant/components/aurora_abb_powerone/sensor.py Outdated Show resolved Hide resolved
tests/components/aurora_abb_powerone/test_sensor.py Outdated Show resolved Hide resolved

except AuroraTimeoutError:
self.data = {}
self.available = False
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, I still think this is a difficult situation. imo, posting something in the logs doesn't have to be seen as something bad, just an FYI. I think we can just use the normal way like every integration does, but just have to teach the user a bit more about how it works via the documentation or the error raised. You could edit the raised error like, "Lost connection to inverter, could it be dark?". We have way more integrations that just log every time the service is responds with a 500 while the next call just succeeds.

I think it would be good to have a third opinion on this matter. Like I get where you come from, but I am just wondering what's the right solution here.

homeassistant/components/aurora_abb_powerone/sensor.py Outdated Show resolved Hide resolved
@home-assistant home-assistant bot marked this pull request as draft September 30, 2023 14:26
@joostlek joostlek added the second-opinion-wanted Add this label when a reviewer needs a second opinion from another member. label Sep 30, 2023
Copy link
Contributor

@austinmroczek austinmroczek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Let's get this year+ PR over the finish line. Thanks for hanging in there @davet2001

@davet2001
Copy link
Contributor Author

@joostlek Thanks for the review. Points should all be addressed now.

Regarding the logs, I do feel that logs and warnings should only occur if something is happening that shouldn't happen. If nothing else, unnecessary logs distract when debugging a real problem.

e.g. my current setup is generating the following error approx twice per second continuously. I treat this as something that should be fixed rather than lived with.

9:43:24 PM – (ERROR) /usr/local/lib/python3.11/site-packages/aiohttp/web_protocol.py - message first occurred at September 10, 2023 at 9:32:19 PM and shows up 4460546 times 

But definitely worth another opinion on this.

@davet2001 davet2001 marked this pull request as ready for review October 5, 2023 21:11
Copy link
Contributor

@emontnemery emontnemery left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @davet2001 👍

@emontnemery emontnemery dismissed stale reviews from edenhaus and joostlek November 22, 2023 18:03

stale

Dev automation moved this from Review in progress to Reviewer approved Nov 22, 2023
@emontnemery emontnemery merged commit 7a727dc into home-assistant:dev Nov 22, 2023
21 checks passed
Dev automation moved this from Reviewer approved to Done Nov 22, 2023
@github-actions github-actions bot locked and limited conversation to collaborators Nov 23, 2023
@davet2001 davet2001 deleted the aurora_dataupdatecoordinator branch November 24, 2023 15:56
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
Dev
  
Done
Development

Successfully merging this pull request may close these issues.

None yet

7 participants