Skip to content

Ignore negative derivative when the input is total_increasing#119141

Merged
emontnemery merged 11 commits intohome-assistant:devfrom
Smankusors:non-negative-derivative
Sep 16, 2024
Merged

Ignore negative derivative when the input is total_increasing#119141
emontnemery merged 11 commits intohome-assistant:devfrom
Smankusors:non-negative-derivative

Conversation

@Smankusors
Copy link
Copy Markdown
Contributor

@Smankusors Smankusors commented Jun 8, 2024

Proposed change

The documentation here says that we can use Utility Meter integration if we have a sensor that periodically resets to zero. But I can't get it work to output changes per second just like the original issue.

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Deprecation (breaking change to happen in the future)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • I have followed the perfect PR recommendations
  • The code has been formatted using Ruff (ruff format homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
  • Untested files have been added to .coveragerc.

To help with the load of incoming pull requests:

Copy link
Copy Markdown

@home-assistant home-assistant bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Smankusors

It seems you haven't yet signed a CLA. Please do so here.

Once you do that we will be able to review and accept this pull request.

Thanks!

@home-assistant
Copy link
Copy Markdown

home-assistant bot commented Jun 8, 2024

Hey there @afaucogney, mind taking a look at this pull request as it has been labeled with an integration (derivative) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of derivative can trigger bot actions by commenting:

  • @home-assistant close Closes the pull request.
  • @home-assistant rename Awesome new title Renames the pull request.
  • @home-assistant reopen Reopen the pull request.
  • @home-assistant unassign derivative Removes the current integration label and assignees on the pull request, add the integration domain after the command.
  • @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the pull request.
  • @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the pull request.

@Smankusors Smankusors marked this pull request as ready for review June 8, 2024 14:44
@Smankusors Smankusors marked this pull request as draft June 8, 2024 15:25
@Smankusors Smankusors force-pushed the non-negative-derivative branch from 5db9c54 to 0be2673 Compare June 8, 2024 15:34
@Smankusors Smankusors marked this pull request as ready for review June 8, 2024 16:58
@elupus
Copy link
Copy Markdown
Contributor

elupus commented Jun 10, 2024

This should not need a configuration option. If we have a STATE_CLASS_TOTAL_INCREASING for the source sensor. We can assume to ignore negative values.

@elupus
Copy link
Copy Markdown
Contributor

elupus commented Jun 10, 2024

What i mean is the derivative sensor should check the state class of the sensor, to know if it should ignore the negative value.

@Smankusors
Copy link
Copy Markdown
Contributor Author

This should not need a configuration option. If we have a STATE_CLASS_TOTAL_INCREASING for the source sensor. We can assume to ignore negative values.
What i mean is the derivative sensor should check the state class of the sensor, to know if it should ignore the negative value.

hmm that could work

but how about this Adguard Home sensor? It doesn't have a state class yet

image

this sensor resets every hour, but not to zero, but to... uh... last 24 hours maybe?

will putting total_increasing class in this sensor make sense? 🤔

@elupus
Copy link
Copy Markdown
Contributor

elupus commented Jun 11, 2024

Well that sensor just looks weird. It resets to something strange. Just looks wrong. It should be corrected to some total sensor that reset to zero.

@frenck frenck added the smash Indicator this PR is close to finish for merging or closing label Jun 12, 2024
@Smankusors
Copy link
Copy Markdown
Contributor Author

Well that sensor just looks weird. It resets to something strange. Just looks wrong. It should be corrected to some total sensor that reset to zero.

it seems to be just how the Adguard works I think

https://community.home-assistant.io/t/adguard-home-dns-queries-decreases-hourly/602626

@elupus
Copy link
Copy Markdown
Contributor

elupus commented Jun 15, 2024

Well it should really be fixed in the adguard integration. Ignoring negative in integration integration is the wrong place to fix this.

@Smankusors
Copy link
Copy Markdown
Contributor Author

Well it should really be fixed in the adguard integration. Ignoring negative in integration integration is the wrong place to fix this.

hmm I think I will go ahead with detecting total_increasing from the input sensor instead of setting it manually

in the meantime, for the Adguard Home sensor, is it possible to manually override it from the end user so that the sensor has a total_increasing class?

Copy link
Copy Markdown
Member

@frenck frenck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi there @Smankusors 👋

There is now a merge conflict on this PR (caused by another PR that was merged). Could you take a look?

Thanks! 👍

../Frenck

@frenck frenck marked this pull request as draft June 22, 2024 16:42
@frenck frenck removed the smash Indicator this PR is close to finish for merging or closing label Jun 22, 2024
@Smankusors Smankusors force-pushed the non-negative-derivative branch from 0be2673 to 1f05e37 Compare July 1, 2024 15:36
@Smankusors Smankusors changed the title Ignore negative derivative feature Ignore negative derivative when the input is total_increasing Jul 1, 2024
@Smankusors
Copy link
Copy Markdown
Contributor Author

Smankusors commented Jul 1, 2024

in the meantime, for the Adguard Home sensor, is it possible to manually override it from the end user so that the sensor has a total_increasing class?

I figured that I could add this to the customize.yaml file, and it works 😀

sensor.adguard_home_dns_queries:
  state_class: total_increasing

I updated the PR now so that it now reads if the sensor is total_increasing instead of configuring it manually. The PR now much smaller than before 😀
wdyt? @elupus


also I updated the docs over home-assistant/home-assistant.io#33152. But I wonder if I should tell the users to fix themselves if their sensor doesn't have total_increasing?

@Smankusors Smankusors marked this pull request as ready for review July 1, 2024 16:02
state = hass.states.get("sensor.power")
assert state is not None

assert round(float(state.state), config["sensor"]["round"]) == 2.00
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of just the magic number 2.00, calculate it or add a comment explaining it

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at the second thought...

  • I don't want to calculate it because it would duplicate the implementation in the unit tests. I want the tests to be calculation-free as much as possible so that they're independent of the algorithm.
  • There's no need to comment because the test is already clear. My existing comment isn't unique to this unit test but applies to the entire component. The existing web documentation pointing to Wikipedia should suffice.

Therefore, I will remove this comment in the next commit. Wdyt?

Comment on lines +360 to +361
times = [20, 30, 40]
values = [10, 30, 0]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's extend with another sample to test the behavior of the state update after a discarded sample

Also remove the last comment because the test is already clear
My existing comment there isn't unique to this unit test but applies to the entire component. The existing web documentation pointing to Wikipedia should suffice.
Copy link
Copy Markdown
Contributor

@emontnemery emontnemery left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @Smankusors 👍

@emontnemery emontnemery merged commit 156a88a into home-assistant:dev Sep 16, 2024
bealex pushed a commit to DevPocket/homeassistant-core that referenced this pull request Sep 16, 2024
…ssistant#119141)

* if the derivative is negative, ignore it

* add option to ignore the negatives or not

* add tests for a new ignore negative derivative

* add missing description when editing

* rename to ignore_negative_derivative
to increase clarity of which negative I mean
in case in the future we want a ignore_negative_value...

* use state_class=total_increasing to ignore the negative derivative

* remove ignore negative from the config

* add test for total_increasing_reset case

* add comments

* update test_total_increasing_reset with history tests
Also remove the last comment because the test is already clear
My existing comment there isn't unique to this unit test but applies to the entire component. The existing web documentation pointing to Wikipedia should suffice.

---------

Co-authored-by: Erik Montnemery <erik@montnemery.com>
@github-actions github-actions bot locked and limited conversation to collaborators Sep 17, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Derivative integration - negative values from total_increasing class entity sensor

4 participants