Update get_history in HAHistory to take a copy of the history of an entity to avoid race conditions#3550
Merged
springfall2008 merged 3 commits intospringfall2008:mainfrom Mar 15, 2026
Merged
Conversation
…ntity, to avoid race conditions. get_history previously returned (a list containing) the history stored in HAHistory. This history could then be accessed concurrently with changes being made to it by update_entity, with no locking. At a minimum, because update_entity sorts the history -- which is not thread safe -- the history may appear transiently empty during these updates. This race condition caused glitches elsewhere in the code; at a minimum, with load data appearing to be transiently empty, leading to poor estimates. This commit fixes the issue by making get_history return a *copy* of the history; and to hold the lock while making this copy, to ensure thread safety. To militate against the performance impact of this, minute_data in utils.py is updated to take an extra argument -- can_modify_history, default to False. If set to True, this will stop it from taking a copy it would otherwise make of the history object provided to it. fetch.py is then updated to set it to true in some places, as appropriate. Hence the fix has almost no impact on performance. Fixes springfall2008#3511
Contributor
There was a problem hiding this comment.
Pull request overview
Improves performance and thread-safety around Home Assistant history handling by avoiding unnecessary deep-copies in minute_data() when safe, while ensuring cached history returned from HAHistory.get_history() cannot be mutated concurrently or by callers.
Changes:
- Add
can_modify_historyflag toutils.minute_data()to optionally skip defensivedeepcopy(). - Make
HAHistory.get_history()return a deep-copied result to avoid exposing internal cached structures across threads. - Opt specific fetch paths into
can_modify_history=Truewhere the history object is not reused after conversion.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| apps/predbat/utils.py | Adds can_modify_history parameter to control whether minute_data() deep-copies input history. |
| apps/predbat/ha.py | Deep-copies history results before returning to improve thread-safety and protect cached history from caller mutation. |
| apps/predbat/fetch.py | Passes can_modify_history=True in a few high-traffic conversions where post-call history reuse doesn’t occur. |
You can also share your feedback on Copilot code review. Take the survey.
Comment on lines
+110
to
112
| result = copy.deepcopy(result) | ||
|
|
||
| return result |
Comment on lines
315
to
319
| max_increment=MAX_INCREMENT, | ||
| interpolate=False, | ||
| debug=False, | ||
| can_modify_history=False, | ||
| ): |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
get_history previously returned (a list containing) the history stored in HAHistory.
This history could then be accessed concurrently with changes being made to it by update_entity, with no locking.
At a minimum, because update_entity sorts the history -- which is not thread safe -- the history may appear transiently empty during these updates.
This race condition caused glitches elsewhere in the code; at a minimum, with load data appearing to be transiently empty, leading to poor estimates.
This commit fixes the issue by making get_history return a copy of the history; and to hold the lock while making this copy, to ensure thread safety.
To militate against the performance impact of this, minute_data in utils.py is updated to take an extra argument -- can_modify_history, default to False. If set to True, this will stop it from taking a copy it would otherwise make of the history object provided to it. fetch.py is then updated to set it to true in some places, as appropriate. Hence the fix has almost no impact on performance.
Fixes #3511 . Tested for two calendar days, see attached screenshot: