Upgraded HASS to 2024.7 getting lots of error logs #349

cweakland · 2024-07-03T17:49:33Z

These look like they may be icloud3 related, I was hoping someone my be able help me resolve this. I did not see these prior to the upgrade. Here is the log in question from my homeassistant.log, I get one every few seconds.

2024-07-03 13:47:04.948 ERROR (MainThread) [homeassistant.core] Error running job: <Job listen * HassJobType.Callback <function Recorder.async_initialize.._event_listener at 0x7fc27ff76480>>
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/core.py", line 1559, in async_fire_internal
self._hass.async_run_hass_job(job, event)
File "/usr/src/homeassistant/homeassistant/core.py", line 938, in async_run_hass_job
hassjob.target(*args)
File "/usr/src/homeassistant/homeassistant/components/recorder/core.py", line 330, in _event_listener
if entity_filter(entity_id):
^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/icloud3/support/recorder_prefilter.py", line 123, in entity_filter
return recorder_entity_filter(entity_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable

kksligh · 2024-07-03T20:10:49Z

same - its literally crashing my instance because of the amount of errors

cweakland · 2024-07-04T02:11:34Z

same - its literally crashing my instance because of the amount of errors

Yep, I had to roll back to last months release, bummer.

gcobb321 · 2024-07-04T02:48:35Z

I am away so I can not issue an update but try this.

Edit the config/custom_components/cloud3/support/recorder_prefilter.py file. insert the following at line 122 before the line causing the error at line 123.

        if entity_id is None: return False

The code should line up with the if entity_id line on 119.
Then restart HA

kksligh · 2024-07-04T03:30:54Z

unfortunately, I literally had to delete it within 30-60 seconds of booting up or it would crash my instance. Not sure why because I have it on a very powerful NUC. It would lock up and when I looked at the logs, it was throwing errors 3000+ times within seconds of booting up - I had several devices so maybe thats it. I'll wait for an official update before reinstalling - love the work!

gcobb321 · 2024-07-04T03:41:03Z

I’m actually in Switzerland on a trip and do not want to update HA while I’m away. That never ends well. The HA guys must have made some changes in 2024.7 that did something. That module injects code into the HA recorder module to be able to stop logging entity changes for the Event Log, info and other fields.

kksligh · 2024-07-04T03:43:29Z

here's what someone responded with....

stickpin · 2024-07-04T08:23:48Z

The problem is recorder_entity_filter itself and not an entity_id.

The temporary fix is actually to add this line:
if recorder_entity_filter is None: return False in the config/custom_components/cloud3/support/recorder_prefilter.py file.
The code should look like this:

    def entity_filter(entity_id):
        """
        Prefilter an entity to see if it should be excluded from
        the recorder history.

        This function is injected into the recorder, replacing the
        original HA recorder_entity_filter module.

        Return:
            False - The entity should is in the filter list
            Run the original HA recorder_entity_filter function -
                The entity is not in the filter list.
        """
        if (entity_id
                and entity_id in hass.data['recorder_prefilter']['exclude_entities']):
            return False
        if recorder_entity_filter is None: return False
        return recorder_entity_filter(entity_id)

gcobb321 · 2024-07-04T08:25:49Z

I am in Switzerland and cannot release an update to iCloud3 to fix this error. However, I have updated the code in GitHub.

Download the support/recorder_prefilter.py file in the iCloud3 GitHub repository here and save it to the icloud3/support directory. (Look for ‘Raw’ in the menu bar above the code on the right. The Download icon is the 2nd icon to the right)
Restart HA.

gcobb321 · 2024-07-04T08:34:04Z

@stickpin
Thanks. I just added your code to recorder_prefilter.py

cweakland · 2024-07-04T11:42:20Z

Thank you both!

stickpin · 2024-07-04T15:15:30Z

@cweakland i think, it makes sense to reopen this bug.
After the change implemented by @gcobb321 your recorder is still broken.

For the time being, I've completely commented _inject_filter in lines 82, 83.

    if hass.data.get('recorder_prefilter') is None:
        rp_data = hass.data['recorder_prefilter'] = {}
        rp_data['injected'] = True
        rp_data['legacy'] = True
        rp_data['exclude_entities'] = []

        try:
            ha_recorder.entity_filter._exclude_e.add(entities)
            return True
        except:
            pass

        rp_data['legacy'] = False

        #if _inject_filter(hass) is False:
        #    return

I will see if I'll have some time today to debug it properly.

cweakland · 2024-07-04T15:42:01Z

By recorder, do you mean the "logbook"? If so, yes, my logbook is pretty empty.

cweakland · 2024-07-04T16:02:14Z

@stickpin The workaround you recommended brought my logbook back to life.

gcobb321 · 2024-07-04T18:05:12Z

I have updated the recorder_filter.py module in the GitHub repositories to comment out the inject filter code suggested very @stickpin

@Snuffy2
Have you been following this?

Snuffy2 · 2024-07-04T18:21:06Z

Nope, but I am now :-) I linked a similar/same issue from another repo where I used recorder_history_prefilter. Interestingly, I'm not seeing the error in my logs and I'm running 3 integrations with it. Will investigate more.

ahamro · 2024-07-04T19:42:42Z

@gcobb321 wrote:

Download the support/recorder_prefilter.py file in the iCloud3 GitHub repository here and save it to the icloud3/support directory. (Look for ‘Raw’ in the menu bar above the code on the right. The Download icon is the 2nd icon to the right) Restart HA.

Thanks for the quick solution. It worked like a charm.

Enjoy your trip!

Cheers,
-Ton.

uandco · 2024-07-04T22:42:39Z

Confirming replacing the file and restarting HA fixes the recorder issue.

thenoid · 2024-07-05T22:58:38Z

homeassistant  | 2024-07-05 16:57:05.946 WARNING (Recorder) [homeassistant.components.recorder.db_schema] State attributes for sensor.icloud3_event_log exceed maximum size of 16384 bytes. This can cause database performance issues; Attributes will not be stored

after applying the patch, this is the new error of fun.

gcobb321 · 2024-07-05T23:11:36Z

Ignore I it. This is a problem a few of us are working on. I’m in Switzerland right now and have not upgraded my R pi to 2024.7 since in away.

swoop124 · 2024-07-08T09:26:04Z

Hello,

i've got the same errors after Update to 2024.7.0 then i rolled back to 2024.6.4 and tried to update to 2024.7.1 but got the same errors:

2024-07-04 13:45:19.970 ERROR (MainThread) [homeassistant.core] Error running job: <Job listen * HassJobType.Callback <function Recorder.async_initialize.<locals>._event_listener at 0x7fa7b5253600>>
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/core.py", line 1559, in async_fire_internal
    self._hass.async_run_hass_job(job, event)
  File "/usr/src/homeassistant/homeassistant/core.py", line 938, in async_run_hass_job
    hassjob.target(*args)
  File "/usr/src/homeassistant/homeassistant/components/recorder/core.py", line 330, in _event_listener
    if entity_filter(entity_id):
       ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/icloud3/support/recorder_prefilter.py", line 123, in entity_filter
    return recorder_entity_filter(entity_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/core.py", line 1559, in async_fire_internal
    self._hass.async_run_hass_job(job, event)
  File "/usr/src/homeassistant/homeassistant/core.py", line 938, in async_run_hass_job
    hassjob.target(*args)
  File "/usr/src/homeassistant/homeassistant/components/recorder/core.py", line 330, in _event_listener
    if entity_filter(entity_id):
       ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/icloud3/support/recorder_prefilter.py", line 123, in entity_filter
    return recorder_entity_filter(entity_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable
2024-07-04 13:45:19.971 ERROR (MainThread) [homeassistant.core] Error running job: <Job listen * HassJobType.Callback <function Recorder.async_initialize.<locals>._event_listener at 0x7fa7b5253600>>
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/core.py", line 1559, in async_fire_internal
    self._hass.async_run_hass_job(job, event)
  File "/usr/src/homeassistant/homeassistant/core.py", line 938, in async_run_hass_job
    hassjob.target(*args)
  File "/usr/src/homeassistant/homeassistant/components/recorder/core.py", line 330, in _event_listener
    if entity_filter(entity_id):
       ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/icloud3/support/recorder_prefilter.py", line 123, in entity_filter
    return recorder_entity_filter(entity_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable
2024-07-04 13:45:19.972 ERROR (MainThread) [homeassistant.core] Error running job: <Job listen * HassJobType.Callback <function Recorder.async_initialize.<locals>._event_listener at 0x7fa7b5253600>>
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/core.py", line 1559, in async_fire_internal
    self._hass.async_run_hass_job(job, event)
  File "/usr/src/homeassistant/homeassistant/core.py", line 938, in async_run_hass_job
    hassjob.target(*args)
  File "/usr/src/homeassistant/homeassistant/components/recorder/core.py", line 330, in _event_listener
    if entity_filter(entity_id):
       ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/icloud3/support/recorder_prefilter.py", line 123, in entity_filter
    return recorder_entity_filter(entity_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable

any news?

gcobb321 · 2024-07-08T13:18:15Z

I am in Switzerland and cannot release an update to iCloud3 to fix this error. I also cannot access my RPi server. However, I have updated the code in GitHub.

Download the support/recorder_prefilter.py file in the iCloud3 GitHub repository here and save it to the icloud3/support directory. (Look for ‘Raw’ in the menu bar above the code on the right. The Download icon is the 2nd icon to the right)
Restart HA.

This is a temporary fix until some updates are done. You may lose some HA log history until this is resolved. I am involved in some discussions on changes to HA that will handle this issue

other than that, there is no more news.

smarthomefamilyverrips · 2024-07-10T10:41:41Z

@gcobb321 will this home-assistant/core#121658 mean the whole icloud3 integration stop working if update to HA 2024.7.2? Or it will only stop storing data in recorder?

u20p17 · 2024-07-10T12:26:13Z

yes, if you update to 2024.7.2 the icloud3 integration will not get loaded... i just restored to 2024.7.1 with the temporary fix mentioned above...

smarthomefamilyverrips · 2024-07-10T12:39:39Z

yes, if you update to 2024.7.2 the icloud3 integration will not get loaded... i just restored to 2024.7.1 with the temporary fix mentioned above...

@u20p17 thanks for your reply, I will just stay on 2024.6.4 then till Gary is back from vacation and had time to apply a permanent fix 👍

broyuken · 2024-07-10T15:45:02Z

This only affects icloud3 correct? Or does it also break icloud3_v3? I haven’t updated to 2024.7.2 yet and am hesitant to now after seeing those release notes.

gcobb321 · 2024-07-10T16:01:38Z

iCloud3 inserts special code into the HA recorder that automatically excludes some text based iCloud3 sensors from the HA log database. HA release 3024.7 broke this and 2024.7.2 prevented iCloud3 from loading if it tried to do this injection.

I am in Switzerland and cannot release an update to iCloud3 to fix this error. However, I have updated the code in GitHub.

Download the support/recorder_prefilter.py file in the iCloud3 GitHub repository here and save it to the icloud3/support directory. (Look for ‘Raw’ in the menu bar above the code on the right. The Download icon is the 2nd icon to the right)
Restart HA.

The Event Log also adds a large item into the log and will generate a warning message in the Error Log. To stop the warning message and exclude this sensor from being added, add the following to the configuration.yaml file. It adds a manual exclude for sensor.

recorder:
  exclude:
    entities:
      - sensor.icloud3_event_log

smarthomefamilyverrips · 2024-07-10T16:05:39Z

This only affects icloud3 correct? Or does it also break icloud3_v3? I haven’t updated to 2024.7.2 yet and am hesitant to now after seeing those release notes.

From how I understand it affects as well iCloud3 as iCloud3_V3 anyone correct me if I am wrong

gcobb321 · 2024-07-10T16:06:41Z

Both iCloud3 and iCloud3 _v3

broyuken · 2024-07-10T16:25:31Z

Got it, I'll hold off on updating then until you get back. No rush, but when are you back just so we have a rough idea?

gcobb321 · 2024-07-10T16:30:10Z

In 3-days. I should have an update out by 7/15. It will disable the HA special patch with instructions until the issue is resolved.

thenoid · 2024-07-10T16:36:23Z

Sounds like we need to get Santa (Gary) some elves to release things when he's taking a break.

gcobb321 · 2024-07-10T17:31:15Z

My next trip is Lisbon to Cape Town on the west coast of Africa from 11/10 to 12/7. Hopefully the HA guys won’t break iCloud3 on the 2024.12 update.

broyuken · 2024-07-10T18:32:12Z

Must be nice to be retired :)

nebulous · 2024-07-11T14:29:14Z

Adding backreference to the hass community side of this:

https://community.home-assistant.io/t/psa-2024-7-recorder-problems/746428

cweakland closed this as completed Jul 4, 2024

cweakland reopened this Jul 4, 2024

gcobb321 mentioned this issue Jul 4, 2024

HA 2024.7.0 Recorder Error gcobb321/icloud3_v3#308

Open

This was referenced Jul 4, 2024

Error running job after upgrade HA to 2024.7 enkama/hass-variables#120

Closed

recorder_prefilter.py: recorder_entity_filter(entity_id): TypeError: 'NoneType' object is not callable gcobb321/recorder_history_prefilter#3

Open

LioBoul mentioned this issue Jul 5, 2024

Energy Dashboard not work anymore home-assistant/core#121282

Closed

reisueber mentioned this issue Jul 6, 2024

History of entities is broken when icloud3 is installed home-assistant/core#121214

Open

doctorkb mentioned this issue Jul 10, 2024

HA doesn't seem to store new values in history twrecked/hass-virtual#111

Closed

frenck mentioned this issue Jul 10, 2024

Block icloud3 custom integration from breaking the recorder home-assistant/core#121658

Merged

19 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgraded HASS to 2024.7 getting lots of error logs #349

Upgraded HASS to 2024.7 getting lots of error logs #349

cweakland commented Jul 3, 2024

kksligh commented Jul 3, 2024

cweakland commented Jul 4, 2024

gcobb321 commented Jul 4, 2024

kksligh commented Jul 4, 2024

gcobb321 commented Jul 4, 2024

kksligh commented Jul 4, 2024

stickpin commented Jul 4, 2024

gcobb321 commented Jul 4, 2024

gcobb321 commented Jul 4, 2024

cweakland commented Jul 4, 2024

stickpin commented Jul 4, 2024

cweakland commented Jul 4, 2024

cweakland commented Jul 4, 2024

gcobb321 commented Jul 4, 2024

Snuffy2 commented Jul 4, 2024

ahamro commented Jul 4, 2024

uandco commented Jul 4, 2024

thenoid commented Jul 5, 2024 •

edited

Loading

gcobb321 commented Jul 5, 2024

swoop124 commented Jul 8, 2024

gcobb321 commented Jul 8, 2024

smarthomefamilyverrips commented Jul 10, 2024

u20p17 commented Jul 10, 2024

smarthomefamilyverrips commented Jul 10, 2024

broyuken commented Jul 10, 2024

gcobb321 commented Jul 10, 2024

smarthomefamilyverrips commented Jul 10, 2024

gcobb321 commented Jul 10, 2024

broyuken commented Jul 10, 2024

gcobb321 commented Jul 10, 2024

thenoid commented Jul 10, 2024

gcobb321 commented Jul 10, 2024

broyuken commented Jul 10, 2024

nebulous commented Jul 11, 2024

Upgraded HASS to 2024.7 getting lots of error logs #349

Upgraded HASS to 2024.7 getting lots of error logs #349

Comments

cweakland commented Jul 3, 2024

kksligh commented Jul 3, 2024

cweakland commented Jul 4, 2024

gcobb321 commented Jul 4, 2024

kksligh commented Jul 4, 2024

gcobb321 commented Jul 4, 2024

kksligh commented Jul 4, 2024

stickpin commented Jul 4, 2024

gcobb321 commented Jul 4, 2024

gcobb321 commented Jul 4, 2024

cweakland commented Jul 4, 2024

stickpin commented Jul 4, 2024

cweakland commented Jul 4, 2024

cweakland commented Jul 4, 2024

gcobb321 commented Jul 4, 2024

Snuffy2 commented Jul 4, 2024

ahamro commented Jul 4, 2024

uandco commented Jul 4, 2024

thenoid commented Jul 5, 2024 • edited Loading

gcobb321 commented Jul 5, 2024

swoop124 commented Jul 8, 2024

gcobb321 commented Jul 8, 2024

smarthomefamilyverrips commented Jul 10, 2024

u20p17 commented Jul 10, 2024

smarthomefamilyverrips commented Jul 10, 2024

broyuken commented Jul 10, 2024

gcobb321 commented Jul 10, 2024

smarthomefamilyverrips commented Jul 10, 2024

gcobb321 commented Jul 10, 2024

broyuken commented Jul 10, 2024

gcobb321 commented Jul 10, 2024

thenoid commented Jul 10, 2024

gcobb321 commented Jul 10, 2024

broyuken commented Jul 10, 2024

nebulous commented Jul 11, 2024

thenoid commented Jul 5, 2024 •

edited

Loading