CPU usage spikes every 5 minutes

[mniquette]

Describe the issue you are experiencing

Typically my Rpi 4b w/ 4G runs at around 3-4% load. When I upgraded to 2022.4, while the typical load was still 3-4%, I was seeing 10-12% spikes every 5 minutes that seems to correspond with the supervisor.store.git activities listed in the supervisor log. I’m guessing this is the Content Trust checking versions. Is that correct? Is there something wrong with my configuration that is causing such large usage spikes? I've seen nothing in the forum about this and I wrote in the 2022.4 blog topic in the forum but received no comments.

What is the used version of the Supervisor?

2022.04.0

What type of installation are you running?

Home Assistant OS

Which operating system are you running on?

Home Assistant Operating System

What is the version of your installed operating system?

7.6

What version of Home Assistant Core is installed?

2022.4.3

Steps to reproduce the issue

1. Nothing special, just normal system operation.
   ...

Anything in the Supervisor logs that might be useful for us?

Here is the supervisor log for two spikes:

22-04-13 15:14:09 INFO (MainThread) [supervisor.updater] Fetching update data from https://version.home-assistant.io/stable.json
22-04-13 15:14:09 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/hassio-addons/repository repository
22-04-13 15:14:09 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/allenporter/stream-addons repository
22-04-13 15:14:09 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/brenner-tobias/ha-addons repository
22-04-13 15:14:09 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/thomasmauerer/hassio-addons repository
22-04-13 15:14:09 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/home-assistant/addons repository
22-04-13 15:14:11 INFO (MainThread) [supervisor.store] Loading add-ons from store: 66 all - 0 new - 0 remove
22-04-13 15:14:11 INFO (MainThread) [supervisor.store] Loading add-ons from store: 66 all - 0 new - 0 remove
22-04-13 15:18:59 INFO (MainThread) [supervisor.homeassistant.api] Updated Home Assistant API token
22-04-13 15:19:13 INFO (MainThread) [supervisor.updater] Fetching update data from https://version.home-assistant.io/stable.json
22-04-13 15:19:13 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/thomasmauerer/hassio-addons repository
22-04-13 15:19:13 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/brenner-tobias/ha-addons repository
22-04-13 15:19:13 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/home-assistant/addons repository
22-04-13 15:19:13 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/allenporter/stream-addons repository
22-04-13 15:19:13 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/hassio-addons/repository repository
22-04-13 15:19:29 INFO (MainThread) [supervisor.store] Loading add-ons from store: 66 all - 0 new - 0 remove
22-04-13 15:19:29 INFO (MainThread) [supervisor.store] Loading add-ons from store: 66 all - 0 new - 0 remove

Additional information

No response

[AdmiralStipe]

Same here, although it's not so much CPU spikes bothering me, but the fact, that Supervisor checks for new updates every five minutes (as I can see from the log, it's the same in your case). That was asked/mentioned on the forum, but noone knew what's it all about. Is it really necessary for Supervisor to check for updates so often (once or maybe twice per day seems enough to me)?

[mniquette]

I don't think it's checking for updates as that was done before 2022.4 without the every 5 minute checks. I think this is determining if the add-ons, etc running on the system are the official versions (Content Trust feature). I'm not sure why this would need to be done every five minutes though. It seems like at system startup and maybe updates would be good times to run the check. Perhaps I'm not fully understanding the issue.

[mdegat01]

No its checking for updates. I'm trying to figure out why now. Doesn't make sense because here's our timings for update and reload tasks

supervisor/supervisor/misc/tasks.py

Lines 22 to 34 in 8c528f7

	RUN_UPDATE_SUPERVISOR = 29100
	RUN_UPDATE_ADDONS = 57600
	RUN_UPDATE_CLI = 28100
	RUN_UPDATE_DNS = 30100
	RUN_UPDATE_AUDIO = 30200
	RUN_UPDATE_MULTICAST = 30300
	RUN_UPDATE_OBSERVER = 30400
	RUN_RELOAD_ADDONS = 10800
	RUN_RELOAD_BACKUPS = 72000
	RUN_RELOAD_HOST = 7600
	RUN_RELOAD_UPDATER = 7200
	RUN_RELOAD_INGRESS = 930

You can see they're all like multiple hours. Something is speeding it up and I'm trying to find out what.

[mdegat01]

It looks like its the hassio component in core
https://github.com/home-assistant/core/blob/c85387290a488ca70bf21378a0913f92d4ec50bf/homeassistant/components/hassio/__init__.py#L104

It must be calling an API every 5 minutes that is triggering an update check

[jasoncodes]

As a temporary mitigation I’ve disabled “Enable polling for updates” on the Supervisor integration which seems to have quietened things down.

[AdmiralStipe]

It looks like its the hassio component in core https://github.com/home-assistant/core/blob/c85387290a488ca70bf21378a0913f92d4ec50bf/homeassistant/components/hassio/__init__.py#L104

It must be calling an API every 5 minutes that is triggering an update check

So, can we (plain users) do anything about it? Or we just wait for it to be fixed in the core?

[mdegat01]

@AdmiralStipe you could try disabling the hassio integration I suppose but that might have other problems, not sure. Unfortunately it's poll time isn't configurable currently.

However a PR has been opened in core to address this home-assistant/core#70075 . So it should be resolved in the next patch.

[AdmiralStipe]

@mdegat01, great, thx, I will manually update the hassio files from the PR for testing in advance.

[AdmiralStipe]

Just FIY to anyone affected by this issue: I replaced the file from the mentioned PR in my local HA isntance and it works - no more 5 minute checks / CPU spikes.
Your mileage may vary, so I suggest either testing the PR manually or waiting for the official patch.

[pashdown]

As a temporary mitigation I’ve disabled “Enable polling for updates” on the Supervisor integration which seems to have quietened things down.

Unfortunately this did nothing for me. I'm still at 90%+ CPU usage on a NUC. Hope the patch hits soon, as this is affecting some sensors with delays.

[mdegat01]

FYI 2022.4.5 out now with the mentioned fix for the hassio component

[mniquette]

Yes, just installed it 30 minutes ago and the spikes have disappeared. Thanks