Amcrest on 2024.5.0 with offline camera causing HA to fail loading completely, got stuck with increased CPU usage

[paludi]

The problem

Updated to 2024.5.0 and 2024.5.1 and noted that HA failed to load competely.
I also noticed an increase in CPU usage (around 10x higher) resulting in CPU usage stabilizing around 30% instead of 3%.

• Rolled back to 2024.4.4 and HA loaded correctly, CP immediately dropped.
• removed the configuration of camera with the Amcrest integration and 2024.5.1 loaded correctly, with CPU being back at 3%.

I have a camera configured that was offline at time of starting HA. (the camera is offline most of the time, I just turn it on when needed)

What version of Home Assistant Core has the issue?

core-2024.5.0 and core-2024.5.1

What was the last working version of Home Assistant Core?

core-2024.4.4

What type of installation are you running?

Home Assistant Container

Integration causing the issue

Amcrest

Link to integration documentation on our website

https://www.home-assistant.io/integrations/amcrest/

Diagnostics information

No response

Example YAML snippet

amcrest:
  - host: 192.168.xxx.xxx
    username: !secret armcrest_camera_user
    password: !secret armcrest_camera_password
    name: "Amcrest Camera"
    binary_sensors:
      - online

Anything in the logs that might be useful for us?

amcrest.exceptions.CommError: HTTPConnectionPool(host='192.168.xxx.xxx', port=80): Max retries exceeded with url: /cgi-bin/magicBox.cgi?action=getMachineName (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6d6bc8c0>: Failed to establish a new connection: [Errno 113] Host is unreachable'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/amcrest/camera.py", line 330, in async_on_demand_update
    self.async_schedule_update_ha_state(True)
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1252, in async_schedule_update_ha_state
    self.hass.async_create_task(
  File "/usr/src/homeassistant/homeassistant/core.py", line 816, in async_create_task
    self.verify_event_loop_thread("async_create_task")
  File "/usr/src/homeassistant/homeassistant/core.py", line 440, in verify_event_loop_thread
    frame.report(
  File "/usr/src/homeassistant/homeassistant/helpers/frame.py", line 162, in report
    _report_integration(what, integration_frame, level, error_if_integration)
  File "/usr/src/homeassistant/homeassistant/helpers/frame.py", line 203, in _report_integration
    raise RuntimeError(
RuntimeError: Detected that integration 'amcrest' calls async_create_task from a thread at homeassistant/components/amcrest/camera.py, line 330: self.async_schedule_update_ha_state(True). Please create a bug report at https://github.com/home-assistant/core/issues?q=is%3Aopen+is%3Aissue+label%3A%22integration%3A+amcrest%22.



additional logs:

    self.connect()
  File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 205, in connect
    conn = self._new_conn()
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 186, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f6d6bc8c0>: Failed to establish a new connection: [Errno 113] Host is unreachable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 799, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='192.168.xxx.xxx', port=80): Max retries exceeded with url: /cgi-bin/magicBox.cgi?action=getMachineName (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6d6bc8c0>: Failed to establish a new connection: [Errno 113] Host is unreachable'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/amcrest/http.py", line 305, in _command
    resp = session.get(
           ^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get
    return self.request("GET", url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='192.168.xxx.xxx', port=80): Max retries exceeded with url: /cgi-bin/magicBox.cgi?action=getMachineName (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6d6bc8c0>: Failed to establish a new connection: [Errno 113] Host is unreachable'))
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/amcrest/__init__.py", line 193, in command
    ret = super().command(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/amcrest/http.py", line 253, in command
    self._generate_token()
  File "/usr/local/lib/python3.12/site-packages/amcrest/http.py", line 150, in _generate_token
    resp = self._command(cmd).content.decode()
           ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/amcrest/http.py", line 327, in _command
    raise CommError(error) from error
amcrest.exceptions.CommError: HTTPConnectionPool(host='192.168.xxx.xxx', port=80): Max retries exceeded with url: /cgi-bin/magicBox.cgi?action=getMachineName (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6d6bc8c0>: Failed to establish a new connection: [Errno 113] Host is unreachable'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/amcrest/camera.py", line 330, in async_on_demand_update
    self.async_schedule_update_ha_state(True)
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1252, in async_schedule_update_ha_state
    self.hass.async_create_task(
  File "/usr/src/homeassistant/homeassistant/core.py", line 816, in async_create_task
    self.verify_event_loop_thread("async_create_task")
  File "/usr/src/homeassistant/homeassistant/core.py", line 440, in verify_event_loop_thread
    frame.report(
  File "/usr/src/homeassistant/homeassistant/helpers/frame.py", line 162, in report
    _report_integration(what, integration_frame, level, error_if_integration)
  File "/usr/src/homeassistant/homeassistant/helpers/frame.py", line 203, in _report_integration
    raise RuntimeError(
RuntimeError: Detected that integration 'amcrest' calls async_create_task from a thread at homeassistant/components/amcrest/camera.py, line 330: self.async_schedule_update_ha_state(True). Please create a bug report at https://github.com/home-assistant/core/issues?q=is%3Aopen+is%3Aissue+label%3A%22integration%3A+amcrest%22.
2024-05-05 11:08:47.681 WARNING (Amcrest Amcrest Camera) [py.warnings] /usr/src/homeassistant/homeassistant/util/logging.py:110: RuntimeWarning: coroutine 'Entity.async_update_ha_state' was never awaited
  log_exception(format_err, *args)
2024-05-05 11:08:47.681 WARNING (Amcrest Amcrest Camera) [homeassistant.components.amcrest] Error while processing events from Amcrest Camera camera: CommError(ConnectionError(MaxRetryError("HTTPConnectionPool(host='192.168.xxx.xxx', port=80): Max retries exceeded with url: /cgi-bin/magicBox.cgi?action=getMachineName (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6d6bc8c0>: Failed to establish a new connection: [Errno 113] Host is unreachable'))")))
2024-05-05 11:08:52.625 WARNING (MainThread) [homeassistant.helpers.entity] Update of camera.amcrest_camera is taking over 10 seconds
2024-05-05 11:08:57.624 WARNING (MainThread) [homeassistant.components.camera] Updating amcrest camera took longer than the scheduled update interval 0:00:15
2024-05-05 11:09:12.625 WARNING (MainThread) [homeassistant.components.camera] Updating amcrest camera took longer than the scheduled update interval 0:00:15
2024-05-05 11:09:27.625 WARNING (MainThread) [homeassistant.components.camera] Updating amcrest camera took longer than the scheduled update interval 0:00:15
2024-05-05 11:09:42.625 WARNING (MainThread) [homeassistant.components.camera] Updating amcrest camera took longer than the scheduled update interval 0:00:15
2024-05-05 11:09:57.625 WARNING (MainThread) [homeassistant.components.camera] Updating amcrest camera took longer than the scheduled update interval 0:00:15
2024-05-05 11:10:08.524 WARNING (MainThread) [homeassistant.components.mqtt.client] Disconnected from MQTT server 192.168.xxx.xxx:1883 (7)
2024-05-05 11:10:12.625 WARNING (MainThread) [homeassistant.components.camera] Updating amcrest camera took longer than the scheduled update interval 0:00:15
2024-05-05 11:10:27.625 WARNING (MainThread) [homeassistant.components.camera] Updating amcrest camera took longer than the scheduled update interval 0:00:15
2024-05-05 11:10:42.626 WARNING (MainThread) [homeassistant.components.camera] Updating amcrest camera took longer than the scheduled update interval 0:00:15
2024-05-05 11:10:57.626 WARNING (MainThread) [homeassistant.components.camera] Updating amcrest camera took longer than the scheduled update interval 0:00:15
2024-05-05 11:11:12.626 WARNING (MainThread) [homeassistant.components.camera] Updating amcrest camera took longer than the scheduled update interval 0:00:15
2024-05-05 11:11:27.626 WARNING (MainThread) [homeassistant.components.camera] Updating amcrest camera took longer than the scheduled update interval 0:00:15
2024-05-05 11:11:42.626 WARNING (MainThread) [homeassistant.components.camera] Updating amcrest camera took longer than the scheduled update interval 0:00:15
2024-05-05 11:11:50.548 WARNING (MainThread) [homeassistant.components.mqtt.client] Disconnected from MQTT server 192.168.xxx.xxx:1883 (7)
2024-05-05 11:11:57.626 WARNING (MainThread) [homeassistant.components.camera] Updating amcrest camera took longer than the scheduled update interval 0:00:15
2024-05-05 11:12:12.627 WARNING (MainThread) [homeassistant.components.camera] Updating amcrest camera took longer than the scheduled update interval 0:00:15
2024-05-05 11:12:27.627 WARNING (MainThread) [homeassistant.components.camera] Updating amcrest camera took longer than the scheduled update interval 0:00:15
2024-05-05 11:12:42.627 WARNING (MainThread) [homeassistant.components.camera] Updating amcrest camera took longer than the scheduled update interval 0:00:15

Additional information

• The camera was offline when starting HA, therefore the camera was not reachable during startup
• when setting "debug: true" in the homeassistant configuration, HA starts normally. "debug: false" (or no setting) reproduces the issue

[home-assistant]

Hey there @flacjacket, mind taking a look at this issue as it has been labeled with an integration (amcrest) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of amcrest can trigger bot actions by commenting:

• @home-assistant close Closes the issue.
• @home-assistant rename Awesome new title Renames the issue.
• @home-assistant reopen Reopen the issue.
• @home-assistant unassign amcrest Removes the current integration label and assignees on the issue, add the integration domain after the command.
• @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue.
• @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

---

amcrest documentation
amcrest source
_{^{(message by IssueLinks)}}