Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Amcrest reports 1000s of errors in log after update to 2022.3.5 #68272

Closed
KennethLavrsen opened this issue Mar 16, 2022 · 22 comments · Fixed by #69640
Closed

Amcrest reports 1000s of errors in log after update to 2022.3.5 #68272

KennethLavrsen opened this issue Mar 16, 2022 · 22 comments · Fixed by #69640

Comments

@KennethLavrsen
Copy link

KennethLavrsen commented Mar 16, 2022

The problem

After upgrade to 2022.3.5 which contained several updates to the Amcrest code my error log is flooded with errors related to Amcrest

Logger: homeassistant.components.amcrest
Source: components/amcrest/__init__.py:261
Integration: Amcrest ([documentation](https://www.home-assistant.io/integrations/amcrest), [issues](https://github.com/home-assistant/home-assistant/issues?q=is%3Aissue+is%3Aopen+label%3A%22integration%3A+amcrest%22))
First occurred: 09:21:56 (4079 occurrences)
Last logged: 22:00:04

Error while processing events from Living room camera camera: CommError(HTTPStatusError("Client error '400 Bad Request' for url 'http://192.168.1.61/cgi-bin/eventManager.cgi?action=attach&codes=[All]'\nFor more information check: https://httpstatuses.com/400"))
Error while processing events from Living room camera camera: CommError(ConnectTimeout(''))
Error while processing events from Living room camera camera: CommError(ConnectError('All connection attempts failed'))
Error while processing events from Living room camera camera: CommError(RemoteProtocolError('Server disconnected without sending a response.'))
Error while processing events from Living room camera camera: CommError(HTTPStatusError("Server error '500 Internal Server Error' for url 'http://192.168.1.61/cgi-bin/eventManager.cgi?action=attach&codes=[All]'\nFor more information check: https://httpstatuses.com/500"))

The cameras are not very new and there has always been an occational glitch that fixed itself with 10-15 lines of warnings.
The CommError(HTTPStatusError("Client error '400 Bad Request' for url 'http://192.168.1.61/cgi-bin/eventManager.cgi?action=attach&codes=[All]'\nFor more information check: https://httpstatuses.com/400")) is new.

What version of Home Assistant Core has the issue?

core-2022.3.5

What was the last working version of Home Assistant Core?

core-2022.3.4

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Amcrest

Link to integration documentation on our website

https://www.home-assistant.io/integrations/amcrest/

Diagnostics information

A sniplet from the log

2022-03-16 09:21:56 WARNING (MainThread) [homeassistant.components.amcrest] Error while processing events from Living room camera camera: CommError(HTTPStatusError("Client error '400 Bad Request' for url 'http://192.168.1.61/cgi-bin/eventManager.cgi?action=attach&codes=[All]'\nFor more information check: https://httpstatuses.com/400"))
2022-03-16 09:21:57 WARNING (MainThread) [homeassistant.components.amcrest] Error while processing events from Living room camera camera: CommError(HTTPStatusError("Client error '400 Bad Request' for url 'http://192.168.1.61/cgi-bin/eventManager.cgi?action=attach&codes=[All]'\nFor more information check: https://httpstatuses.com/400"))
2022-03-16 09:21:57 WARNING (MainThread) [homeassistant.components.amcrest] Error while processing events from Living room camera camera: CommError(HTTPStatusError("Client error '400 Bad Request' for url 'http://192.168.1.61/cgi-bin/eventManager.cgi?action=attach&codes=[All]'\nFor more information check: https://httpstatuses.com/400"))
2022-03-16 09:21:57 WARNING (MainThread) [homeassistant.components.amcrest] Error while processing events from Living room camera camera: CommError(HTTPStatusError("Client error '400 Bad Request' for url 'http://192.168.1.61/cgi-bin/eventManager.cgi?action=attach&codes=[All]'\nFor more information check: https://httpstatuses.com/400"))

It just continues like that with 1000s of lines - just for half a day

Occationally this variation is seen

2022-03-16 11:36:57 WARNING (MainThread) [homeassistant.components.amcrest] Error while processing events from Living room camera camera: CommError(HTTPStatusError("Client error '400 Bad Request' for url 'http://192.168.1.61/cgi-bin/eventManager.cgi?action=attach&codes=[All]'\nFor more information check: https://httpstatuses.com/400"))
2022-03-16 11:36:57 ERROR (MainThread) [homeassistant.components.amcrest] Living room camera camera offline: Too many errors
2022-03-16 11:36:57 WARNING (MainThread) [homeassistant.components.amcrest] Error while processing events from Living room camera camera: CommError(HTTPStatusError("Client error '400 Bad Request' for url 'http://192.168.1.61/cgi-bin/eventManager.cgi?action=attach&codes=[All]'\nFor more information check: https://httpstatuses.com/400"))
2022-03-16 11:37:57 ERROR (MainThread) [homeassistant.components.amcrest] Living room camera camera back online
2022-03-16 11:37:57 WARNING (MainThread) [homeassistant.components.amcrest] Error while processing events from Living room camera camera: CommError(HTTPStatusError("Client error '400 Bad Request' for url 'http://192.168.1.61/cgi-bin/eventManager.cgi?action=attach&codes=[All]'\nFor more information check: https://httpstatuses.com/400"))

Example YAML snippet

I have two cameras

amcrest:
  - host: !secret amcrest_living
    username: admin
    password: !secret amcrest_password
    stream_source: snapshot
    binary_sensors:
      - motion_detected
      - online
    sensors:
      - sdcard
      - ptz_preset
    name: Living room camera
    resolution: high
  - host: !secret amcrest_kitchen
    username: admin
    password: !secret amcrest_password
    stream_source: snapshot
    binary_sensors:
      - motion_detected
      - online
    sensors:
      - ptz_preset
    name: Kitchen camera
    resolution: high

Anything in the logs that might be useful for us?

No response

Additional information

UPDATE: Please read my comments below. The errors changed after a firmware upgrade of the camera but things are still not really good

@probot-home-assistant
Copy link

amcrest documentation
amcrest source
(message by IssueLinks)

@probot-home-assistant
Copy link

Hey there @flacjacket, mind taking a look at this issue as it has been labeled with an integration (amcrest) you are listed as a code owner for? Thanks!
(message by CodeOwnersMention)

@KennethLavrsen
Copy link
Author

I started digging into this. I found that my old Amcrest had firmware upgrade since I checked two years ago. Also also removed the - sdcard from the config for the Living Room camera. I still have problems but it is new errors. I am now trying the different ways to stream to see what works best. But the error has changed. The camera works but not stable.

@KennethLavrsen
Copy link
Author

I have tried snapshot, mjpeg, and rtsp. All the same result.

I also lowered the frame rate from 25 to 5 and changed the bit stream to 2 Mbit.

I have a software on my iPad streaming the rtsp stream from same camera without a problem.

I get the log flooded with

2022-03-16 23:45:24 WARNING (MainThread) [amcrest.http] <AMC000EW_KT531F:AMC000EWK7SWKT531F> Trying again due to error: RemoteProtocolError('Server disconnected without sending a response.')
2022-03-16 23:45:24 ERROR (MainThread) [homeassistant.components.amcrest.camera] Could not get image from Living room camera camera due to error: CommError
2022-03-16 23:45:24 WARNING (MainThread) [amcrest.http] <AMC000EW_KT531F:AMC000EWK7SWKT531F> Trying again due to error: RemoteProtocolError('Server disconnected without sending a response.')
2022-03-16 23:45:24 ERROR (MainThread) [homeassistant.components.amcrest.camera] Could not get image from Living room camera camera due to error: CommError
2022-03-16 23:45:47 WARNING (MainThread) [amcrest.http] <AMC000EW_KT531F:AMC000EWK7SWKT531F> Trying again due to error: RemoteProtocolError('Server disconnected without sending a response.')
2022-03-16 23:45:47 ERROR (MainThread) [homeassistant.components.amcrest.camera] Could not get image from Living room camera camera due to error: CommError
2022-03-16 23:46:07 WARNING (MainThread) [amcrest.http] <AMC000EW_KT531F:AMC000EWK7SWKT531F> Trying again due to error: RemoteProtocolError('Server disconnected without sending a response.')
2022-03-16 23:46:07 ERROR (MainThread) [homeassistant.components.amcrest.camera] Could not get image from Living room camera camera due to error: CommError
2022-03-16 23:46:57 WARNING (MainThread) [amcrest.http] <AMC000EW_KT531F:AMC000EWK7SWKT531F> Trying again due to error: RemoteProtocolError('Server disconnected without sending a response.')
2022-03-16 23:46:57 ERROR (MainThread) [homeassistant.components.amcrest.camera] Could not get image from Living room camera camera due to error: CommError
2022-03-16 23:47:13 WARNING (MainThread) [amcrest.http] <AMC000EW_KT531F:AMC000EWK7SWKT531F> Trying again due to error: RemoteProtocolError('Server disconnected without sending a response.')
2022-03-16 23:47:13 ERROR (MainThread) [homeassistant.components.amcrest.camera] Could not get image from Living room camera camera due to error: CommError
2022-03-16 23:47:36 WARNING (MainThread) [amcrest.http] <AMC000EW_KT531F:AMC000EWK7SWKT531F> Trying again due to error: RemoteProtocolError('Server disconnected without sending a response.')
2022-03-16 23:47:36 ERROR (MainThread) [homeassistant.components.amcrest.camera] Could not get image from Living room camera camera due to error: CommError
2022-03-16 23:47:36 WARNING (MainThread) [amcrest.http] <AMC000EW_KT531F:AMC000EWK7SWKT531F> Trying again due to error: RemoteProtocolError('Server disconnected without sending a response.')
2022-03-16 23:47:36 ERROR (MainThread) [homeassistant.components.amcrest.camera] Could not get image from Living room camera camera due to error: CommError
2022-03-16 23:47:36 WARNING (MainThread) [amcrest.http] <AMC000EW_KT531F:AMC000EWK7SWKT531F> Trying again due to error: RemoteProtocolError('Server disconnected without sending a response.')
2022-03-16 23:47:36 ERROR (MainThread) [homeassistant.components.amcrest.camera] Could not get image from Living room camera camera due to error: CommError
2022-03-16 23:47:37 WARNING (MainThread) [amcrest.http] <AMC000EW_KT531F:AMC000EWK7SWKT531F> Trying again due to error: RemoteProtocolError('Server disconnected without sending a response.')
2022-03-16 23:47:37 ERROR (MainThread) [homeassistant.components.amcrest.camera] Could not get image from Living room camera camera due to error: CommError
2022-03-16 23:47:53 WARNING (MainThread) [amcrest.http] <AMC000EW_KT531F:AMC000EWK7SWKT531F> Trying again due to error: HTTPStatusError("Server error '500 Internal Server Error' for url 'http://192.168.1.61/cgi-bin/snapshot.cgi'\nFor more information check: https://httpstatuses.com/500")
2022-03-16 23:48:08 ERROR (MainThread) [homeassistant.components.amcrest.camera] Could not get image from Living room camera camera due to error: CommError

@KennethLavrsen
Copy link
Author

KennethLavrsen commented Mar 16, 2022

Last update (for today). The low frame rate made it worse. I put it back to 25 fps but with 2Mbit frame rate and I changed to VBR. I now get the error in the log every minute when I look at the stream. PTZ works which is the most important for me.

I also change the picture entity card that shows the camera from camera_view auto to live. That seems to also lower the number of errors but rise the load on the link when viewing that tab of Lovelace.

I forgot to say that the camera is IP2M-841W and it is wired, not wifi.

@dshokouhi
Copy link
Member

dshokouhi commented Mar 17, 2022

Well the fixes in 2022.3.5 fixed the amcrest issues I had where the binary sensors would get stuck in a state and never update until a HA restart. I do see the errors here but nowhere near the same amount of errors. Maybe 20-30 per day per camera. I have several IP2M-841W and a couple IP4M-1051 all on wifi.

@KennethLavrsen
Copy link
Author

What exact firmware version? And can you share your yaml config?

@dshokouhi
Copy link
Member

IP4M-1051 - V2.620.00AC000.3.R, Build Date: 2019-08-03
IP2M-841 - V2.420.AC00.18.R, Build Date: 2019-07-29

example of one camera:

amcrest:
 - host: 192.168.1.162
   port: 1029
   username: !secret amcrest_user
   password: !secret amcrest_pass
   name: "Kitchen"
   resolution: high
   stream_source: rtsp
   binary_sensors:
   - motion_detected
   - audio_detected

@Madseason41
Copy link

Madseason41 commented Mar 17, 2022

Same here, I think my issues started on the 2022.3.1 release. I haven't changed anything on my camera and the firmware has been at V2.420.AC01.3.R, 6/2/18, for quite some time.

The camera initially works on home assistant start, then the stream goes to unavailable and the camera is unusable (in home assistant). Works fine thru motion eye.

Camera model number: ip2m-853ew

yaml:

amcrest:
  - host: 192.168.1.108
    username: !secret std_uname
    password: !secret std_pwd
    name: Carport Amcrest
    stream_source: rtsp

I have tried swapping to snapshot as my stream source as well. No luck.

@jonostanck
Copy link

The same for me. Before 2022.3.1 amcrest integration was working fine. Occasional connection errors were, well, occasional and did not result in dropped streams or other camera functionalities not being available through HA.

I have three IP3M-941B cameras at 2.520.AC00.18.R firmware.

amcrest:
  - host: 192.168.15.20
    name: 'CAM1'
    username: !secret amcrest_user
    password: !secret amcrest_password
    scan_interval: 5
    stream_source: rtsp
    resolution: low
    binary_sensors:
      - motion_detected
  - host: 192.168.15.21
    name: 'CAM2'
    username: !secret amcrest_user
    password: !secret amcrest_password
    scan_interval: 5
    stream_source: rtsp
    resolution: low
    binary_sensors:
      - motion_detected
  - host: 192.168.15.22
    name: 'CAM3'
    username: !secret amcrest_user
    password: !secret amcrest_password
    scan_interval: 5
    stream_source: rtsp
    resolution: low
    binary_sensors:
      - motion_detected

My error log is showing:

Logger: homeassistant.components.amcrest
Source: components/amcrest/__init__.py:216
Integration: Amcrest (documentation, issues)
First occurred: 04:10:28 (869 occurrences)
Last logged: 13:07:02

CAM1 camera offline: Too many errors
CAM2 camera offline: Too many errors
CAM3 camera offline: Too many errors

Hope this will get resolved soon. Thanks!

@drewancil
Copy link

I have 4 Amcrest Cameras, 3 of which I am running through the HA Integration. They are:

Amcrest IP8M-2493EW-V2
Amcrest IP8M-T2599EW
Amcrest IP4M-1041B

The 4th camera is an AD410 doorbell which I'm running the Dahua integration as it has support for the doorbell functions.

I am not experiencing the issues you outlined with the Amcrest integration - the logs are clear. I'm not sure how helpful this comment is other then to provide a configuration that is working. My config is:

amcrest:

  • host: 192.168.4.190
    username: !secret amcrest_username
    password: !secret amcrest_password
    name: Exterior Camera HA

  • host: 192.168.4.139
    username: !secret amcrest_username
    password: !secret amcrest_password
    name: Utility Room Camera HA

  • host: 192.168.4.248
    username: !secret amcrest_username
    password: !secret amcrest_password
    name: Living Room Camera HA
    switches:

    • privacy_mode
      binary_sensors:
    • online

@Madseason41
Copy link

@drewancil Just out of curiosity what firmware versions are you running on your cameras?

@drewancil
Copy link

2493 - V2.800.00AC001.0.R, Build Date: 2021-07-12
2599 - V2.800.00AC001.0.R, Build Date: 2021-07-12
1041 - V2.800.0000000.15.R, Build Date: 2021-07-16

@KennethLavrsen
Copy link
Author

KennethLavrsen commented Mar 18, 2022

My problem camera has firmware V2.420.AC00.18.R, Build Date: 2020-03-30
And I have also tried the previous 2019 version.
Both are buggy as hell.
The Rest API is supposed to accept credentials in URL but it demands Digest type credentials.
The snapshot.cgi is broken. It hangs for many seconds and times out. That is what creates the errors
Piece of junk that Amcrest never finished the firmware for´and clearly does not support any longer in any way. So I am never buying Amcrest again. Right now my best combo is to load the same camera as both ONVIF and Amcrest. ONVIF for video and Amcrest to control PTZ. And that will be short lived because I will find a better camera and throw away this junk. Life is too short. It seems the previous versions of Python libs for Amcrest worked around the issues but newer is probably based on more recent cameras and now fails with the 2019/2020 firmwares

@Madelinot
Copy link

Similar issue here. Two out of my five Amcrest cameras are reporting thousands or errors, and are offline now (in HA).

@Zoriontsu
Copy link

Experiencing same problem since March22 update. Cameras have been performing great for several years:
image
The cameras model IP5M-T1179E seem to work fine. The other two fail with "two many errors" in log.

@hawkeye217
Copy link

Just updated my HA to 2022.3.5 (from 2021.12.x) and I too am seeing the same errors as everyone else for my 2 Amcrest cameras. Everything was great before the update.

@flacjacket
Copy link
Contributor

So when we moved the integration to use async methods, we needed to change the underlying library from requests to httpx. This change affected streaming from the cameras (which is how the non-polling sensors listen for events) as when the camera would disconnect and reconnect, there would not be an error that would force us to reconnect (see #66661 for example). We added a read timeout which would be handled and cause a re-connection, but it seems like that isn't a robust solution for all amcrest cameras. It may be best to change back the synchronous requests at least for waiting for events until we can figure out a better streaming solution. I'm not sure how aiohttp handles this, but we need them to support digest authentication, which has an open issue there. We may also want to reach out to httpx for a fix on the library side to let us raise an error that we can handle when the camera is disconnected.

@mockdot
Copy link

mockdot commented Mar 24, 2022

Having same issue with Amcrest camera IP4M-1026B, downgraded to 2022.3.4 for the time being until a new patch is issued.

@codepoet4
Copy link

Same issue here with multiple IP2M-841B cameras.

@Zoriontsu
Copy link

Zoriontsu commented Apr 4, 2022

Other than staying on 2022.3.4, are there any plans to address this? I haven't noticed any related entries in the [2022.4.] pre-releases.
I have some pretty cool object recognition automations setup with deepstack and my Amcrest cameras, and the whole setup is no longer functional. It took me several brands of IP Cams to finally settle on Amcrest because I found the integration robust.

@flacjacket
Copy link
Contributor

I have a draft fix up at #69640, but I haven't had a chance to actually test anything out with that, I should be able to get to that this weekend.

@github-actions github-actions bot locked and limited conversation to collaborators Jun 3, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.