Cameras Stop Working

[bwthor]

I've had the BI/HA Integration running for quite some time, and have had zero issues. This past week I've started losing all my cameras after a number of hours. BI is working with no issues. In order to get them working again, I need to Reload the Integration, and then it works just fine again. Logs below show some issues.

I'd be more than happy to look at other areas for logs and such if someone can point me in the right direction.
HA is updated to the latest and BI is as well (5.8.1.1).

2023-11-26 17:10:54.949 ERROR (MainThread) [custom_components.blueiris.camera] Error getting new camera image from ThorNVR Front Door: [Errno 104] Connection reset by peer
2023-11-26 17:10:54.955 ERROR (MainThread) [custom_components.blueiris.camera] Error getting new camera image from ThorNVR Backyard: [Errno 104] Connection reset by peer
2023-11-26 17:11:05.046 ERROR (MainThread) [custom_components.blueiris.camera] Timeout getting camera image from ThorNVR Front Door
2023-11-26 17:11:14.477 ERROR (MainThread) [custom_components.blueiris.camera] Timeout getting camera image from ThorNVR Garage
2023-11-26 17:11:14.481 ERROR (MainThread) [custom_components.blueiris.camera] Timeout getting camera image from ThorNVR Backyard
2023-11-26 17:11:15.143 ERROR (MainThread) [custom_components.blueiris.camera] Timeout getting camera image from ThorNVR Front Door
2023-11-26 17:11:21.337 WARNING (MainThread) [custom_components.blueiris.api.blue_iris_api] Request #0 to BlueIris (http://192.168.0.12:85) failed, Data: {'cmd': 'camlist', 'session': '6ae0456931407055795c6ca534033ec5'}, Response: {'result': 'fail', 'session': '6ae0456931407055795c6ca534033ec5', 'data': {'reason': 'Invalid session'}}
2023-11-26 17:11:21.450 WARNING (MainThread) [custom_components.blueiris.api.blue_iris_api] Request #1 to BlueIris (http://192.168.0.12:85) failed, Data: {'cmd': 'camlist', 'session': '6ae0456931407055795c6ca534033ec5'}, Response: {'result': 'fail', 'session': '6ae0456931407055795c6ca534033ec5', 'data': {'reason': 'Invalid session'}}
2023-11-26 17:11:21.527 ERROR (MainThread) [custom_components.blueiris.managers.home_assistant] Failed to async_update, Error: 'str' object has no attribute 'get', Line: 196

[kramttocs]

Did you take a BI update or HA update around the time it started? Let's focus on trying to find what changed. The integration hasn't changed for quite some time so it's unlikely to be the root cause though possible it needs changed to meet the change of something else.

[bwthor]

I have updated both within the last week as I typically do when updates are released but I can't pinpoint when it started for sure. I know I should leave well enough alone, but I never learn my lesson.

But like anything else, as soon as I posted this, I haven't had any issues. I'll post back if I do, but it did fail me 4-5 times over the past few days.

[kramttocs]

No worries - I do the same. Trial by fire :)
Yeah, let me know and we can see what we can find.

[bwthor]

They stopped working about an hour after my post last night. A few hours ago I deleted the integration, and re-added it using the local BlueIris IP address and port. I may have used the URL before, but not sure. Anyway, got it up and running again, and I see it is down again. Any ideas what logs I should be looking at?

Editing to add, they never just start working again. Only way to make them work is to reload the integration. Restarting BI does nothing.

[kramttocs]

Same sequence of errors as before?

[kramttocs]

And I know you said it was working fine in the past so a BI config isn't likely but just to check as a comparison- you have the authentication settings configured as specified here? https://github.com/elad-bar/ha-blueiris/blob/master/docs/blueiris-server.md

[bwthor]

Yes, same sequence as before. No issues with BI, but on my panel, the four cameras I have just go blank. Connecting from another computer, there is no camera feed either, so not isolated to the computer running the touch panel.

I'm pretty sure I had things set up like in the link you shared. Definitely was using a separate user on LAN only. Not sure about the Require login from Non-Lan Only setting, because I think my proxy was messing with that setting. This time around I was using my regular admin login, but will switch that back and see.

Hard thing is that I can't replicate this, I just have to wait until it fails. I'd assume something should be in the logs, but I'm not seeing anything that makes sense.

[bwthor]

So, again after my last post I have had zero issues. Seems completely random.

[kazzaw]

Also having this issue - Cameras seem to go offline through the integration. Reloading the integration appears to bring them back up. I notice cameras fo offline more frequently when changing server/camera settings. Usually it would recover, however it would appear from a recent blueiris update this may have changed.

[kramttocs]

I am seeing this also - I don't use the video streams in HA so I added a couple when @bwthor reported it. What's interesting is that the mqtt sensors still work correctly - I still get motion detected in HA even though the camera entity isn't showing video.

[bwthor]

I am really hesitant to post this, but I think my cameras have been up for four days. I have changed nothing since the last time they stopped. Now they will quit in a few hours...

[kazzaw]

Yeah I have also seen the same… if it stops working I’ll blame you

[stevemcan]

I have the same although one camera continues to work, all the other just go blank. reloading the intergration brings them all back.

[wmichael3]

Hey, I've been running this integration for months without a burp. I know, not helpful, but in the early days I recall a few burps and what I ended up doing was to exempt the HA IP address from authentication to BI. You do that with the carat (^) before the IP under the Web Server advanced settings | Limit access by IP address. Just a thought. I do understand why some would not like that approach, but I thought it safe enough for my world and then I have the secure session keys and login page enabled for all other IPs.

[DaveStLou]

I'm seeing the same issue. Similar to others, mine has been running fine since originally installed but now all cameras go blank. I have found that disabling and re-enabling the integration gets it working but that's not a good workaround.

[dkirby-ms]

This started happening to me yesterday, right after an updated to HAOS 12.2. BI integration shows most of the entities of the cameras are "no longer being provided by the Blue Iris integration" but the profiles and schedule entities still work.

Edit: This was my own issue. When I upgraded the BI integration had to be reconfigured and each of my cameras and its entities re-checked in the BI config page.

[kramttocs]

Is everyone getting an invalid session error when this happens? Best guess is something on BIs end changed since the integration hasn't and I wouldn't expect an HA change to impact authentication with BI. Opening a request to BI to see if anything has changed recently around that would be the best initial option.

[permster]

You can actually reproduce this issue fairly easily. Stop the Blue Iris Service, wait until you get a "failed to connect" message in the HA system logs (may take a few minutes), then start the Blue Iris Service. Wait a few more minutes and you should get the "invalid session" system log.

I actually built an automation to reload the configuration of a single camera which seems to fix all cameras as a result. This requires the fire_event: true in your configuration.yaml.

alias: General - Blue Iris Reload
description: ""
trigger:
  - platform: event
    event_type: system_log_event
    event_data:
      name: custom_components.blueiris.api.blue_iris_api
condition:
  - condition: template
    value_template: "{{ '\\'reason\\': \\'Invalid session\\'' in trigger.event.data.message[0] }}"
action:
  - service: homeassistant.reload_config_entry
    data: {}
    target:
      entity_id: camera.blue_iris_courtyard
  - service: notify.mobile_app_android_phone
    data:
      message: Blue Iris integration restoration has been attempted.  Please verify!
      title: Reload Blue Iris integration
mode: single

[DaveStLou]

As a stopgap I set up the automation from @permster but for some reason it didn't trigger even though I had two events (below). Any idea why?

Request #0 to BlueIris (http://10.42.8.50:81) failed, Data: {'cmd': 'camlist', 'session': '37d8168405e63dcc1886083a466f7039'}, Response: {'result': 'fail', 'session': '37d8168405e63dcc1886083a466f7039', 'data': {'reason': 'Invalid session'}}
Request #1 to BlueIris (http://10.42.8.50:81) failed, Data: {'cmd': 'camlist', 'session': '37d8168405e63dcc1886083a466f7039'}, Response: {'result': 'fail', 'session': '37d8168405e63dcc1886083a466f7039', 'data': {'reason': 'Invalid session'}} 

I ran it manually and the action reset all the cams but I'd like to get it to trigger without intervention until this issue is resolved.

[permster]

Did you enable the system log fire_event: true in your configuration.yaml as documented here. Make sure your restart HA after the config update.

[DaveStLou]

That was it. Thank you!

[Kedryn]

I have same problem (way longer than 3 weeks ago). cams stop working and come back absolutely random.
What i've noticed is that cams disappears from the integration, when i get a timeout on the Ha lovelace page that shows them. So it's something between that and HA. Also because the the Tapo app shows them online.

Is it normal for cams to disappear and come back again from the BI integration?

i saw "devices" go down as low as 2 for few seconds.

[tcouey]

Any updates on this? I've been having the same issue for a couple of months now. I was hoping the latest update to the integration and HA 2024.1.0 would fix it, but it has not. They still all go offline after some amount of time (a few hours). No issue that I can find in Blue Iris.

[kramttocs]

No real updates. BI support told me there was a change:

Yes if you send an invalid session to the /video? endpoint, it will [now] return invalid session. Previously, it would redirect to the login page. That didn't make sense and our UI3 developer asked for this change.

But I haven't found a connection between it and this behavior.

[kozoke]

Same issue. Started when I updated blue iris to the most current release. I rolled back to the 12/5/23 version (very easy) and it appears to be working fine. Thanks for the ack above. Look forward to an update. :-)

[SpudsterZ]

Ditto here, HOWEVER everything has been stable for the past couple days after the last restart of the integration. I do see BI upgraded to 5.8.4.4 on the 16th, so here's to hoping the that resolved this issue.

[PrestonMcAfee]

Had this issue today, HA 2024.1.6, BI 5.8.5.7. Reloading the BI integration fixed for me. Updating BI to 5.8.6 to see if it helps. The update, which briefly shuts off BI, did not affect camera access.

I haven't yet gotten to using the BI integration to do anything other than a quick way of looking at cameras, as I find the BI web interface works well, but I have plans to turn on lights when motion is detected in BI, and this loss of connection would put a crimp in that.

Thanks, permster, for a workaround!

[kramttocs]

@PrestonMcAfee The camera stream is separate from the mqtt traffic. You will use mqtt for the motion detection so you'll be good there.

If you have any questions on the mqtt part, check out the guide here in GitHub (on mobile so can't easily link it).

[PrestonMcAfee]

Thanks! I still have a lot to learn.

…

On Sat, Feb 3, 2024 at 12:07 PM kramttocs ***@***.***> wrote: @PrestonMcAfee <https://github.com/PrestonMcAfee> No, the camera stream is separate from the mqtt traffic. You will use mqtt for the motion detection so you'll be good there. — Reply to this email directly, view it on GitHub <#217 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKANTXLPN6UGWONKKTGHXE3YR2KIZAVCNFSM6AAAAAA74ALH2CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRVGQ2DGNZTHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[kramttocs]

You bet! Here is the link for that manual: https://github.com/elad-bar/ha-blueiris/blob/master/docs/blueiris-server.md

[SpudsterZ]

Ditto here, HOWEVER everything has been stable for the past couple days after the last restart of the integration. I do see BI upgraded to 5.8.4.4 on the 16th, so here's to hoping the that resolved this issue.

Well, it hasn't been 100% since I last posted the above, but it's only been about 3 times where the cameras went wonky. In one occasion, the integration reload didn't work, requiring a full HA restart.

So not perfect, but far and above better than the near daily issue that was once seen.

[trailiajohn]

I'm also having this issue. The automation above sometimes works, but even manually reloading the integration I couldn't get my feeds back without a full HA restart.

[kramttocs]

That seems like a different issue (or multiple) as restarting the integration generates a new bi auth session. Did you see anything in the bi logs? Like an invalid login or anything?

[trailiajohn]

BI logs do show the logins when rebooting the integration without any error on the BI side. MQ works great, but cameras take a full HA restart to resolve the problem in my Dashboard.

[SpudsterZ]

Just had a situation where the automation above restarted the integration, but it did not recover, requiring a full HA restart. Checking BI logs shows this happened when BI performed an automated update and restarted itself.

[jatgm1]

So I also have this issue. It's super annoying. I use them for motion sensors.. none of my lights are working now and it's super annoying, I smashed my toe on a door frame in the dark.. how can I make this work for now? Or like just in general.

[kramttocs]

The camera stream issue should be completely isolated from mqtt which is what is used for motion detection.

[jatgm1]

Oh crap it did a backup and some stuff stopped working i think a restart fixed it, correlation is not always causation, my apologies, disregard that part. The cameras do blank out though and won't come back without reloading the integration.

[kramttocs]

No worries - hope the toe turns out ok :)

[jatgm1]

Has anyone figured out what's wrong and how to make it work now?

[rlsgit]

Same here...appears to be a logout occurring...I'm going to dig deeper...Just moved from Shinobi to BlueIris, and this hurts...

[wmichael3]

If you exempt your HA IP address from requiring authentication in BI all of these problems will disappear.

[rlsgit]

If you exempt your HA IP address from requiring authentication in BI all of these problems will disappear.

Well, that was harder to find than I thought, but will keep an eye on it...no to the other erroneous log entries the integration sends...

[wmichael3]

Sorry, should have added detail: in Blue Iris main settings: (cog icon at top left). Web Server, Advanced, Limit access by IP address, in the box enter a carat "^" followed by the IP of your HA instance. For example ^192.168.1.10

Since doing this it has been perfect. Not a solution, but a workaround.

[bwthor]

If you exempt your HA IP address from requiring authentication in BI all of these problems will disappear.

I use NGINX Reverse Proxy, and I seem to recall when I set this, it allowed anyone to log on, even from outside. It was 6+ months ago and my memory is junk, but that seems to be the issue. Either way, double check the access from outside if you do set this.

[rlsgit]

Sorry, should have added detail: in Blue Iris main settings: (cog icon at top left). Web Server, Advanced, Limit access by IP address, in the box enter a carat "^" followed by the IP of your HA instance. For example ^192.168.1.10

Since doing this it has been perfect. Not a solution, but a workaround.

Thanks, I did +192.168... and will keep an eye on it.

[rlsgit]

If you exempt your HA IP address from requiring authentication in BI all of these problems will disappear.

I use NGINX Reverse Proxy, and I seem to recall when I set this, it allowed anyone to log on, even from outside. It was 6+ months ago and my memory is junk, but that seems to be the issue. Either way, double check the access from outside if you do set this.

So, I'm home only, not commercial, and do have access into my system by rule and secured for me. But I agree it's a bit scary. I'm a dev by nature and have written some HA stuff, so may poke around a but. I also hat log entries that indicate potential issues. I've done ALOT to clean my system up and hate seeing some integrations that just generate "noise"...

[trailiajohn]

BI documentation says you need an account named "admin", which I added, but it's still not working for me. I always end up on the login page.

[johnjoemorgan]

Sorry, should have added detail: in Blue Iris main settings: (cog icon at top left). Web Server, Advanced, Limit access by IP address, in the box enter a carat "^" followed by the IP of your HA instance. For example ^192.168.1.10

Since doing this it has been perfect. Not a solution, but a workaround.

Tryng this now

[johnjoemorgan]

Sorry, should have added detail: in Blue Iris main settings: (cog icon at top left). Web Server, Advanced, Limit access by IP address, in the box enter a carat "^" followed by the IP of your HA instance. For example ^192.168.1.10

Since doing this it has been perfect. Not a solution, but a workaround.

Been working flawlessly for last 3 hours. Both via direct local IP and via ZeroTier. That 2.5 hours longer than any other time in the last 6 months or whenever this problem started

Thanks @wmichael3

[wmichael3]

One other thing I've noticed that I didn't see mentioned in this thread is the hitting of the connection limit in BI, which is set to 50 by default. It seems that the HA integration creates a lot of connections - maybe 1 per camera? - and if you have a couple people using HA and surfing cameras you can all of a sudden get broken camera images or thumbnails without any obvious error message. I bumped my connection limit to 75 and that fixed it. (We have 7 cameras) Reloading the integration probably terminates all existing connections so there is a chance that this is another underlying issue along with authentication, which I think is the main one.

Exempt the IP and increase the connection limit and I bet it will solve just about every problem. I see you have it set to 99, which is good. I've never had to reload the BI integration after a BI update or restart.

[johnjoemorgan]

Yep 16 cameras in my set up but I didn't change that setting yesterday (maybe I had in the past). Everything is still running smoothly. No feed has become unavailable from BI

[rlsgit]

All seems fine now with those changes...still some log entries in HA i'd like to clean up though...

…

________________________________ From: johnjoemorgan ***@***.***> Sent: Wednesday, April 10, 2024 9:34 PM To: elad-bar/ha-blueiris ***@***.***> Cc: Robert Stephens ***@***.***>; Comment ***@***.***> Subject: Re: [elad-bar/ha-blueiris] Cameras Stop Working (Issue #217) Yep 16 cameras in my set up but I didn't change that setting yesterday (maybe I had in the past). Everything is still running smoothly. No feed has become unavailable from BI — Reply to this email directly, view it on GitHub<#217 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ADCD5HKSGTQ2F7RDE2AO5JTY4XSA3AVCNFSM6AAAAAA74ALH2CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBYG43TKMRRGY>. You are receiving this because you commented.Message ID: ***@***.***>

[andrea-mattioli]

Hello everyone, I also had this problem. I have a somewhat complex network at home with segregations for video surveillance, IoT, and other networks. By doing network dumps, I practically noticed that Home Assistant uses port 1900/UDP on the multicast address 239.255.255.250 to keep the media player of the cameras active. By testing and dropping this connection, after a while, it sets the camera entity to "inactive," and the streaming flow doesn't work. If I restart the integration, it starts again, but after a short while, it stops as it receives drops from the firewall and gets the error:
Error opening stream (HTTP_UNAUTHORIZED, Server returned 401 Unauthorized (authorization failed), http://192.168.1.3:81/h264/Esterno/temp.m3u8?session=517d367311ce3486391806c02f2c4de2)
I hope this can be helpful to someone.