Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Option to restart plugin automatically when cameras go offline #391

Closed
dnestico opened this issue May 25, 2022 · 25 comments
Closed

Option to restart plugin automatically when cameras go offline #391

dnestico opened this issue May 25, 2022 · 25 comments
Labels
bug Something isn't working enhancement New feature or request

Comments

@dnestico
Copy link

dnestico commented May 25, 2022

For whatever my cameras seem to go unavailable on the bridge/addon on a daily basis when they are not even offline in the wyze app, restarting the plugin sometimes fixes this.

So an option to automatically restart the addon when certain or all cameras go offline would help because this really is annoying and I'm considering using the RTSP firmware instead on these specific cameras that go unavailable for no reason on the bridge.

Screen Shot 2022-05-24 at 10 37 36 PM

@mjb83
Copy link

mjb83 commented May 29, 2022

I will second this. I have a front door camera that seems to crash all the time and usually restarting the plugin fixes it.

@r14n
Copy link

r14n commented Jun 15, 2022

All my cameras seem to crash (in WyzeBridge - not in Wyze app). I'm not sure why exactly.

@mrlt8
Copy link
Owner

mrlt8 commented Jun 15, 2022

Can you post any info from the logs so we can try to see if we can restart them automatically?

@r14n
Copy link

r14n commented Jun 16, 2022

Sure. In HomeAssistant is there a way to view the entire log? I think it's a limited section when I view the Log from the addon page.

@mrlt8
Copy link
Owner

mrlt8 commented Jun 16, 2022

You should be able to see the full logs on http://homeassistant.local:8123/config/logs and select the bridge from the drop down in the top right.

@r14n
Copy link

r14n commented Jun 16, 2022

You should be able to see the full logs on http://homeassistant.local:8123/config/logs and select the bridge from the drop down in the top right.

Thanks! I didn't know about that area of HA. However it still doesn't display the entire log, when I refresh I can see that earlier lines are gone. Anyway here's the logs I have manually concatenated by refreshing that page.

I see it's just always timing out. Also having it running is absolutely hammering my network, Zoom calls dropping etc, maybe related issue?

  • Serving Flask app 'frontend' (lazy loading)
  • Environment: production
    WARNING: This is a development server. Do not use it in a production deployment.
    Use a production WSGI server instead.
  • Debug mode: off
    2022/06/16 08:49:51 [MainProcess] 🚀 STARTING DOCKER-WYZE-BRIDGE v1.6.2
    2022/06/16 08:49:51 [MainProcess] 🏠 Home Assistant Mode
    2022/06/16 08:49:51 [MainProcess] 📚 Using 'user' from local cache...
    2022/06/16 08:49:51 [MainProcess] * Running on all addresses (0.0.0.0)
    WARNING: This is a development server. Do not use it in a production deployment.
  • Running on http://127.0.0.1:5000
  • Running on http://172.30.33.2:5000 (Press CTRL+C to quit)
    2022/06/16 08:49:51 [MainProcess] 📚 Using 'cameras' from local cache...
    2022/06/16 08:49:51 [MainProcess]
    🎬 STARTING ALL 6 CAMERAS
    2022/06/16 08:49:51 [MainProcess] Starting rtsp-simple-server v0.19.1
    2022/06/16 08:49:51 [MainProcess] 🎉 Connecting to WyzeCam V3 - Basement on 192.168.100.43 (1/3)
    2022/06/16 08:49:51 [MainProcess] 🎉 Connecting to WyzeCam V3 - Front window cam on 192.168.100.23 (1/3)
    2022/06/16 08:49:51 [MainProcess] 🎉 Connecting to WyzeCam V3 - Back Yard Cam on 192.168.100.17 (1/3)
    2022/06/16 08:49:51 [MainProcess] 🎉 Connecting to WyzeCam Outdoor - Outdoor 3 on 192.168.200.104 (1/3)
    2022/06/16 08:49:51 [MainProcess] 🎉 Connecting to WyzeCam Outdoor - Back Door on 192.168.200.103 (1/3)
    2022/06/16 08:49:51 [MainProcess] 🎉 Connecting to WyzeCam Outdoor - Front Porch on 192.168.200.102 (1/3)
    2022/06/16 08:49:53 [Back Yard Cam] 📡 Getting 120kb/s HD stream (20fps) via LAN mode (WiFi: 62%) FW: 4.36.9.139 🔒 (DTLS) (2/3)
    2022/06/16 08:49:53 [Back Yard Cam] WARNING: Skipping smaller frame at start of stream (frame_size=1)
    2022/06/16 08:49:53 [Front window cam] 📡 Getting 120kb/s HD stream (20fps) via LAN mode (WiFi: 55%) FW: 4.36.9.139 🔒 (DTLS) (2/3)
    2022/06/16 08:49:54 [Basement] ☁️ Connected via NON-LAN MODE! Reconnecting
    2022/06/16 08:49:54 [Front window cam] WARNING: Skipping smaller frame at start of stream (frame_size=1)
    2022/06/16 08:49:54 [MainProcess] 🎉 Connecting to WyzeCam V3 - Basement on 192.168.100.43 (1/3)
    2022/06/16 08:49:56 [RTSP][BACK-YARD-CAM] ✅ '/back-yard-cam' stream is UP! (3/3)
    2022/06/16 08:49:57 [Back Yard Cam] WARNING: Frame not available yet
    2022/06/16 08:49:57 [Front window cam] WARNING: Frame not available yet
    2022/06/16 08:49:57 [RTSP][FRONT-WINDOW-CAM] ✅ '/front-window-cam' stream is UP! (3/3)
    2022/06/16 08:49:57 [Basement] 📡 Getting 120kb/s HD stream (20fps) via LAN mode (WiFi: 51%) FW: 4.36.9.139 🔒 (DTLS) (2/3)
    2022/06/16 08:49:57 [Basement] WARNING: Skipping smaller frame at start of stream (frame_size=1)
    2022/06/16 08:50:11 [Back Door] IOTC_ER_TIMEOUT
    2022/06/16 08:50:11 [Front Porch] IOTC_ER_TIMEOUT
    2022/06/16 08:50:11 [Outdoor 3] IOTC_ER_TIMEOUT
    2022/06/16 08:50:13 [MainProcess] ⏰ Timed out connecting to Outdoor 3 (20s).
    2022/06/16 08:50:13 [MainProcess] ⏰ Timed out connecting to Back Door (20s).
    2022/06/16 08:50:13 [MainProcess] ⏰ Timed out connecting to Front Porch (20s).
    2022/06/16 08:50:17 [Basement] WARNING: Still waiting for first frame. Updating frame size.
    2022/06/16 08:50:17 [Basement] Requesting frame_size=0 and bitrate=120
    2022/06/16 08:50:18 [Basement] WARNING: Frame not available yet
    2022/06/16 08:50:23 [MainProcess] 🎉 Connecting to WyzeCam Outdoor - Outdoor 3 on 192.168.200.104 (1/3)
    2022/06/16 08:50:23 [MainProcess] 🎉 Connecting to WyzeCam Outdoor - Back Door on 192.168.200.103 (1/3)
    2022/06/16 08:50:23 [MainProcess] 🎉 Connecting to WyzeCam Outdoor - Front Porch on 192.168.200.102 (1/3)

[tee @ 0x3f0cbd0] dimensions not set
Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
Error initializing output stream 0:0 --
2022/06/16 08:50:38 [Basement] Stream did not receive a frame for over 20s
2022/06/16 08:50:38 [MainProcess] 🎉 Connecting to WyzeCam V3 - Basement on 192.168.100.43 (1/3)
2022/06/16 08:50:42 [Basement] 📡 Getting 120kb/s HD stream (20fps) via LAN mode (WiFi: 51%) FW: 4.36.9.139 🔒 (DTLS) (2/3)
2022/06/16 08:50:42 [Basement] WARNING: Skipping smaller frame at start of stream (frame_size=1)
2022/06/16 08:50:43 [Back Door] IOTC_ER_TIMEOUT
2022/06/16 08:50:43 [Outdoor 3] IOTC_ER_TIMEOUT
2022/06/16 08:50:43 [Front Porch] IOTC_ER_TIMEOUT
2022/06/16 08:50:45 [MainProcess] ⏰ Timed out connecting to Outdoor 3 (20s).
2022/06/16 08:50:45 [MainProcess] ⏰ Timed out connecting to Back Door (20s).
2022/06/16 08:50:45 [MainProcess] ⏰ Timed out connecting to Front Porch (20s).
2022/06/16 08:50:55 [MainProcess] 🎉 Connecting to WyzeCam Outdoor - Outdoor 3 on 192.168.200.104 (1/3)
2022/06/16 08:50:55 [MainProcess] 🎉 Connecting to WyzeCam Outdoor - Back Door on 192.168.200.103 (1/3)
2022/06/16 08:50:56 [MainProcess] 🎉 Connecting to WyzeCam Outdoor - Front Porch on 192.168.200.102 (1/3)
2022/06/16 08:51:00 [Basement] WARNING: Frame not available yet

2022/06/16 08:51:06 [RTSP][BASEMENT] ✅ '/basement' stream is UP! (3/3)
2022/06/16 08:51:10 [RTSP][BASEMENT] 📖 New client reading
2022/06/16 08:51:16 [Outdoor 3] IOTC_ER_TIMEOUT
2022/06/16 08:51:16 [Front Porch] IOTC_ER_TIMEOUT
2022/06/16 08:51:16 [Back Door] IOTC_ER_TIMEOUT
2022/06/16 08:51:16 [RTSP][BASEMENT] 📕 Client stopped reading
2022/06/16 08:51:18 [MainProcess] ⏰ Timed out connecting to Outdoor 3 (20s).
2022/06/16 08:51:18 [MainProcess] ⏰ Timed out connecting to Back Door (20s).
2022/06/16 08:51:18 [MainProcess] ⏰ Timed out connecting to Front Porch (20s).
2022/06/16 08:51:28 [MainProcess] 🎉 Connecting to WyzeCam Outdoor - Outdoor 3 on 192.168.200.104 (1/3)
2022/06/16 08:51:28 [MainProcess] 🎉 Connecting to WyzeCam Outdoor - Back Door on 192.168.200.103 (1/3)
2022/06/16 08:51:28 [MainProcess] 🎉 Connecting to WyzeCam Outdoor - Front Porch on 192.168.200.102 (1/3)
2022/06/16 08:51:48 [Front Porch] IOTC_ER_TIMEOUT
2022/06/16 08:51:48 [Outdoor 3] IOTC_ER_TIMEOUT
2022/06/16 08:51:48 [Back Door] IOTC_ER_TIMEOUT

2022/06/16 08:51:50 [MainProcess] ⏰ Timed out connecting to Outdoor 3 (20s).
2022/06/16 08:51:50 [MainProcess] ⏰ Timed out connecting to Back Door (20s).
2022/06/16 08:51:50 [MainProcess] ⏰ Timed out connecting to Front Porch (20s).
2022/06/16 08:52:00 [MainProcess] 🎉 Connecting to WyzeCam Outdoor - Outdoor 3 on 192.168.200.104 (1/3)
2022/06/16 08:52:00 [MainProcess] 🎉 Connecting to WyzeCam Outdoor - Back Door on 192.168.200.103 (1/3)
2022/06/16 08:52:00 [MainProcess] 🎉 Connecting to WyzeCam Outdoor - Front Porch on 192.168.200.102 (1/3)
2022/06/16 08:52:20 [Front Porch] IOTC_ER_TIMEOUT
2022/06/16 08:52:20 [Back Door] IOTC_ER_TIMEOUT
2022/06/16 08:52:20 [Outdoor 3] IOTC_ER_TIMEOUT
2022/06/16 08:52:22 [MainProcess] ⏰ Timed out connecting to Outdoor 3 (20s).
2022/06/16 08:52:22 [MainProcess] ⏰ Timed out connecting to Back Door (20s).
2022/06/16 08:52:22 [MainProcess] ⏰ Timed out connecting to Front Porch (20s).
2022/06/16 08:52:32 [MainProcess] 🎉 Connecting to WyzeCam Outdoor - Outdoor 3 on 192.168.200.104 (1/3)
2022/06/16 08:52:32 [MainProcess] 🎉 Connecting to WyzeCam Outdoor - Back Door on 192.168.200.103 (1/3)
2022/06/16 08:52:32 [MainProcess] 🎉 Connecting to WyzeCam Outdoor - Front Porch on 192.168.200.102 (1/3)
2022/06/16 08:52:52 [Outdoor 3] IOTC_ER_TIMEOUT
2022/06/16 08:52:52 [Back Door] IOTC_ER_TIMEOUT
2022/06/16 08:52:52 [Front Porch] IOTC_ER_TIMEOUT

@r14n
Copy link

r14n commented Jun 22, 2022

@mrlt8 hi, anything in the logs that you see that can help me get this working well?

@z062007vette
Copy link

I have a similar issue I think. Here is the logs just from today. I can pull more if needed. Hoping this helps.

mrlt8-wyze-bridge1-copy.csv

@mrlt8 mrlt8 added bug Something isn't working enhancement New feature or request labels Jun 23, 2022
mrlt8 added a commit that referenced this issue Jul 3, 2022
* Add flask endpoints to restart connection to cams/rtsp-server #391

* Restart buttons for webui
@mdabbs
Copy link

mdabbs commented Aug 12, 2022

The endpoints seem to crash when I use them. But something a little more automatic would be nice. I just get tons of "2022/08/12 21:12:39 [WyzeBridge] ⏰ Timed out connecting to BridgeTest (20s)." Maybe something that when it gets a few of these in a row, it would restart that camera or something.

@mrlt8
Copy link
Owner

mrlt8 commented Aug 13, 2022

Unfortunately, I haven't been able to find a way to reboot the cameras over the TUTK protocol - I believe the wyze app reboots the cameras over their web API so that's something we might be able to use.

@mrlt8
Copy link
Owner

mrlt8 commented Aug 13, 2022

The endpoints seem to crash when I use them.

I see that bug now.

⏰ Timed out connecting to BridgeTest (20s)."

And this one is a separate bug. Just ran into it myself - your camera wasn't timing out the bridge was just using the wrong timestamp!

Will be pushing a fix for both of these!

@mdabbs
Copy link

mdabbs commented Aug 13, 2022

I may have used the wrong term. The physical cameras don't need to be rebooted, its just eh docker container needs to be restarted. So whatever happens with that would just need to happen after a certain number of timeouts. I never actually reboot or power cycle the physical camera, I just have to hit ctrl-c and restart the docker container.

mrlt8 added a commit that referenced this issue Aug 13, 2022
@kiwi-cam
Copy link
Contributor

I have a similar issue with just one of my 4 cameras. I can barely get 15 minutes of uptime from it before it's connection drops. The Camera appears fine in the Wyze app - although does seem to take a little longer than others to connect sometimes (related??). I've reset and re-flashed the camera.

If it's helpful, here are my debug logs:
WyzeBridgeLogs.txt

@dnestico
Copy link
Author

@kiwi-cam Same, I even moved the bridge closer to all the cameras and this still happens, I wouldn't say its as bad as 15 minutes for me but its definitely getting annoying and I've already put the RTSP firmware on one of V3s and that one has never gone down once.

@serenityrox
Copy link

serenityrox commented Aug 14, 2022 via email

@dnestico
Copy link
Author

@serenityrox Gonna have to try this, I have it on any right now.

@dnestico
Copy link
Author

@serenityrox

So I have it on P2P mode only and I don't think it's working? At least for all cameras.

LOGS:

2022/08/13 20:45:36 [WyzeBridge] 🚀 STARTING DOCKER-WYZE-BRIDGE v1.8.2
2022/08/13 20:45:36 [WyzeBridge] 🏠 Home Assistant Mode

  • Serving Flask app 'frontend'
  • Debug mode: off
    2022/08/13 20:45:36 [WyzeBridge] ♻️ FORCED REFRESH - Ignoring local 'user' data
    2022/08/13 20:45:36 [WyzeBridge] ♻️ FORCED REFRESH - Ignoring local 'auth' data
    2022/08/13 20:45:36 [WyzeBridge] ☁️ Fetching 'auth' from the Wyze API...
    2022/08/13 20:45:36 [WyzeBridge] WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
  • Running on all addresses (0.0.0.0)
  • Running on http://127.0.0.1:5000
  • Running on http://172.30.33.2:5000
    2022/08/13 20:45:36 [WyzeBridge] Press CTRL+C to quit
    2022/08/13 20:45:36 [WyzeBridge] 💾 Saving 'auth' to local cache...
    2022/08/13 20:45:36 [WyzeBridge] ☁️ Fetching 'user' from the Wyze API...
    2022/08/13 20:45:36 [WyzeBridge] 💾 Saving 'user' to local cache...
    2022/08/13 20:45:36 [WyzeBridge] ♻️ FORCED REFRESH - Ignoring local 'cameras' data
    2022/08/13 20:45:36 [WyzeBridge] ☁️ Fetching 'cameras' from the Wyze API...
    2022/08/13 20:45:37 [WyzeBridge] 💾 Saving 'cameras' to local cache...
    2022/08/13 20:45:37 [WyzeBridge]
    🪄 BLACKLIST MODE ON
    2022/08/13 20:45:37 [WyzeBridge]
    🎬 STARTING 5 OF 6 CAMERAS
    2022/08/13 20:45:38 [WyzeBridge] Starting rtsp-simple-server v0.19.3
    2022/08/13 20:45:38 [WyzeBridge] 🎉 Connecting to WyzeCam V3 - Luna cam RTSP on 192.168.2.105 (1/3)
    2022/08/13 20:45:38 [WyzeBridge] 🎉 Connecting to WyzeCam Doorbell - Front Doorbell on 192.168.2.13 (1/3)
    2022/08/13 20:45:38 [Luna cam RTSP] ⌛️ Waiting for on-demand connection...
    2022/08/13 20:45:38 [WyzeBridge] 🎉 Connecting to WyzeCam V3 - Driveway on 192.168.2.30 (1/3)
    2022/08/13 20:45:38 [Front Doorbell] ⌛️ Waiting for on-demand connection...
    2022/08/13 20:45:38 [WyzeBridge] 🎉 Connecting to WyzeCam V2 - Garage Cam on 192.168.2.31 (1/3)
    2022/08/13 20:45:38 [Driveway] ⌛️ Waiting for on-demand connection...
    2022/08/13 20:45:38 [WyzeBridge] 🎉 Connecting to WyzeCam V2 - Porch on 192.168.2.44 (1/3)
    2022/08/13 20:45:38 [Garage Cam] ⌛️ Waiting for on-demand connection...
    2022/08/13 20:45:38 [Porch] ⌛️ Waiting for on-demand connection...
    2022/08/13 20:45:58 [WyzeBridge] 127.0.0.1 - - [13/Aug/2022 20:45:58] "GET /events/start/porch HTTP/0.9" 200 -
    2022/08/13 20:46:00 [Porch] 📡 Getting 120kb/s HD stream (15fps) via LAN mode (WiFi: 80%) FW: 4.9.9.1355 🔒 (DTLS) (2/3)
    2022/08/13 20:46:00 [Porch] 🔊 Audio Enabled - MULAW/8,000Hz
    2022/08/13 20:46:02 [RTSP][PORCH] ✅ '/porch' stream is UP! (3/3)
    Output file #0 does not contain any stream
    2022/08/13 20:46:05 [RTSP][PORCH] 📖 New client reading

@dnestico
Copy link
Author

dnestico commented Aug 14, 2022

Ok update I gave it a minute and all the cameras say LAN mode now and camera streams seem to load instantly now.., but at the top you can see it didn't say LAN.

Now to test if this fixes anything.

mrlt8 added a commit that referenced this issue Aug 14, 2022
* Add individual camera api endpoints #436 #494

* Use updated camera data

* use name_uri instead of nickname

* icon/color based on status/on-demand #498

* Fix boa_interval style #504

* Server side event for camera status

* Update camera status icon color

* Update compatibility table

* Fix restarting connections #391

* Prevent camera from wrongly timing out #391 #484

* kill zombies?

* More verbose http exceptions #505

* Update preview button for all non-HLS

* Load preview on connection and message on lost connection

* copy dictionary

* version bump

* update preview when connected to camera
@mrlt8
Copy link
Owner

mrlt8 commented Aug 14, 2022

v1.8.3 should fix the timeout and restart connection issues.

@dnestico
Copy link
Author

Alright will update tomorrow, so far what is locking it to P2P mode its loading nice and fast so I think ill keep that too even after I update.

@serenityrox
Copy link

serenityrox commented Aug 14, 2022 via email

@mjb83
Copy link

mjb83 commented Aug 14, 2022

v1.8.3 should fix the timeout and restart connection issues.

Updated to 1.8.3 about 2 hours ago, all 3 cameras are up stable and running. I switched to P2P mode as well, it seems to be working well I'll update again if I see any issues but so far so good.

EDIT: 12 hours and still relatively stable. I say relative because it looks like they are crashing periodically, but they do seem to auto restart fairly quickly.

@kiwi-cam
Copy link
Contributor

I updated about 4 hours ago. Looking better here too - all cameras are still up.
I’ve stuck with ANY (LAN mode).

@z062007vette
Copy link

Latest update is much more stable. A few disconnects, but recovers on it's own. Awesome work dev!

@dnestico
Copy link
Author

So far can confirm the exact same!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

8 participants