Prioritize RTC over MSE

[bdoiron74]

Describe what you are trying to accomplish and why in non technical terms
Right now with 0.14.0-da913d8 on Windows 11/Chrome, the live video is always MSE. According to the FAQ, this is being done on purpose:

How can I switch the live view type?
In Frigate 0.13, a dropdown menu in the UI allowed users to select between jsmpeg, MSE, or WebRTC. In 0.14, Frigate intelligently selects the best type of live view method for the cameras on your dashboard based on network conditions and features, so this is no longer user selectable.

If I connect to the go2rtc port 1984 and stream video from there, I get RTC - so it is availible.

Describe the solution you'd like
Some way to prioritize RTC over MSE over msjpeg

Describe alternatives you've considered

Additional context
With MSE, the live view seems to slowly drift away from actual 'live'. By right clicking on a live view tile and showing controls, you can see the progress bar creeping towards the left. Once the progress bar gets all the way to the left, the stream shows a still image from the past. So ultimately the problem is that it never 'catches up', meaning I am constantly refreshing the page to restart the live views. Also, with multiple cameras on a page, they don't all drift at the same rate and fall out of sync.

With RTC mode through go2rtc, I never have to refresh. Any minor network interruptions might stall the feed briefly, but it immediately recovers on its own - and all cameras stay perfectly in sync.

MSE progress bar:

[hawkeye217]

WebRTC requires extra configuration, opening/forwarding ports, etc. and does not support H.265/HEVC. Frigate used to default to WebRTC on Safari, and that caused lots of issues for users. MSE is much more well-supported and doesn't require additional setup, hence why it became the default in 0.14.

We don't have any intention on making the mode user-selectable again at this point, but we're looking at ways of improving the intelligent mode selection in future versions of Frigate.

I'm going to close this, but if you'd like to open a camera support discussion, that could be helpful to us - there are some ways to see why your MSE player is buffering, and we might be able to write some code to mitigate that or use that to more intelligently select the live mode.

[ss6s]

Hello, that's a bit of a shame. For me, MSE does not work properly with one camera and goes buffering. I'll probably have to go back to 0.13.

[NickM-27]

Have you tried the latest builds which fix the buffering?

[hawkeye217]

Feel free to stick with 0.13 if that works best for you. However, the OP and I worked together for a while and because of his help, I committed code to improve MSE so that it performs better than WebRTC now. Feel free to grab the latest dev image and give it a try.

[ss6s]

hi, unfortunately no - I have now tested it with 0.14.1-453a8d7. Unfortunately, the buffering via mse is still there. The buffering is also when I call the stream directly via gortc and mse. via webrtc the stream is clean and works without reloading.

[hawkeye217]

If you open a separate support discussion with your config and the requested details, perhaps we can help you solve the issue with the one camera.

[folosleg]

We got older cameras they produce a h264 feed, but for some odd reason they can't be played through MSE, but works flawless over RTC. Only solution at the moment is to ffmpeg reencode them, but it's resource intensive and absolutely unnecessary.

I'd like to suggest a simle option, something like this:

cameras:
cam1:
live:
stream_name: cam1#rtc
cam2
live:
stream_name: cam2#mse

obviously the default can stay MSE, but maybe it is a viable option, might be useful for some folks out there...

Cheers

[hawkeye217]

Frigate 0.14.1 now correctly falls back to WebRTC if configured when MSE fails. However, MSE should work with H.264 without issue. Feel free to open a support discussion and we can try to help sort out your issue.

[alwon1]

I created a issue on this without seeing this one but the reason I wanted it is different. In my case I'm using CloudFlare tunnels to allow access and would like to avoid potentially running into problems with the terms of use by streaming a bunch of media through them and webrtc allows too do that but I have no way to even make it try webrtc because mse will always work first. Maybe instead a way to disable mse on the server could be added instead

[tarvip]

Frigate 0.14.1 now correctly falls back to WebRTC if configured when MSE fails.

I wonder what would be the downside of making it configurable per camera. I also have issues with one camera where MSE fails but WebRTC works fine (it is not Frigate issue, it does not work even directly in go2rtc). It will eventually fall back to WebRTC, but it can take 20 or more seconds. For testing purpose modified the code (0.15.2) so that it always uses WebRTC, now loads much faster.

[amatsoukas]

I also would like an option to disable mse all together, or at least be able to change the priority to webrtc. I mostly use safari as a browser, and with mse some times it works for a while, but most of the time, after a minute or less the stream breaks and it switches to webrtc. But once it switches to webrtc, it works flawlessly. The thing is, that if you use pip from safari, and this switch happens, the stream in pip freezes and never recovers. So i would prefer to be webrtc from the beginning, which is more stable in safari.