Skip to content

Commit

Permalink
Support ManagedMediaSource and update docs to reflect iOS 17.1+ suppo…
Browse files Browse the repository at this point in the history 
…rts MSE (#8160)

* Refactor media source handling in MsePlayer.js and Birdseye.jsx to support ManagedMediaSource

* lint

* Update docs to reflect iOS supporting mse

---------

Co-authored-by: Sergey Krashevich <svk@svk.su>
  • Loading branch information
@NickM-27 @skrashevich
NickM-27 and skrashevich committed Oct 14, 2023
1 parent 4c7ea01 commit fa6c6c5
Show file tree
Hide file tree
Showing 4 changed files with 35 additions and 27 deletions.
10 changes: 5 additions & 5 deletions docs/docs/configuration/live.md
View file
Expand Up @@ -9,11 +9,11 @@ Frigate has different live view options, some of which require the bundled `go2r

Live view options can be selected while viewing the live stream. The options are:

| Source | Latency | Frame Rate | Resolution | Audio | Requires go2rtc | Other Limitations |
| ------ | ------- | ------------------------------------- | -------------- | ---------------------------- | --------------- | -------------------------------------------- |
| jsmpeg | low | same as `detect -> fps`, capped at 10 | same as detect | no | no | none |
| mse | low | native | native | yes (depends on audio codec) | yes | not supported on iOS, Firefox is h.264 only |
| webrtc | lowest | native | native | yes (depends on audio codec) | yes | requires extra config, doesn't support h.265 |
| Source | Latency | Frame Rate | Resolution | Audio | Requires go2rtc | Other Limitations |
| ------ | ------- | ------------------------------------- | -------------- | ---------------------------- | --------------- | ------------------------------------------------- |
| jsmpeg | low | same as `detect -> fps`, capped at 10 | same as detect | no | no | none |
| mse | low | native | native | yes (depends on audio codec) | yes | iPhone requires iOS 17.1+, Firefox is h.264 only |
| webrtc | lowest | native | native | yes (depends on audio codec) | yes | requires extra config, doesn't support h.265 |

### Audio Support

Expand Down
44 changes: 26 additions & 18 deletions web/src/components/MsePlayer.js
View file
Expand Up @@ -157,12 +157,9 @@ class VideoRTC extends HTMLElement {
if (this.ws) this.ws.send(JSON.stringify(value));
}

codecs(type) {
const test =
type === 'mse'
? (codec) => MediaSource.isTypeSupported(`video/mp4; codecs="${codec}"`)
: (codec) => this.video.canPlayType(`video/mp4; codecs="${codec}"`);
return this.CODECS.filter(test).join();
/** @param {Function} isSupported */
codecs(isSupported) {
return this.CODECS.filter(codec => isSupported(`video/mp4; codecs="${codec}"`)).join();
}

/**
Expand Down Expand Up @@ -311,7 +308,7 @@ class VideoRTC extends HTMLElement {

const modes = [];

if (this.mode.indexOf('mse') >= 0 && 'MediaSource' in window) {
if (this.mode.indexOf('mse') >= 0 && ('MediaSource' in window || 'ManagedMediaSource' in window)) {
// iPhone
modes.push('mse');
this.onmse();
Expand Down Expand Up @@ -363,18 +360,29 @@ class VideoRTC extends HTMLElement {
}

onmse() {
const ms = new MediaSource();
ms.addEventListener(
'sourceopen',
() => {
/** @type {MediaSource} */
let ms;

if ('ManagedMediaSource' in window) {
const MediaSource = window.ManagedMediaSource;

ms = new MediaSource();
ms.addEventListener('sourceopen', () => {
this.send({type: 'mse', value: this.codecs(MediaSource.isTypeSupported)});
}, {once: true});

this.video.disableRemotePlayback = true;
this.video.srcObject = ms;
} else {
ms = new MediaSource();
ms.addEventListener('sourceopen', () => {
URL.revokeObjectURL(this.video.src);
this.send({ type: 'mse', value: this.codecs('mse') });
},
{ once: true }
);
this.send({type: 'mse', value: this.codecs(MediaSource.isTypeSupported)});
}, {once: true});

this.video.src = URL.createObjectURL(ms);
this.video.srcObject = null;
this.video.src = URL.createObjectURL(ms);
this.video.srcObject = null;
}
this.play();

this.mseCodecs = '';
Expand Down Expand Up @@ -580,7 +588,7 @@ class VideoRTC extends HTMLElement {
video2.src = `data:video/mp4;base64,${VideoRTC.btoa(data)}`;
};

this.send({ type: 'mp4', value: this.codecs('mp4') });
this.send({ type: 'mp4', value: this.codecs(this.video.canPlayType) });
}

static btoa(buffer) {
Expand Down
4 changes: 2 additions & 2 deletions web/src/routes/Birdseye.jsx
View file
Expand Up @@ -35,7 +35,7 @@ export default function Birdseye() {
let player;
const playerClass = ptzCameras.length || isMaxWidth ? 'w-full' : 'max-w-5xl xl:w-1/2';
if (viewSource == 'mse' && config.birdseye.restream) {
if ('MediaSource' in window) {
if ('MediaSource' in window || 'ManagedMediaSource' in window) {
player = (
<Fragment>
<div className={playerClass}>
Expand All @@ -50,7 +50,7 @@ export default function Birdseye() {
player = (
<Fragment>
<div className="w-5xl text-center text-sm">
MSE is not supported on iOS devices. You'll need to use jsmpeg or webRTC. See the docs for more info.
MSE is only supported on iOS 17.1+. You'll need to update if available or use jsmpeg / webRTC streams. See the docs for more info.
</div>
</Fragment>
);
Expand Down
4 changes: 2 additions & 2 deletions web/src/routes/Camera.jsx
View file
Expand Up @@ -116,7 +116,7 @@ export default function Camera({ camera }) {
let player;
if (viewMode === 'live') {
if (viewSource == 'mse' && restreamEnabled) {
if ('MediaSource' in window) {
if ('MediaSource' in window || 'ManagedMediaSource' in window) {
player = (
<Fragment>
<div className="max-w-5xl">
Expand All @@ -133,7 +133,7 @@ export default function Camera({ camera }) {
player = (
<Fragment>
<div className="w-5xl text-center text-sm">
MSE is not supported on iOS devices. You'll need to use jsmpeg or webRTC. See the docs for more info.
MSE is only supported on iOS 17.1+. You'll need to update if available or use jsmpeg / webRTC streams. See the docs for more info.
</div>
</Fragment>
);
Expand Down

0 comments on commit fa6c6c5

Please sign in to comment.