Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support ManagedMediaSource and update docs to reflect iOS 17.1+ supports MSE #8160

Merged
merged 3 commits into from Oct 14, 2023
Merged

Support ManagedMediaSource and update docs to reflect iOS 17.1+ supports MSE #8160

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
fe54b65
Refactor media source handling in MsePlayer.js and Birdseye.jsx to su…
skrashevich Sep 27, 2023
cafdacf
lint
skrashevich Sep 27, 2023
69f0877
Update docs to reflect iOS supporting mse
NickM-27 Oct 13, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
10 changes: 5 additions & 5 deletions docs/docs/configuration/live.md
View file
Open in desktop
Expand Up @@ -9,11 +9,11 @@ Frigate has different live view options, some of which require the bundled `go2r

Live view options can be selected while viewing the live stream. The options are:

| Source | Latency | Frame Rate | Resolution | Audio | Requires go2rtc | Other Limitations |
| ------ | ------- | ------------------------------------- | -------------- | ---------------------------- | --------------- | -------------------------------------------- |
| jsmpeg | low | same as `detect -> fps`, capped at 10 | same as detect | no | no | none |
| mse | low | native | native | yes (depends on audio codec) | yes | not supported on iOS, Firefox is h.264 only |
| webrtc | lowest | native | native | yes (depends on audio codec) | yes | requires extra config, doesn't support h.265 |
| Source | Latency | Frame Rate | Resolution | Audio | Requires go2rtc | Other Limitations |
| ------ | ------- | ------------------------------------- | -------------- | ---------------------------- | --------------- | ------------------------------------------------- |
| jsmpeg | low | same as `detect -> fps`, capped at 10 | same as detect | no | no | none |
| mse | low | native | native | yes (depends on audio codec) | yes | iPhone requires iOS 17.1+, Firefox is h.264 only |
| webrtc | lowest | native | native | yes (depends on audio codec) | yes | requires extra config, doesn't support h.265 |

### Audio Support

Expand Down
44 changes: 26 additions & 18 deletions web/src/components/MsePlayer.js
View file
Open in desktop
Expand Up @@ -157,12 +157,9 @@ class VideoRTC extends HTMLElement {
if (this.ws) this.ws.send(JSON.stringify(value));
}

codecs(type) {
const test =
type === 'mse'
? (codec) => MediaSource.isTypeSupported(`video/mp4; codecs="${codec}"`)
: (codec) => this.video.canPlayType(`video/mp4; codecs="${codec}"`);
return this.CODECS.filter(test).join();
/** @param {Function} isSupported */
codecs(isSupported) {
return this.CODECS.filter(codec => isSupported(`video/mp4; codecs="${codec}"`)).join();
}

/**
Expand Down Expand Up @@ -311,7 +308,7 @@ class VideoRTC extends HTMLElement {

const modes = [];

if (this.mode.indexOf('mse') >= 0 && 'MediaSource' in window) {
if (this.mode.indexOf('mse') >= 0 && ('MediaSource' in window || 'ManagedMediaSource' in window)) {
// iPhone
modes.push('mse');
this.onmse();
Expand Down Expand Up @@ -363,18 +360,29 @@ class VideoRTC extends HTMLElement {
}

onmse() {
const ms = new MediaSource();
ms.addEventListener(
'sourceopen',
() => {
/** @type {MediaSource} */
let ms;

if ('ManagedMediaSource' in window) {
const MediaSource = window.ManagedMediaSource;

ms = new MediaSource();
ms.addEventListener('sourceopen', () => {
this.send({type: 'mse', value: this.codecs(MediaSource.isTypeSupported)});
}, {once: true});

this.video.disableRemotePlayback = true;
this.video.srcObject = ms;
} else {
ms = new MediaSource();
ms.addEventListener('sourceopen', () => {
URL.revokeObjectURL(this.video.src);
this.send({ type: 'mse', value: this.codecs('mse') });
},
{ once: true }
);
this.send({type: 'mse', value: this.codecs(MediaSource.isTypeSupported)});
}, {once: true});

this.video.src = URL.createObjectURL(ms);
this.video.srcObject = null;
this.video.src = URL.createObjectURL(ms);
this.video.srcObject = null;
}
this.play();

this.mseCodecs = '';
Expand Down Expand Up @@ -580,7 +588,7 @@ class VideoRTC extends HTMLElement {
video2.src = `data:video/mp4;base64,${VideoRTC.btoa(data)}`;
};

this.send({ type: 'mp4', value: this.codecs('mp4') });
this.send({ type: 'mp4', value: this.codecs(this.video.canPlayType) });
}

static btoa(buffer) {
Expand Down
4 changes: 2 additions & 2 deletions web/src/routes/Birdseye.jsx
View file
Open in desktop
Expand Up @@ -35,7 +35,7 @@ export default function Birdseye() {
let player;
const playerClass = ptzCameras.length || isMaxWidth ? 'w-full' : 'max-w-5xl xl:w-1/2';
if (viewSource == 'mse' && config.birdseye.restream) {
if ('MediaSource' in window) {
if ('MediaSource' in window || 'ManagedMediaSource' in window) {
player = (
<Fragment>
<div className={playerClass}>
Expand All @@ -50,7 +50,7 @@ export default function Birdseye() {
player = (
<Fragment>
<div className="w-5xl text-center text-sm">
MSE is not supported on iOS devices. You'll need to use jsmpeg or webRTC. See the docs for more info.
MSE is only supported on iOS 17.1+. You'll need to update if available or use jsmpeg / webRTC streams. See the docs for more info.
</div>
</Fragment>
);
Expand Down
4 changes: 2 additions & 2 deletions web/src/routes/Camera.jsx
View file
Open in desktop
Expand Up @@ -116,7 +116,7 @@ export default function Camera({ camera }) {
let player;
if (viewMode === 'live') {
if (viewSource == 'mse' && restreamEnabled) {
if ('MediaSource' in window) {
if ('MediaSource' in window || 'ManagedMediaSource' in window) {
player = (
<Fragment>
<div className="max-w-5xl">
Expand All @@ -133,7 +133,7 @@ export default function Camera({ camera }) {
player = (
<Fragment>
<div className="w-5xl text-center text-sm">
MSE is not supported on iOS devices. You'll need to use jsmpeg or webRTC. See the docs for more info.
MSE is only supported on iOS 17.1+. You'll need to update if available or use jsmpeg / webRTC streams. See the docs for more info.
</div>
</Fragment>
);
Expand Down