Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segmented video architecture to support live streaming and long videos #1237

Closed
scanlime opened this issue Oct 10, 2018 · 23 comments

Comments

7 participants
@scanlime
Copy link
Contributor

commented Oct 10, 2018

Hi, I'm quite interested in solutions to #970 and #151. Both of these issues have at their root that PeerTube currently requires each video to be a single media file.

For background: There are multiple possible solutions to this problem, including very common ones (HTTP Live Streaming or DASH) and more experimental (github/Novage/p2p-media-loader). The most typical solution would be to use a growing .m3u8 playlist full of very short mp4 files which optionally grows during a broadcast. This same playlist format can be used for non-streaming (video on demand) content to provide fast loading and seeking even on very long streams.

I'm opening this issue to ask the PeerTube developers whether it's realistic to expect a solution to these issues in the current architecture. If so, this may be a good place to discuss a migration plan, which may involve a way for the protocol and servers to support both segmented and whole video files. If not, it would be good to know why not, and we can perhaps move on to other projects that still try to integrate at the ActivityPub level, and this may be a good place to plan for protocol interoperability.

@scanlime

This comment has been minimized.

Copy link
Contributor Author

commented Oct 13, 2018

In #1253, Chocobozzz says the video format is stable. I assume this means Peertube wouldn't consider replacing either current format (mp4 over HTTP, and the same but wrapped in WebTorrent) with an alternative segmented format (HLS, DASH, etc).

Replacing the format entirely would be a compelling option in some ways, since it leads to a straightforward server without any duplication of the videos. It burdens the client with remuxing in the event users want to download the video as a single file.

Another option might be to include HLS (.m3u8) as well as playlist-of-torrent formats alongside the current monolithic video files. This would either require duplicate storage space, or on-the-fly remuxing.

Another alternative is that videos could have multiple types of backend storage, with the missing formats generated on-the-fly until they've been requested frequently enough to cache.

Would it be at all possible to switch entirely to a new segmented format, or are we definitely stuck supporting multiple formats?

@scanlime

This comment has been minimized.

Copy link
Contributor Author

commented Oct 13, 2018

Another format option: Segmented MP4 wrapped in DASH. We can use a DASH MPD file to describe all available resolutions, and we can use multiple files if necessary during a live-stream. After live-streaming, or for single-file uploads, we'd remux into a segmented MP4 that can be used as-is or the individual segments can be referenced from the DASH MPD.

Mp4box is a tool to remux mp4 files using segments,
https://gpac.wp.imt.fr/2011/02/02/mp4box-fragmentation-segmentation-splitting-and-interleaving/

@scanlime

This comment has been minimized.

Copy link
Contributor Author

commented Oct 13, 2018

Looks like ffmpeg can produce fragmented moov atoms on its own too. Not sure if we'd need to create the manifest ourselves in that case,

https://stackoverflow.com/a/9734251

@Serkan-devel

This comment has been minimized.

Copy link

commented Dec 18, 2018

I thought I head something like this but under the term, stream rebroadcasting.
On a talk about webrtc and the phoenix web framework, they mentioned that there will be something like this in the future
screenshot_20181218-131717

@Chocobozzz

This comment has been minimized.

Copy link
Owner

commented Dec 18, 2018

It was in 2016 so technically we are in the future :p

There is https://github.com/Novage/p2p-media-loader that is also interesting

@Serkan-devel

This comment has been minimized.

Copy link

commented Dec 18, 2018

are there any roadblocks to implement it?

@scanlime

This comment has been minimized.

Copy link
Contributor Author

commented Dec 18, 2018

@Serkan-devel

This comment has been minimized.

Copy link

commented Dec 18, 2018

that might require many messages sent for a video

@Chocobozzz

This comment has been minimized.

Copy link
Owner

commented Jan 14, 2019

@scanlime Adding the DASH/HLS playlist URLs in the AP project, while continuing to provide the torrent file.

@SerCom-KC I'll try to investigate how we could integrate this library in PeerTube this year. But p2p-media-loader still lacks support of checking segment integrity and I don't know yet how we'll be able to provide multiple URLs for a single segment (redundancy system/CDN). (edit by rigelk: Chocobozzz made a PR adding segment integrity and it has been merged into p2p-media-loader)

@Chocobozzz

This comment has been minimized.

Copy link
Owner

commented Jan 24, 2019

WIP to add HLS support in our player (in addition to webtorrent): #1566. It uses p2p-media-loader to share segments with WebRTC. Then I'll try to work on the server side, to generate the playlists/segments.

HLS transcoding will be opt-in, because we'll need to have the mp4 raw files + mpeg segments (therefore multiply the videos storage by 2).

If our HLS player works well, we can imagine adding support of livestream (#151) in the future in it (livestream is not be possible/easy with webtorrent).

Dash support for videojs is not implemented in p2p-media-loader yet.

@mrlika

This comment has been minimized.

Copy link

commented Feb 5, 2019

mpeg segments

It is better to use fmp4 segments in HLS because they do not require transmuxing. MPEG-TS segments are transmuxed into fmp4 segments on the fly to be added for playback. And you can have all the segments in one fmp4 file. In HLS manifest you should specify byte ranges of all the segments including initialization segment.
In this way, you can have both raw mp4 and HLS manifest for it. But in this case HLS is overhead because raw fmp4 file is natively supported almost everywhere. HLS is good when you have multiple qualities of the same video - few fmp4 files with fragments of the same duration but of different qualities.

See example:

fMP4 file: https://storage.googleapis.com/wvtemp/rkuroiwa/hls_single_segment/sintel_single_segment_hls.mp4

HLS manifest for it: https://storage.googleapis.com/wvtemp/rkuroiwa/hls_single_segment/sintel_1080p_single_segment.m3u8

@mrlika

This comment has been minimized.

Copy link

commented Feb 5, 2019

I think single fMP4 is the same thing that is progressively downloaded and played by WebTorrent library

@Chocobozzz

This comment has been minimized.

Copy link
Owner

commented Feb 6, 2019

@mrlika Thanks for your feedback.

I considered fragmented mp4 so we don't have to generate 2 different files for every resolution, but unfortunately the library we use to stream webtorrent files does not support them: jhiesey/videostream#20

@jimmywarting

This comment has been minimized.

Copy link

commented Feb 6, 2019

(livestream is not be possible/easy with webtorrent).

Would mutable torrent and webtorrent/webtorrent#886 solve live streaming by using webtorrent?

@Chocobozzz

This comment has been minimized.

Copy link
Owner

commented Feb 6, 2019

Would mutable torrent and webtorrent/webtorrent#886 solve live streaming by using webtorrent?

I don't really know, but I don't think we can find a cleaner way to do live streaming than using a well known protocol (HLS) with a custom loader that do P2P for some segments (p2p-media-loader).

@Chocobozzz

This comment has been minimized.

Copy link
Owner

commented Feb 6, 2019

I considered fragmented mp4 so we don't have to generate 2 different files for every resolution, but unfortunately the library we use to stream webtorrent files does not support them: jhiesey/videostream#20

Or maybe I can try to generate a fragmented mp4 file that also contains the moov atom at the beginning of the file 🤔

EDIT: does not work

@Chocobozzz

This comment has been minimized.

Copy link
Owner

commented Feb 11, 2019

Implemented in #1566

We now use a fragmented mp4 instead multiple ts segments. But we still have 2 different files per resolution: 1 for HLS and 1 for WebTorrent. One day, if we are able to stream in the browser fragmented mp4 files (coming from webtorrent) we'll use only 1 file.

@Chocobozzz Chocobozzz closed this Feb 11, 2019

@Chocobozzz Chocobozzz added this to Done in Roadmap 2019 via automation Feb 11, 2019

@Chocobozzz Chocobozzz changed the title Discuss segmented video architecture to support live streaming and long videos Segmented video architecture to support live streaming and long videos Feb 11, 2019

@Chocobozzz

This comment has been minimized.

@scanlime

This comment has been minimized.

Copy link
Contributor Author

commented Feb 12, 2019

Cool! Loads very fast on Firefox on laptop here. Not sure if mobile is supported yet, I tried on iOS and it was a black screen.

@scanlime

This comment has been minimized.

Copy link
Contributor Author

commented Feb 12, 2019

Also, is there any chance we could have a demo of a longer video? I have a lot of videos that are 5 or 10 hours long, and it'd be great to see something like that load instantly.

@Chocobozzz

This comment has been minimized.

Copy link
Owner

commented Feb 14, 2019

@scanlime Should work on Android. I'll see why iOS does not work.

Also, is there any chance we could have a demo of a longer video? I have a lot of videos that are 5 or 10 hours long, and it'd be great to see something like that load instantly.

No sorry, but please try it :)

@ncryptid

This comment has been minimized.

Copy link

commented Mar 24, 2019

Sorry for commenting on a closed issue, but I was wondering how adding fragmented videos affects live streaming. Is that still something in the "maybe, but unlikely/difficult to implement" pile of requested features or is it now something that could conceivably be done?

@tcitworld

This comment has been minimized.

Copy link
Collaborator

commented Mar 24, 2019

@ncryptid Please see #151 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.