Optimized video loading using HLS

[jb55]

We can optimize video streaming via HLS. HLS is a way to do adaptive bitrate streaming for poor connections. The ideal scenario is to monitor video links from the network and then kick off an HLS transcoding process via a custom video transcoding proxy. Something similar to imgproxy but for videos and one that support HLS.

After a quick google it looks like Cloudflare might support this via their "Cloudflare Stream Delivery" service:

https://www.cloudflare.com/products/stream-delivery/

Cloudflare’s global Anycast network ensures fast video delivery, with shorter video startup times and reduced buffering, no matter where your visitors are located.

We cache and deliver HTTP(S) video content, including adaptive bitrate formats, to your visitors at a flat-rate price, saving you on origin server bandwidth costs.

Our rich ecosystem of performance and security services, tuned specifically for video, offer optimal uptime and delivery.

This should increase video performance a ton, which would be a big deal

[jb55]

@fishcakeday have ya'll looked into this?

[fishcakeday]

DASH is fine and all, but HLS is more widely supported. As for the CF, I have, and it is priced per streamed minute, hence the costs are prohibitive. I do have a prototype of transcoder that does it, and I already added support to Damus for HLS in my last contribution some time ago.

[alltheseas]

HLS works fine. If you search my npub for "baywatch" I reference a npub that shares a bunch of HLS streams

[jb55]

For some reason I thought HLS was a live streaming thing, it looks like its most supported by apple so that would be ideal.

The goal would be for damus to detect any video link and find an HLS version of it via some proxy somehow

[jb55]

To be clear. The point is not to just support HLS via m3u8 links, which we already have and maybe mistakenly add a "live" label to.

The goal is to improve video across the board by an HLS proxy. This may be expensive as @fishcakeday mentions so we need to evaluate this as maybe a purple-only thing.

[jb55]

As a potential stepping stone to an HLS proxy, we could at least create a service that transcodes the video on upload and posts an m3u8 link instead of mov/mp4

[fishcakeday]

To be clear. The point is not to just support HLS via m3u8 links, which we already have and maybe mistakenly add a "live" label to.

The goal is to improve video across the board by an HLS proxy. This may be expensive as @fishcakeday mentions so we need to evaluate this as maybe a purple-only thing.

Will, you reviewed my CR and I clearly stated that “live” is added to the HLS videos that are live, VOD is not labeled live.

[jb55]

ok good! I must have missed that.

[fishcakeday]

I would also not recommend any proxying of the videos, or you end up with a huge bill and inefficient implementation that does the work over and over and produces suboptimal quality

[jb55]

inefficient implementation that does the work over and over and produces suboptimal quality

one idea I had was a proxy that just puts the original file in the manifest with no segments on first load, then a transcoding job associated with the original video would kick off and update the manifest over time for future requests

I am not sure if there are services that do this already. This could be its own project like imgproxy.

[fishcakeday]

NB service can technically speaking do this already, serve the original while the transcoding is not finished. Then, depending on the client agent string in the header, serve m3u8 with multiple levels of bitrate that are optimized for quality and fast delivery in chunks.

[jb55]

That would be amazing, I am super interested in implementing that if its possible on your end.

[fishcakeday]

You do not need to do anything on Damus, it is the matter of me taking my initial prototype to the production level, and it would work as is. The problem is costs for initial transcoding and potential double storage. Social media does not need that many bitrates, 3 would work fine usually. Time to transcode is somewhere between 1 and 3 minutes per minute of video in 3 resolutions. Short videos under 30 seconds are done very fast. This all was prototyped on a scalable infra, so no hard limit on how many videos are uploaded at the same time.

[jb55]

interesting, would this result in a post with m3u8 as the extension or would mp4$ return an hls content type?

[jb55]

I would also not recommend any proxying of the videos, or you end up with a huge bill and inefficient implementation that does the work over and over and produces suboptimal quality

the issue still remains for non-nostr-build videos. I guess we could look into a proxy for those cases... even if it was just for privacy. which reminds me we still need:

• Image Proxy #2098

[fishcakeday]

no change in the name, and it does not have to matter, served mime type would dictate how the video is processed. For example, user get and https://example.com/video.mov as a link, while in the background service transcodes, the original is served. Once the transcoding is done, the serving endpoint starts serving m3u8 content when the request comes with appropriate mime type. Since not all clients support it, let's say if UA is "damus/*" we serve m3u8, otherwise we continue serving the original.

[jb55]

I see! I don't think we set our user agent, but we could if that helps on your end. I think it might be some generic apple UA?

[jb55]

Started gathering Media-related optimization issues at:

• Media Optimization #2214

[fishcakeday]

You do, according to my observations.

Video UA:

[jb55]

ah ok good to know, apple must do this for us