Don't pre-apprend baseurl to each fragment in DASH/other fragmented protocols

[wiiaboo]

• I've verified and I assure that I'm running youtube-dl 2017.05.01
• At least skimmed through README and most notably FAQ and BUGS sections
• Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

• Feature request (request for a new functionality)

---

Example URL:

• https://www.youtube.com/watch?v=XmL19DOP_Ls (9h53m40s)

As it is now, with each fragment url being pre-appended by the baseURL from the manifest, the JSON exported from this URL reaches 31.1MB. If I change Youtube-DL to not do this, the resulting JSON is only 5.45MB.

Considering baseURL is already available in the ".requested_formats[*].url" key, the API user could check if the fragment URL starts with 'http' and pre-append the baseURL itself if not, or some other new key could be added for the baseURL.

[yan12125]

Related: #9109, which removes fragment URLs completely

[wiiaboo]

Fragment URLs are still needed for mpv until FFmpeg gets a DASH demuxer, though. It's not like the huge JSON prohibits playback, it's just a lot of repeated text for no reasonable gain.

[yan12125]

#9109 is a step towards live DASH streams. (#10787) As fragment URLs change as time goes by, it's not a good idea for youtube-dl to place them in extractor outputs (-j). Do you know how mpv handles live DASH streams?

[wiiaboo]

It doesn't, most likely. Any example URLs?

mpv doesn't continually get the JSON from Youtube-DL, it just does a single query, so it's not essential to produce fragments list at all with live content. Things like streamlink/livestreamer or youtube-dl -o - | mpv - would be required to support livestreams. There hasn't been any demand for that on mpv's side, since there's still HLS streams, which do work fine.

[yan12125]

Facebook live videos use DASH. A current example is https://www.facebook.com/cs50/videos/10154276859382127/. You can find more on https://www.facebook.com/livemap. Note that you'll need to login to get video URLs on the livemap.

…

On Sat, May 6, 2017 at 12:34 AM, RiCON ***@***.***> wrote: It doesn't, most likely. Any example URLs? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#12990 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB2RGYFsJW2OwquvE3FkOfx_pYJJzIoOks5r20-NgaJpZM4NR_ZJ> .

[wiiaboo]

That works fine though, that's not segmented dash.

Videos from the livemap though don't work, since the urls are just dash manifests.

[wiiaboo]

Youtube-DL doesn't do anything with it either. It just downloads the manifest.

[yan12125]

Youtube-DL doesn't do anything with it either. It just downloads the manifest.

Live DASH streams are not supported yet, so I decide to just return the DASH URL so that users can feed it to capable players. For example:

MP4Client $(youtube-dl -g url_of_the_live_video)

I mention the case here because I hope there will be a good design so that both mpv and youtube-dl will work for live DASH in the future. Personally I don't like the youtube-dl -o - | mpv - way as it's not the same smooth as the youtube-dl integration in mpv.

[wiiaboo]

The future will be a working/valid DASH demuxer in FFmpeg. It's probably out-of-scope for mpv.

Trying the patch in the ML it seems to work at least with Youtube's and Vimeo's non-live .mpd.

[yan12125]

Once DASH support lands in ffmpeg, mpv doesn't need fragment URLs from youtube-dl anymore, right?

For now, minimizing -j output is still a good feature for static (non-live) MPDs.

[wiiaboo]

Since it's a huge patch with potentially a few security issues it won't be merged so soon (maybe months).
mpv would be fine with live DASH not working meanwhile, as far as my opinion goes.