Disable Fragments

[EnginePod]

• I've verified and I assure that I'm running youtube-dl 2016.12.01

Description of your issue, suggested solution and other information

When I use the -j argument to get JSON output I sometimes get ranges included and for longer videos the data's so big that it causes things to crash.

Here's one video among many with this problem: http://www.youtube.com/watch?v=gAAIw2NIOxc
Now try it with JSON by running this: youtube-dl -j 'http://www.youtube.com/watch?v=gAAIw2NIOxc' and it will flood the console with fragments.

Is there a way I can disable fragments so I get less data (and avoid the crashes)?

[yan12125]

Not possible. There are 8010 URLs because this video is divided into 8010 fragments. You may want to filter the output with jq.

[EnginePod]

@yan12125 But there are full versions of the video without fragments available (which you can find in the JSON). If you just output the regular link with -g like: youtube-dl -g 'http://www.youtube.com/watch?v=gAAIw2NIOxc' then you'll have just 1 URL with the full video (without any fragments).

[EnginePod]

I also tried jq out just now, but things are processed really slowly as youtube-dl needs to output all the fragments before jq can process it.

Aren't the fragments only needed for ffmpeg to combine the different parts?

[yan12125]

What -j prints is the information that are passed from information extractors to downloaders. Currently DASH manifest parsing is implemented in extractors, so there are lots of fragment URLs. There was a proposal at #9109 that moves the parsing into downloaders. However, seems the author is not interested in it anymore. You may want to check whether the pull request is still working or not and open a new one.

you'll have just 1 URL with the full video

That URL points to DASH segments, which can't be used directly

[EnginePod]

But the fragments are created by youtube-dl, right?
I mean if you were to open the DASH file or the source of the video then you wouldn't find any fragments.

So shouldn't it just be a matter of passing an argument like --no-fragments and avoid creating the fragments if the argument is set?

[dstftw]

But the fragments are created by youtube-dl, right?

Wrong.

I mean if you were to open the DASH file or the source of the video then you wouldn't find any fragments.

Wrong. This video is DASH served in segments by YouTube.

[EnginePod]

@dstftw I actually checked the DASH earlier and it didn't contain a single fragment. The file was less than 50 KB while the output of youtube-dl was over 10 MB.

[yan12125]

There are indded segment URLs in the 50kB DASH manifest. Please check the MPEG DASH standard for how it works. What youtube-dl can do is moving those URLs to somewhere else, like what #9109 was attempting to do, but not removing/disabling/deleting them.

[EnginePod]

I just looked at the DASH again and there isn't a range inside any of the URLs.
Am I completely missing out on something here?

I'm talking about the URLs that look like this:
[URL]/0-1000 (start - end byte range)
[URL]/1000-2000
...

[yan12125]

Yes apparently you missed something

...
          <SegmentURL media="range/51888-210829"/>
          <SegmentURL media="range/210830-369818"/>
          <SegmentURL media="range/369819-528412"/>
          <SegmentURL media="range/528413-687261"/>
          <SegmentURL media="range/687262-846295"/>
          <SegmentURL media="range/846296-1004764"/>
          <SegmentURL media="range/1004765-1163700"/>
          <SegmentURL media="range/1163701-1322686"/>
          <SegmentURL media="range/1322687-1481388"/>
...