youtube-dl is downloading altered video/mp4 files

[saundershayes]

#6251 seems to be an issue again.

When I run youtube-dl with --fixup never I still get files with the incorrect size. For example, I run:

youtube-dl --fixup never -f 134 -o video.mp4 https://www.youtube.com/embed/dTDYoJZyXuc
stat video.mp4

I see that the file size is 40827280, but the file size should be 40829316 according to YouTube video web requests and the YouTube manifest.

What can I do to get the unaltered video files from YouTube?

[dstftw]

-f 134 is served in segments. DASH manifest contains gaps according to metadata:

<Initialization sourceURL="range/0-708"/>
<SegmentURL media="range/2745-161548"/>

If you subtract two boundary values you'll get the difference you have.

[saundershayes]

But YouTube still expects the files to include those gaps, meaning that youtube-dl is not downloading the same files stored on the YouTube server. This bug will affect byte range requests (the offsets into the file) and file requests (which need accurate file sizes) creating issues for users that need the actual YouTube files.

Are there any options I can use to download the actual YouTube files even with the bytes that are not included in the mpd segment list?

[saundershayes]

It is incorrect not to download the byte range from 709-2744. Youtube-dl is not downloading those bytes leading to altered files as I mentioned above! That's a huge issue for those of us that rely on accurate downloads from youtube-dl.

[jaimeMF]

Note that, at least currently, the file has the correct size:

$ cat Sintel\ Silent\ Video-dTDYoJZyXuc.info.json | jq .filesize
32298722
$ stat Sintel\ Silent\ Video-dTDYoJZyXuc.mp4
  File: 'Sintel Silent Video-dTDYoJZyXuc.mp4'
  Size: 32298722    Blocks: 63088      IO Block: 4096   regular file
Device: 1000002h/16777218d  Inode: 4355654     Links: 1
Access: (0644/-rw-r--r--)  Uid: (  501/jaimemf93)   Gid: (   20/   staff)
Access: 2015-12-07 10:51:01.000000000 +0100
Modify: 2015-12-07 09:54:52.000000000 +0100
Change: 2015-12-07 10:51:01.000000000 +0100
 Birth: 2015-12-07 09:54:52.000000000 +0100

It is incorrect not to download the byte range from 709-2744

If Youtube doesn't list that range I don't think we should download it (it could be unavailable), if they listed it we would definitely download it.

[saundershayes]

Yes that is lucky! The YouTube MPD is now different from the last time I checked!

When the MPD is in this format as it is now:

"BaseURL yt:contentLength="32298722"BaseURL
SegmentBase indexRange="709-2744" indexRangeExact="true"
Initialization range="0-708"
SegmentBase"

youtube-dl handles the download correctly. However, when the segment list is given explicitly, as was happening before, youtube-dl was missing the indexRange="709-2744". With this new MPD, you can see that youtube-dl was missing important bytes from the YouTube video files and handling the earlier MPD format incorrectly by omitting the index.

It seems that the YouTube MPDs are not always complete in terms of information given. If there is a gap in the YouTube MPD segment list, the file should still be downloaded with the correct file size and byte range offsets!