Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DASH video download seems corrupted #15641

Closed
vincentwoo opened this issue Feb 18, 2018 · 7 comments
Closed

DASH video download seems corrupted #15641

vincentwoo opened this issue Feb 18, 2018 · 7 comments
Labels

Comments

@vincentwoo
Copy link

@vincentwoo vincentwoo commented Feb 18, 2018

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2018.02.11. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

  • I've verified and I assure that I'm running youtube-dl 2018.02.11

Before submitting an issue make sure you have:

  • At least skimmed through the README, most notably the FAQ and BUGS sections
  • Searched the bugtracker for similar issues including closed ones
  • Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

  • Bug report (encountered problems with youtube-dl)
  • Site support request (request for adding support for a new site)
  • Feature request (request for a new functionality)
  • Question
  • Other

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

Add the -v flag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '-F', 'https://www.youtube.com/watch?v=htVDXD368eU']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2018.02.11
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.16299
[debug] exe versions: ffmpeg N-90069-gdd8351b118, ffprobe N-90069-gdd8351b118
[debug] Proxy map: {}
...

Description of your issue, suggested solution and other information

Hi, thanks so much for the work on this great tool.

I was trying to download a video that YouTube had just archived from my live stream. Before YT reencodes these videos, they are available for a time as DASH videos instead (vp9 encoding takes much longer). I downloaded this video (which is no longer available in DASH), but merging AV with ffmpeg seemed to fail. Upon telling youtube-dl to just keep the separate tracks, I took a look at the video stream with ffmpeg:

C:\Users\vwoo\Videos>ffmpeg.exe -i "PUBG with Hubert-htVDXD368eU.f304.mp4"
ffmpeg version N-90069-gdd8351b118 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 7.2.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libmfx --enable-amf --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth
  libavutil      56.  7.101 / 56.  7.101
  libavcodec     58. 11.101 / 58. 11.101
  libavformat    58.  9.100 / 58.  9.100
  libavdevice    58.  1.100 / 58.  1.100
  libavfilter     7. 12.100 /  7. 12.100
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001ca0bf29ac0] Found duplicated MOOV Atom. Skipped it
    Last message repeated 1272 times
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'PUBG with Hubert-htVDXD368eU.f304.mp4':
  Metadata:
    creation_time   : 2018-02-18T07:33:56.000000Z
    major_brand     : dash
    minor_version   : 0
    compatible_brands: iso6avc1mp41
  Duration: 00:00:05.00, start: 0.000000, bitrate: 19962602 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 2560x1440 [SAR 1:1 DAR 16:9], 19946401 kb/s, 60 fps, 60 tbr, 90k tbn, 120 tbc (default)
    Metadata:
      creation_time   : 2018-02-18T07:33:56.000000Z
      handler_name    : VideoHandler

Here you can see that ffmpeg thinks the video is only 5 seconds long (it's actually over an hour long), and that it sees a ton of duplicate MOOV atoms. I'm not 100% positive this is a problem with youtube-dl, because I can play this video track in MPC-HC, but I can't open it in my video editing software (Vegas 15) nor manipulate it correctly with ffmpeg. Any insight would be very much appreciated!

@dstftw
Copy link
Collaborator

@dstftw dstftw commented Feb 18, 2018

Carefully read new issue template and provide all requested information.

@dstftw dstftw closed this Feb 18, 2018
@dstftw dstftw added the incomplete label Feb 18, 2018
@vincentwoo
Copy link
Author

@vincentwoo vincentwoo commented Feb 18, 2018

@dstftw I believe I have followed the template, can you tell me what is missing?

@dstftw
Copy link
Collaborator

@dstftw dstftw commented Feb 18, 2018

Complete verbose log of youtube-dl command used for downloading.

@vincentwoo
Copy link
Author

@vincentwoo vincentwoo commented Feb 18, 2018

As explained, part of the problem is that DASH streams are ephemeral on YouTube, so I can't show you a verbose log of the DASH dump anymore. Should I try to recreate a fresh stream to cause the issue again?

@dstftw
Copy link
Collaborator

@dstftw dstftw commented Feb 18, 2018

That's not true. All video-only and audio-only videos are basically DASH, some are true segmented DASH. Here is the video still available even as segmented DASH:

> py -3.6 .\youtube_dl\__main__.py 9xYLXHx3g80
[youtube] 9xYLXHx3g80: Downloading webpage
[youtube] 9xYLXHx3g80: Downloading video info webpage
[youtube] 9xYLXHx3g80: Extracting video information
[youtube] 9xYLXHx3g80: Downloading MPD manifest
[youtube] 9xYLXHx3g80: Downloading MPD manifest
[dashsegments] Total fragments: 337
[download] Destination: Show and tell for ASMR-9xYLXHx3g80.f137.mp4
[download]   0.9% of ~352.23MiB at  2.07MiB/s ETA 09:13
ERROR: Interrupted by user
@vincentwoo
Copy link
Author

@vincentwoo vincentwoo commented Mar 6, 2018

@dstftw Hi, I've encountered this issue on another stream and have collected verbose logging:

C:\Users\vwoo\Videos>youtube-dl.exe -v -F https://www.youtube.com/watch?v=UiP9HyDFvvQ
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '-F', 'https://www.youtube.com/watch?v=UiP9HyDFvvQ']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2018.02.11
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.16299
[debug] exe versions: ffmpeg N-90069-gdd8351b118, ffprobe N-90069-gdd8351b118
[debug] Proxy map: {}
[youtube] UiP9HyDFvvQ: Downloading webpage
[youtube] UiP9HyDFvvQ: Downloading video info webpage
[youtube] UiP9HyDFvvQ: Extracting video information
[youtube] UiP9HyDFvvQ: Downloading m3u8 information
[youtube] UiP9HyDFvvQ: Downloading MPD manifest
[info] Available formats for UiP9HyDFvvQ:
format code  extension  resolution note
139          m4a        audio only DASH audio   64k , m4a_dash container, mp4a.40.5@ 48k (24000Hz)
140          m4a        audio only DASH audio  144k , m4a_dash container, mp4a.40.2@128k (48000Hz)
160          mp4        256x144    DASH video  124k , mp4_dash container, avc1.42c00b, 30fps, video only
133          mp4        426x240    DASH video  258k , mp4_dash container, avc1.4d4015, 30fps, video only
134          mp4        640x360    DASH video  646k , mp4_dash container, avc1.4d401e, 30fps, video only
135          mp4        854x480    DASH video 1171k , mp4_dash container, avc1.4d401f, 30fps, video only
136          mp4        1280x720   DASH video 2326k , mp4_dash container, avc1.4d401f, 30fps, video only
298          mp4        1280x720   DASH video 3481k , mp4_dash container, avc1.4d4020, 60fps, video only
137          mp4        1920x1080  DASH video 4347k , mp4_dash container, avc1.640028, 30fps, video only
299          mp4        1920x1080  DASH video 5791k , mp4_dash container, avc1.64002a, 60fps, video only
264          mp4        2560x1440  DASH video 10411k , mp4_dash container, avc1.640032, 30fps, video only
304          mp4        2560x1440  DASH video 15608k , mp4_dash container, avc1.640033, 60fps, video only
91           mp4        256x144    HLS  197k , avc1.42c00b, 30.0fps, mp4a.40.5@ 48k
92           mp4        426x240    HLS  338k , avc1.4d4015, 30.0fps, mp4a.40.5@ 48k
93           mp4        640x360    HLS  829k , avc1.4d401e, 30.0fps, mp4a.40.2@128k
94           mp4        854x480    HLS 1380k , avc1.4d401f, 30.0fps, mp4a.40.2@128k
300          mp4        1280x720   3806k , avc1.4d4020, 60.0fps, mp4a.40.2
301          mp4        1920x1080  6231k , avc1.64002a, 60.0fps, mp4a.40.2 (best)

C:\Users\vwoo\Videos>youtube-dl.exe -v -f bestvideo+bestaudio -k https://www.youtube.com/watch?v=UiP9HyDFvvQ
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '-f', 'bestvideo+bestaudio', '-k', 'https://www.youtube.com/watch?v=UiP9HyDFvvQ']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2018.02.11
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.16299
[debug] exe versions: ffmpeg N-90069-gdd8351b118, ffprobe N-90069-gdd8351b118
[debug] Proxy map: {}
[youtube] UiP9HyDFvvQ: Downloading webpage
[youtube] UiP9HyDFvvQ: Downloading video info webpage
[youtube] UiP9HyDFvvQ: Extracting video information
[youtube] UiP9HyDFvvQ: Downloading m3u8 information
[youtube] UiP9HyDFvvQ: Downloading MPD manifest
[debug] Invoking downloader on 'https://r3---sn-nvopjoxu-25ve.googlevideo.com/videoplayback/id/UiP9HyDFvvQ.0/itag/304/source/yt_live_broadcast/requiressl/yes/force_finished/1/playlist_type/DVR/gcr/us/ei/JVOeWvWEM4uj_APk1JroAQ/ratebypass/yes/cmbypass/yes/mime/video%2Fmp4/live/1/gir/yes/noclen/1/dur/5.000/ip/157.131.198.82/ipbits/0/expire/1520347013/sparams/cmbypass,dur,ei,expire,force_finished,gcr,gir,id,initcwndbps,ip,ipbits,itag,live,mime,mm,mn,ms,mv,noclen,pl,playlist_type,ratebypass,requiressl,source/signature/1C2ECFC449336D06582977B78F4602CF08662DDC.7082721480A577666B999109317553A25A2057DA/key/cms1/initcwndbps/7450/mm/32/mn/sn-nvopjoxu-25ve/ms/lv/mt/1520325268/mv/m/pl/17/'
[dashsegments] Total fragments: 864
[download] Destination: Overwatch with Steven and Anthony-UiP9HyDFvvQ.f304.mp4
[download] 100% of 7.47GiB in 15:09
[debug] Invoking downloader on 'https://r3---sn-nvopjoxu-25ve.googlevideo.com/videoplayback/id/UiP9HyDFvvQ.0/itag/140/source/yt_live_broadcast/requiressl/yes/force_finished/1/playlist_type/DVR/gcr/us/ei/JVOeWvWEM4uj_APk1JroAQ/ratebypass/yes/cmbypass/yes/mime/audio%2Fmp4/live/1/gir/yes/noclen/1/dur/5.000/ip/157.131.198.82/ipbits/0/expire/1520347013/sparams/cmbypass,dur,ei,expire,force_finished,gcr,gir,id,initcwndbps,ip,ipbits,itag,live,mime,mm,mn,ms,mv,noclen,pl,playlist_type,ratebypass,requiressl,source/signature/6252EECEFB0917CC60C6243073353E7E69543EBA.0B9DC114787D7BBCF59AB14EACCB2CE5090B3222/key/cms1/initcwndbps/7450/mm/32/mn/sn-nvopjoxu-25ve/ms/lv/mt/1520325268/mv/m/pl/17/'
[dashsegments] Total fragments: 864
[download] Destination: Overwatch with Steven and Anthony-UiP9HyDFvvQ.f140.m4a
[download] 100% of 67.40MiB in 01:23
[ffmpeg] Merging formats into "Overwatch with Steven and Anthony-UiP9HyDFvvQ.mp4"
[debug] ffmpeg command line: ffmpeg -y -i "file:Overwatch with Steven and Anthony-UiP9HyDFvvQ.f304.mp4" -i "file:Overwatch with Steven and Anthony-UiP9HyDFvvQ.f140.m4a" -c copy -map "0:v:0" -map "1:a:0" "file:Overwatch with Steven and Anthony-UiP9HyDFvvQ.temp.mp4"

Trying to examine the resulting file with ffprobe shows odd results, and the file doesn't render in most video software:

C:\Users\vwoo\Videos>ffprobe.exe -v verbose "Overwatch with Steven and Anthony-UiP9HyDFvvQ.mp4"
ffprobe version N-90069-gdd8351b118 Copyright (c) 2007-2018 the FFmpeg developers
  built with gcc 7.2.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libmfx --enable-amf --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth
  libavutil      56.  7.101 / 56.  7.101
  libavcodec     58. 11.101 / 58. 11.101
  libavformat    58.  9.100 / 58.  9.100
  libavdevice    58.  1.100 / 58.  1.100
  libavfilter     7. 12.100 /  7. 12.100
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
[h264 @ 000001961a15fa00] Reinit context to 2560x1440, pix_fmt: yuv420p
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Overwatch with Steven and Anthony-UiP9HyDFvvQ.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.9.100
  Duration: 00:00:09.23, start: 0.000000, bitrate: 7007199 kb/s
    Stream #0:0(und): Video: h264 (High), 1 reference frame (avc1 / 0x31637661), yuv420p(tv, bt709, left), 2560x1440 [SAR 1:1 DAR 16:9], 8153286 kb/s, 32977 fps, 60 tbr, 90k tbn, 120 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 58834 kb/s (default)
    Metadata:
      handler_name    : SoundHandler

Note the duration as 9 minutes. The actual duration of the video is something like 1 hour.

The individual components are also difficult to manipulate as well. Trying to reencode the f304.mp4 video source yields the afforementioned MOOV atom problems.

Does this help?

@bartoszek
Copy link

@bartoszek bartoszek commented Feb 17, 2019

Have the same issue, any progress?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.