Enhancement request: graceful error handling and reporting on premature videos

[zmwangx]

I occasionally run into errors on newly uploaded YouTube videos. I ask YouTube to email me about new uploads of certain channels, and when the emails arrive, I will proceed to youtube-dl them. Sometimes the processing on YouTube's side may not be entirely done yet, and I end up getting dumps like the following one:

Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/bin/youtube-dl/__main__.py", line 19, in <module>
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 355, in main
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 345, in _real_main
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1121, in download
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 564, in extract_info
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 617, in process_ie_result
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 553, in extract_info
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 241, in extract
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 952, in _real_extract
TypeError: 'NoneType' object has no attribute '__getitem__'

Wait a couple minutes and the download finishes just fine.

Is it possible to employ more graceful error handling and (better yet) generate user-friendly error messages? (If that's not possible, maybe define some recognizable exceptions? Builtin exceptions are usually pretty hard to interpret.)

[phihag]

Thank you for the report. I believe this error to be fixed in youtube-dl 2014.12.10. Note that it's only the DASH manifest failing; you'll still get the video, only maybe in a lower quality.

[zmwangx]

Thanks. In fact, I wrote a custom script that lists all formats with -F, parses out the highest quality DASH streams, and then uses -f to download them. DASH manifest failing means -F would fail without --youtube-skip-dash-manifest.

[phihag]

Why did you write a custom script? What's wrong with -f bestaudio,bestvideo (or if you want to combine them, -f bestaudio+bestvideo/best)?

[zmwangx]

Oh, I didn't realize bestvideo+bestaudio include DASH streams. The thing is, I wrote the script over a year ago (I believe), when YouTube first dropped 1080p in non-DASH format. I only did some monkey patching later to keep up with format changes, but never carefully read the change log. I suppose the bestvideo, bestaudio feature was added later or extended to DASH streams later. (In fact, IIRC FFmpeg merging was also introduced later, which I somehow picked up — I was manually calling FFmpeg before that).

Anyway, there are still two things my script does that -f bestvideo+bestaudio doesn't support out of the box:

• My script prints the format codes and descriptions of the downloaded streams;
• My script checks for 60fps streams with lower resolution. Currently 60fps is only supported up to 1080p, so there are a lot of cases where the highest resolution is for instance 2160p@30, while the video is also available at 1080p@60, and I want to download both (as 60fps is in many cases much smoother).