Youtube-dl latest fails (`insecure string pickle`) when downloading from Youtube

[jjarava]

Hi!

I'm trying to get a video from Youtube with the latest version (2016.02.13) and the download fails with the following message:

$ youtube-dl https://www.youtube.com/watch?v=qlk4JDOiivM
[youtube] qlk4JDOiivM: Downloading webpage
[youtube] qlk4JDOiivM: Downloading video info webpage
[youtube] qlk4JDOiivM: Extracting video information
[youtube] qlk4JDOiivM: Downloading MPD manifest
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/bin/youtube-dl/__main__.py", line 19, in <module>
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 411, in main
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 401, in _real_main
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1692, in download
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 677, in extract_info
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 722, in process_ie_result
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1345, in process_video_result
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 35, in __init__
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 92, in _determine_executables
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 92, in <genexpr>
  File "/usr/local/bin/youtube-dl/youtube_dl/utils.py", line 1546, in get_exe_version
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1334, in _execute_child
    child_exception = pickle.loads(data)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1382, in loads
    return Unpickler(file).load()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 858, in load
    dispatch[key](self)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 966, in load_string
    raise ValueError, "insecure string pickle"

Running with verbose flags:

 $ youtube-dl -v https://www.youtube.com/watch?v=qlk4JDOiivM
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'https://www.youtube.com/watch?v=qlk4JDOiivM']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.02.13
[debug] Python version 2.7.10 - Darwin-15.3.0-x86_64-i386-64bit
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/bin/youtube-dl/__main__.py", line 19, in <module>
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 411, in main
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 378, in _real_main
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 346, in __init__
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1944, in print_debug_header
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 51, in get_versions
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 35, in __init__
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 92, in _determine_executables
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 92, in <genexpr>
  File "/usr/local/bin/youtube-dl/youtube_dl/utils.py", line 1546, in get_exe_version
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1334, in _execute_child
    child_exception = pickle.loads(data)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1382, in loads
    return Unpickler(file).load()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 858, in load
    dispatch[key](self)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 966, in load_string
    raise ValueError, "insecure string pickle"
ValueError: insecure string pickle

[yan12125]

Duplicate of #6840. A workaround is using latest Python 3.

[jjarava]

Had a look at #6840. So if I've understood the situation, there's nothing that can be done to fix it with Python v2.7?

What I find odd is that I believe I haven't changed anything (not the python version I'm pretty sure) lately and youtube-dl was working well.

Not only that, but I've checked in another Mac that is also running El Capitan with an older YouTube-DL version and the issue doesn't happen!!

Mac-mini:~ javier$ youtube-dl --version
2016.02.05.1
Mac-mini:~ javier$ python --version
Python 2.7.10
Mac-mini:~ javier$ which -a python
/usr/bin/python
Mac-mini:Series javier$ youtube-dl https://www.youtube.com/watch?v=qlk4JDOiivM[youtube] qlk4JDOiivM: Downloading webpage
[youtube] qlk4JDOiivM: Downloading video info webpage
[youtube] qlk4JDOiivM: Extracting video information
[youtube] qlk4JDOiivM: Downloading DASH manifest
[download] Destination: On Cyber - the grugq - Power of Community 2015-qlk4JDOiivM.f247.webm
[download] 100% of 270.33MiB in 00:12[download] Destination: On Cyber - the grugq - Power of Community 2015-qlk4JDOiivM.f251.webm
[download] 100% of 58.46MiB in 00:02[ffmpeg] Merging formats into "On Cyber - the grugq - Power of Community 2015-qlk4JDOiivM.webm"
Deleting original file On Cyber - the grugq - Power of Community 2015-qlk4JDOiivM.f247.webm (pass -k to keep)
Deleting original file On Cyber - the grugq - Power of Community 2015-qlk4JDOiivM.f251.webm (pass -k to keep)
Mac-mini:Series javier$ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.11.2
BuildVersion:   15C50

Is there a way to go back to the older YouTube-DL version to test?

[Hrxn]

https://yt-dl.org/downloads/

[yan12125]

Apply this commit to subprocess.py. Please don't do anything unless you have fully understand what you're doing.

[iphoting]

In any case, just to put it out there, rolling back to version 2016.02.10 works for me.

[jjarava]

I've tried rolling back the software but 2016.02.10, 2016.02.09.1and 2016.02.05.1 all fail for me.

I don't understand why it works on one machine and fails on the other!
I've updated youtube-dl in the "working" machine to the latest version and it still works!!

On both machines, the python version is supposedly the same:

Python 2.7.10 (default, Oct 23 2015, 18:05:06)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin

Is there a way to track down what is the change that "breaks" the program? What is the reason we get this exception (and that wasn't there previously?)

(also, what's the way to "reopen" the issue?)

[yan12125]

I guess it's caused by Mac OS X kernel bugs. What we can do is using some workarounds. Unfornately, all known workarounds for youtube-dl require changing Python itself - either use a newer Python or patch subprocess.py. These may not be clean solutions, but they works.

what's the way to "reopen" the issue?

If an issue is closed by a collaborator (like this one), only collaborators can reopen it.