Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

insecure string pickle #6840

Closed
shelbyKiraM opened this issue Sep 12, 2015 · 50 comments
Closed

insecure string pickle #6840

shelbyKiraM opened this issue Sep 12, 2015 · 50 comments

Comments

@shelbyKiraM
Copy link

Got this... Not sure about issue:

$ youtube-dl "https://www.youtube.com/watch?v=-AshJ-75sdA"     1 ↵
[youtube] -AshJ-75sdA: Downloading webpage
[youtube] -AshJ-75sdA: Downloading video info webpage
[youtube] -AshJ-75sdA: Extracting video information
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/bin/youtube-dl/__main__.py", line 19, in <module>
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 410, in main
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 400, in _real_main
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1658, in download
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 671, in extract_info
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 717, in process_ie_result
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1311, in process_video_result
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 35, in __init__
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 91, in _determine_executables
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 91, in <genexpr>
  File "/usr/local/bin/youtube-dl/youtube_dl/utils.py", line 1471, in get_exe_version
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1334, in _execute_child
    child_exception = pickle.loads(data)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1382, in loads
    return Unpickler(file).load()
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 858, in load
    dispatch[key](self)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 966, in load_string
    raise ValueError, "insecure string pickle"
ValueError: insecure string pickle
@jaimeMF
Copy link
Collaborator

jaimeMF commented Sep 12, 2015

Post the full output you get witht he --verbose option.

@shelbyKiraM
Copy link
Author

$ youtube-dl --verbose "https://www.youtube.com/watch?v=-AshJ-75sdA"
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'--verbose', u'https://www.youtube.com/watch?v=-AshJ-75sdA']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2015.09.09
[debug] Python version 2.7.10 - Darwin-15.0.0-x86_64-i386-64bit
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/bin/youtube-dl/__main__.py", line 19, in <module>
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 410, in main
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 377, in _real_main
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 340, in __init__
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1904, in print_debug_header
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 51, in get_versions
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 35, in __init__
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 91, in _determine_executables
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 91, in <genexpr>
  File "/usr/local/bin/youtube-dl/youtube_dl/utils.py", line 1471, in get_exe_version
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1334, in _execute_child
    child_exception = pickle.loads(data)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1382, in loads
    return Unpickler(file).load()
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 858, in load
    dispatch[key](self)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 966, in load_string
    raise ValueError, "insecure string pickle"
ValueError: insecure string pickle

@jaimeMF
Copy link
Collaborator

jaimeMF commented Sep 13, 2015

That's weird, can you post the output of ffmpeg -version, ffprobe -version, avconv -version and avprobe -version?. Does it also happens if you use the system python: /usr/bin/python $(which youtube-dl)?
Do you know if it only happens with a specific youtube-dl version or has something changed in your system?

@shelbyKiraM
Copy link
Author

Uh. Mac OS X 10.11 GM now.
On Sun, Sep 13, 2015 at 10:33 Jaime Marquínez Ferrándiz <
notifications@github.com> wrote:

That's weird, can you post the output of ffmpeg -version, ffprobe -version,
avconv -version and avprobe -version?. Does it also happens if you use
the system python: /usr/bin/python $(which youtube-dl)?
Do you know if it only happens with a specific youtube-dl version or has
something changed in your system?


Reply to this email directly or view it on GitHub
#6840 (comment).

@shelbyKiraM
Copy link
Author

ffmpeg -version
ffmpeg version 2.7.2 Copyright (c) 2000-2015 the FFmpeg developers
built with Apple LLVM version 7.0.0 (clang-700.0.53)
configuration: --prefix=/usr/local/Cellar/ffmpeg/2.7.2_1 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-opencl --enable-libx264 --enable-libmp3lame --enable-libvo-aacenc --enable-libxvid --enable-vda
libavutil      54. 27.100 / 54. 27.100
libavcodec     56. 41.100 / 56. 41.100
libavformat    56. 36.100 / 56. 36.100
libavdevice    56.  4.100 / 56.  4.100
libavfilter     5. 16.101 /  5. 16.101
libavresample   2.  1.  0 /  2.  1.  0
libswscale      3.  1.101 /  3.  1.101
libswresample   1.  2.100 /  1.  2.100
libpostproc    53.  3.100 / 53.  3.100

ffprobe -version
ffprobe version 2.7.2 Copyright (c) 2007-2015 the FFmpeg developers
built with Apple LLVM version 7.0.0 (clang-700.0.53)
configuration: --prefix=/usr/local/Cellar/ffmpeg/2.7.2_1 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-opencl --enable-libx264 --enable-libmp3lame --enable-libvo-aacenc --enable-libxvid --enable-vda
libavutil      54. 27.100 / 54. 27.100
libavcodec     56. 41.100 / 56. 41.100
libavformat    56. 36.100 / 56. 36.100
libavdevice    56.  4.100 / 56.  4.100
libavfilter     5. 16.101 /  5. 16.101
libavresample   2.  1.  0 /  2.  1.  0
libswscale      3.  1.101 /  3.  1.101
libswresample   1.  2.100 /  1.  2.100
libpostproc    53.  3.100 / 53.  3.100

avconv -version
zsh: command not found: avconv

avprobe -version
zsh: command not found: avprobe

@jaimeMF
Copy link
Collaborator

jaimeMF commented Sep 16, 2015

I'm not sure, but if you have upgraded to Mac OS X 10.11 you may need to reinstall/rebuild some programs with homebrew.

As I said, could you try with the builtin python: /usr/bin/python $(which youtube-dl) --verbose URL?

Could yo try to edit /usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py at line 1334 to look like this:

            if data != "":
                try:
                    _eintr_retry_call(os.waitpid, self.pid, 0)
                except OSError as e:
                    if e.errno != errno.ECHILD:
                        raise
                # The following line is new             
                print(data)
                child_exception = pickle.loads(data)
                raise child_exception

and run youtube-dl --version?

@shelbyKiraM
Copy link
Author

$ /usr/bin/python $(which youtube-dl) --verbose "https://www.youtube.com/watch?v=-AshJ-75sdA"
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'--verbose', u'https://www.youtube.com/watch?v=-AshJ-75sdA']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2015.09.09
[debug] Python version 2.7.10 - Darwin-15.0.0-x86_64-i386-64bit
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/bin/youtube-dl/__main__.py", line 19, in <module>
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 410, in main
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 377, in _real_main
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 340, in __init__
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1904, in print_debug_header
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 51, in get_versions
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 35, in __init__
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 91, in _determine_executables
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 91, in <genexpr>
  File "/usr/local/bin/youtube-dl/youtube_dl/utils.py", line 1471, in get_exe_version
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1334, in _execute_child
    child_exception = pickle.loads(data)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1382, in loads
    return Unpickler(file).load()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 858, in load
    dispatch[key](self)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 966, in load_string
    raise ValueError, "insecure string pickle"
ValueError: insecure string pickle

After modifications:

$ youtube-dl --version
2015.09.09
$ youtube-dl "https://www.youtube.com/watch?v=-AshJ-75sdA"
[youtube] -AshJ-75sdA: Downloading webpage
[youtube] -AshJ-75sdA: Downloading video info webpage
[youtube] -AshJ-75sdA: Extracting video information
[youtube] -AshJ-75sdA: Downloading js player new-en_US-vflSnomqH
[youtube] -AshJ-75sdA: Downloading DASH manifest
cexceptions
OSError
p0
(I2
S'No such file or directory'
p1
tp2
Rp3
(dp4
S'child_traceback'
p5
S'Traceback (most recent call last):\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1290, in _execute_child\n    os.execvp(executable, args)\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 346, in execvp\n    _execvpe(file, args)\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/bin/youtube-dl/__main__.py", line 19, in <module>
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 410, in main
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 400, in _real_main
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1658, in download
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 671, in extract_info
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 717, in process_ie_result
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1311, in process_video_result
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 35, in __init__
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 91, in _determine_executables
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 91, in <genexpr>
  File "/usr/local/bin/youtube-dl/youtube_dl/utils.py", line 1471, in get_exe_version
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1336, in _execute_child
    child_exception = pickle.loads(data)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1382, in loads
    return Unpickler(file).load()
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 858, in load
    dispatch[key](self)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 966, in load_string
    raise ValueError, "insecure string pickle"
ValueError: insecure string pickle

@jaimeMF
Copy link
Collaborator

jaimeMF commented Sep 17, 2015

Sorry, instead of print(data) could you use print(repr(data)) I want to make sure I try with the correct data.

Could you install python3 with homebrew and run python3 (which youtube-dl) --verbose URL to see if it also fails?

@shelbyKiraM
Copy link
Author

After edit: "Sorry, instead of print(data) could you use print(repr(data)) I want to make sure I try with the correct data."

$ youtube-dl "https://www.youtube.com/watch?v=-AshJ-75sdA"
[youtube] -AshJ-75sdA: Downloading webpage
[youtube] -AshJ-75sdA: Downloading video info webpage
[youtube] -AshJ-75sdA: Extracting video information
[youtube] -AshJ-75sdA: Downloading DASH manifest
'cexceptions\nOSError\np0\n(I2\nS\'No such file or directory\'\np1\ntp2\nRp3\n(dp4\nS\'child_traceback\'\np5\nS\'Traceback (most recent call last):\\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1290, in _execute_child\\n    os.execvp(executable, args)\\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 346, in execvp\\n    _execvpe(file, args)\\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks'
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/bin/youtube-dl/__main__.py", line 19, in <module>
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 410, in main
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 400, in _real_main
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1658, in download
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 671, in extract_info
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 717, in process_ie_result
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1311, in process_video_result
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 35, in __init__
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 91, in _determine_executables
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 91, in <genexpr>
  File "/usr/local/bin/youtube-dl/youtube_dl/utils.py", line 1471, in get_exe_version
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1336, in _execute_child
    child_exception = pickle.loads(data)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1382, in loads
    return Unpickler(file).load()
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 858, in load
    dispatch[key](self)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 966, in load_string
    raise ValueError, "insecure string pickle"
ValueError: insecure string pickle

Already installed, and...

$ python3 $(which youtube-dl) --verbose "https://www.youtube.com/watch?v=-AshJ-75sdA"
/usr/local/opt/python3/bin/python3.5: can't find '__main__' module in '/usr/local/bin/youtube-dl'

Cleaned that output for ya.

'cexceptions
OSError
p0
(I2
'No such file or directory'
p1
tp2
Rp3
(dp4
'child_traceback'
p5
'Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1290, in _execute_child
    os.execvp(executable, args)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 346, in execvp
    _execvpe(file, args)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks'

@jaimeMF
Copy link
Collaborator

jaimeMF commented Sep 17, 2015

I don't know why you get that output when using python3, it should works the same as python2.

There's something wrong in the data variable, it should look like this:

'cexceptions\nOSError\np0\n(I2\nS\'No such file or directory\'\np1\ntp2\nRp3\n(dp4\nS\'child_traceback\'\np5\nS\'Traceback (most recent call last):\\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1290, in _execute_child\\n    os.execvp(executable, args)\\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 346, in execvp\\n    _execvpe(file, args)\\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 382, in _execvpe\\n    func(fullname, *argrest)\\nOSError: [Errno 2] No such file or directory\\n\'\np6\nsb.'

in your case it cuts the last part (Python.framework/Versions/2.7/lib/python2.7/os.py", line 382, in _execvpe\\n func(fullname, *argrest)\\nOSError: [Errno 2] No such file or directory\\n\'\np6\nsb.), it could be a python bug. Could you run the following script with python script_name.py?

import subprocess

for exe in ['ffmpeg', 'avconv']:
    print(exe)
    out, _ = subprocess.Popen([exe, '-version'], stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()
    print(out)

@shelbyKiraM
Copy link
Author

ffmpeg
ffmpeg version 2.8 Copyright (c) 2000-2015 the FFmpeg developers
built with Apple LLVM version 7.0.0 (clang-700.0.53)
configuration: --prefix=/usr/local/Cellar/ffmpeg/2.8 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-opencl --enable-libx264 --enable-libmp3lame --enable-libvo-aacenc --enable-libxvid --enable-vda
libavutil      54. 31.100 / 54. 31.100
libavcodec     56. 60.100 / 56. 60.100
libavformat    56. 40.101 / 56. 40.101
libavdevice    56.  4.100 / 56.  4.100
libavfilter     5. 40.101 /  5. 40.101
libavresample   2.  1.  0 /  2.  1.  0
libswscale      3.  1.101 /  3.  1.101
libswresample   1.  2.101 /  1.  2.101
libpostproc    53.  3.100 / 53.  3.100

avconv
'cexceptions\nOSError\np0\n(I2\nS\'No such file or directory\'\np1\ntp2\nRp3\n(dp4\nS\'child_traceback\'\np5\nS\'Traceback (most recent call last):\\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1290, in _execute_child\\n    os.execvp(executable, args)\\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 346, in execvp\\n    _execvpe(file, args)\\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 382, in _execvpe\\n    func(fullname, *argrest)\\nOSError: [Errno 2] No such file or directory\\n\'\np6\nsb.'
Traceback (most recent call last):
  File "<stdin>", line 5, in <module>
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1337, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

@shelbyKiraM
Copy link
Author

With python3

ffmpeg
b'ffmpeg version 2.8 Copyright (c) 2000-2015 the FFmpeg developers\nbuilt with Apple LLVM version 7.0.0 (clang-700.0.53)\nconfiguration: --prefix=/usr/local/Cellar/ffmpeg/2.8 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-opencl --enable-libx264 --enable-libmp3lame --enable-libvo-aacenc --enable-libxvid --enable-vda\nlibavutil      54. 31.100 / 54. 31.100\nlibavcodec     56. 60.100 / 56. 60.100\nlibavformat    56. 40.101 / 56. 40.101\nlibavdevice    56.  4.100 / 56.  4.100\nlibavfilter     5. 40.101 /  5. 40.101\nlibavresample   2.  1.  0 /  2.  1.  0\nlibswscale      3.  1.101 /  3.  1.101\nlibswresample   1.  2.101 /  1.  2.101\nlibpostproc    53.  3.100 / 53.  3.100\n'
avconv
Traceback (most recent call last):
  File "<stdin>", line 5, in <module>
  File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/subprocess.py", line 950, in __init__
    restore_signals, start_new_session)
  File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/subprocess.py", line 1540, in _execute_child
    raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: 'avconv'

@jaimeMF
Copy link
Collaborator

jaimeMF commented Sep 18, 2015

First of all: installing the libav and rtmpdump packages with homebrew will probably avoid this error. (if you install them, make sure you use brew unlink libav before helping us debugging this issue)

Try with this script:

import subprocess

for exe in ['avprobe', 'avconv', 'ffmpeg', 'ffprobe']:
    print(exe)
    out, _ = subprocess.Popen([exe, '-version'], stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()
    print(out)

That will test the executables in same order as youtube-dl. If that works, then there is some weird interaction with the rest of youtube-dl's code.

@shelbyKiraM
Copy link
Author

20:01:39 user@Daedalus.local local master brew install libav
20:02:08 user@Daedalus.local local master brew install rtmpdump
20:02:15 user@Daedalus.local local master brew unlink libav
20:02:46 user@Daedalus.local local master echo "import subprocess

for exe in ['avprobe', 'avconv', 'ffmpeg', 'ffprobe']:
    print(exe)
    out, _ = subprocess.Popen([exe, '-version'], stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()
    print(out)"|python
avprobe
'cexceptions\nOSError\np0\n(I2\nS\'No such file or directory\'\np1\ntp2\nRp3\n(dp4\nS\'child_traceback\'\np5\nS\'Traceback (most recent call last):\\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1290, in _execute_child\\n    os.execvp(executable, args)\\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 346, in execvp\\n    _execvpe(file, args)\\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 382, in _execvpe\\n    func(fullname, *argrest)\\nOSError: [Errno 2] No such file or directory\\n\'\np6\nsb.'
Traceback (most recent call last):
  File "<stdin>", line 5, in <module>
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1337, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

And, after install, looks like it's getting it, but it isn't.

youtube-dl "https://www.youtube.com/watch?v=3p8X6rE_Irw"
[youtube] 3p8X6rE_Irw: Downloading webpage
[youtube] 3p8X6rE_Irw: Downloading video info webpage
[youtube] 3p8X6rE_Irw: Extracting video information
[youtube] 3p8X6rE_Irw: Downloading DASH manifest
[youtube] 3p8X6rE_Irw: Downloading DASH manifest
'cexceptions\nOSError\np0\n(I2\nS\'No such file or directory\'\np1\ntp2\nRp3\n(dp4\nS\'child_traceback\'\np5\nS\'Traceback (most recent call last):\\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1290, in _execute_child\\n    os.execvp(executable, args)\\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 346, in execvp\\n    _execvpe(file, args)\\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 382, in _execvpe\\n    func(fullname, *argrest)\\nOSError: [Errno 2] No such file or directory\\n\'\np6\nsb.'
'cexceptions\nOSError\np0\n(I2\nS\'No such file or directory\'\np1\ntp2\nRp3\n(dp4\nS\'child_traceback\'\np5\nS\'Traceback (most recent call last):\\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1290, in _execute_child\\n    os.execvp(executable, args)\\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 346, in execvp\\n    _execvpe(file, args)\\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 382, in _execvpe\\n    func(fullname, *argrest)\\nOSError: [Errno 2] No such file or directory\\n\'\np6\nsb.'
'cexceptions\nOSError\np0\n(I2\nS\'No such file or directory\'\np1\ntp2\nRp3\n(dp4\nS\'child_traceback\'\np5\nS\'Traceback (most recent call last):\\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1290, in _execute_child\\n    os.execvp(executable, args)\\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 346, in execvp\\n    _execvpe(file, args)\\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 382, in _execvpe\\n    func(fullname, *argrest)\\nOSError: [Errno 2] No such file or directory\\n\'\np6\nsb.'
'cexceptions\nOSError\np0\n(I2\nS\'No such file or directory\'\np1\ntp2\nRp3\n(dp4\nS\'child_traceback\'\np5\nS\'Traceback (most recent call last):\\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1290, in _execute_child\\n    os.execvp(executable, args)\\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 346, in execvp\\n    _execvpe(file, args)\\n  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 382, in _execvpe\\n    func(fullname, *argrest)\\nOSError: [Errno 2] No such file or directory\\n\'\np6\nsb.'
[download] Destination: 'I have two scientific reasons why I am not an atheist' - Atheist Experience show #737-3p8X6rE_Irw.f135.mp4
[download] 100% of 46.37MiB in 00:09
[download] Destination: 'I have two scientific reasons why I am not an atheist' - Atheist Experience show #737-3p8X6rE_Irw.f141.m4a
[download] 100% of 22.24MiB in 00:07
[ffmpeg] Merging formats into "'I have two scientific reasons why I am not an atheist' - Atheist Experience show #737-3p8X6rE_Irw.mp4"
Deleting original file 'I have two scientific reasons why I am not an atheist' - Atheist Experience show #737-3p8X6rE_Irw.f135.mp4 (pass -k to keep)
Deleting original file 'I have two scientific reasons why I am not an atheist' - Atheist Experience show #737-3p8X6rE_Irw.f141.m4a (pass -k to keep)

@jaimeMF
Copy link
Collaborator

jaimeMF commented Sep 19, 2015

The second output is the expected one if you don't have libav installed (or if you run brew unlink libav), is everything working now?

@shelbyKiraM
Copy link
Author

Pretty sure you had me unlink libav… relinked and it works.

$ brew link libav
Linking /usr/local/Cellar/libav/11.4_1... 4 symlinks created
$ youtube-dl "https://www.youtube.com/watch?v=3p8X6rE_Irw"
[youtube] 3p8X6rE_Irw: Downloading webpage
[youtube] 3p8X6rE_Irw: Downloading video info webpage
[youtube] 3p8X6rE_Irw: Extracting video information
[youtube] 3p8X6rE_Irw: Downloading DASH manifest
[youtube] 3p8X6rE_Irw: Downloading DASH manifest
[download] Destination: 'I have two scientific reasons why I am not an atheist' - Atheist Experience show #737-3p8X6rE_Irw.f135.mp4
[download] 100% of 46.37MiB in 00:04
[download] Destination: 'I have two scientific reasons why I am not an atheist' - Atheist Experience show #737-3p8X6rE_Irw.f141.m4a
[download] 100% of 22.24MiB in 00:05
[ffmpeg] Merging formats into "'I have two scientific reasons why I am not an atheist' - Atheist Experience show #737-3p8X6rE_Irw.mp4"
Deleting original file 'I have two scientific reasons why I am not an atheist' - Atheist Experience show #737-3p8X6rE_Irw.f135.mp4 (pass -k to keep)
Deleting original file 'I have two scientific reasons why I am not an atheist' - Atheist Experience show #737-3p8X6rE_Irw.f141.m4a (pass -k to keep)

@jaimeMF
Copy link
Collaborator

jaimeMF commented Sep 20, 2015

What I meant is that when you unlink libav you should be in the same state as when you initially got the insecure string pickle, but now it doesn't fail (the 'cexceptions\n...' message is expected because that's what print(repr(data)) does when it can't find an executable: avconv and avprobe in this case). So it seems to be working now, the merge is completed and youtube-dl succeeds .

@shelbyKiraM
Copy link
Author

Huh. Wondering how that got unlinked, but glad it got fixed. Bloody odd. Resolving this issue.

@jaimeMF
Copy link
Collaborator

jaimeMF commented Sep 20, 2015

It was definitely odd. Remember to remove the print(repr(data)) line from /usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py, otherwise you may get those messages in STDOUT and break some pipe.

@bvankuik
Copy link

Got the same issue. Installing libav, ffmpeg and rtmpdump made it work for me. Note that this was on a fresh homebrew install on a fresh El Capitan install.

@nhoizey
Copy link

nhoizey commented Nov 9, 2015

I had the same issue, just installing libav fixed it.

@adriweb
Copy link

adriweb commented Nov 22, 2015

Can confirm, installing libav with brew fixed the issue on OS X 10.11.1, with youtube-dl 2015.11.21.

@timvisher
Copy link

brew install libav also fixed this issue for me.

😺  sw_vers; brew --version; youtube-dl --version
ProductName:    Mac OS X
ProductVersion: 10.11.1
BuildVersion:   15B42
0.9.5 (git revision b3a74; last commit 2015-11-30)
2015.11.24

@kenorb
Copy link

kenorb commented Dec 5, 2015

I had the same problem and I had already libav installed.

It didn't work in Python 2.7.10:

youtube-dl "https://vimeo.com/139483119"

but it worked in Python 3:

python3 $(which youtube-dl) "https://vimeo.com/139483119"

I had to install rtmpdump, then it worked fined.

@yan12125
Copy link
Collaborator

Thanks for the suggestion. It may be a good idea for matplotlib but not for youtube-dl as we try to minimimize external dependencies as much as possible. Also, this issue can be closed when Python 2.7.12 is released.

@IanEdington
Copy link

@kenorb thanks that worked for me. I had to add builtin since which is aliased to something else in my system:

python3 $(builtin which youtube-dl) "https://vimeo.com/139483119"

@sg-s
Copy link

sg-s commented Feb 25, 2016

@nhoizey same thing worked for me. I had the same error, I simply

brew install libav

and everything works now

@flowerornament
Copy link

brew install libav rtmpdump worked for me (10.3.11). Strange though because it worked fine a couple weeks ago and haven't changed anything since then other than installing new packages, though today mysteriously I had to chown /usr/local.

@aalemayhu
Copy link

+1 for the brew install libav, made it work.

@wires
Copy link

wires commented Mar 5, 2016

FWIW another confirmation that updating libav worked (OSX 10.11.1) and so does python3.

After brew install:

[debug] Command-line args: [u'-tci', u'--verbose', u'--write-info-json', ...]
[debug] youtube-dl version 2016.03.01
[debug] Python version 2.7.10 - Darwin-15.0.0-x86_64-i386-64bit
[debug] exe versions: avconv 11.4, avprobe 11.4, ffmpeg 2.4.4, ffprobe 2.4.4, rtmpdump 2.4

🍰 🎉 ⛵

Notice the python version; after brew unlink it throws the error again:

[debug] youtube-dl version 2016.03.01
[debug] Python version 2.7.10 - Darwin-15.0.0-x86_64-i386-64bit
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/bin/youtube-dl/__main__.py", line 19, in <module>
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 411, in main
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 378, in _real_main
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 346, in __init__
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1944, in print_debug_header
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 51, in get_versions
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 35, in __init__
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 92, in _determine_executables
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 92, in <genexpr>
  File "/usr/local/bin/youtube-dl/youtube_dl/utils.py", line 1547, in get_exe_version
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1334, in _execute_child
    child_exception = pickle.loads(data)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1382, in loads
    return Unpickler(file).load()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 858, in load
    dispatch[key](self)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 966, in load_string
    raise ValueError, "insecure string pickle"
ValueError: insecure string pickle

But python Three has no problem with this however

» python3 (which youtube-dl) --verbose "https://www.youtube.com/watch?v=mtH8eqNKy7s"
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['--verbose', 'https://www.youtube.com/watch?v=mtH8eqNKy7s']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.03.01
[debug] Python version 3.5.0 - Darwin-15.0.0-x86_64-i386-64bit
[debug] exe versions: none
[debug] Proxy map: {}

@sg-s
Copy link

sg-s commented Mar 5, 2016

interesting -- i recently switched to python3, but installed youtube-dl when i was using 2.7. i wonder if that's the issue here

@arderyp
Copy link

arderyp commented Apr 30, 2016

installing libav fixed this error for me [youtube-dl 2016.04.24, python 2.7.11, osx 10.11.4 (15E65)]

@donnykurnia
Copy link

libav should be added as dependency.

@yan12125
Copy link
Collaborator

yan12125 commented May 8, 2016

ffmpeg/libav is currently an optional dependency. If you don't need to merge video/audio formats, add metadata, embed subtitles/thumbnails or any other features relying on ffmpeg/libav, there's no need to install it. This bug is caused by broken Python instead of the lack of ffmpeg/libav. Installing ffmpeg/libav is just a workaround.

@GRMrGecko
Copy link

I had to install rtmpdump to fix mine.

@yan12125
Copy link
Collaborator

For those who are affected: in http://bugs.python.org/issue26083, @gpshead applied a patch on Python 2.x branch, while the patch is considered buggy. Use Python 3 before the Python bug is fixed.

@yan12125
Copy link
Collaborator

yan12125 commented Jul 5, 2016

Python 2.7.12 is released, [1] closing. rtmpdump and libav/ffmpeg are no longer necessary for basic usages.

[1] https://hg.python.org/cpython/raw-file/v2.7.12/Misc/NEWS

@lazd
Copy link

lazd commented Apr 3, 2017

Oddly enough, on Python 2.7.10, youtube-dl worked perfectly for me yesterday, but today I got the insecure string pickle error. I installed both rtmpdump and libav with brew and still had the error. Installing Python 2.7.13 fixed the issue.

@daviewales
Copy link

daviewales commented Aug 31, 2017

The default Python on Mac is still stuck at 2.7.10 (at least on macOS 10.11).
Youtube-dl works fine with brew installed Python3, but still has this pickle error without libav with the default Python. Can we add a more helpful error message, or set the brew version of youtube-dl to require python3 by default?

@yan12125
Copy link
Collaborator

add a more helpful error message

It's not trivial to detect the reason that causes "insecure string pickle" reliably. I'm not sure it's even possible or not. If you have a concrete approach, feel free to open a new pull request.

set the brew version of youtube-dl to require python3 by default?

Mandatory dependency may not be a good idea. If I remember it correctly Homebrew has recommended or optional dependencies. You may want to forward this idea to the Homebrew team.

@daviewales
Copy link

daviewales commented Aug 31, 2017 via email

@yan12125
Copy link
Collaborator

Do you know how the binary chooses which Python runs it?

Just took a look. Homebrew also uses /usr/bin/env python, so the first python binary in $PATH is picked.

@DrHyde
Copy link

DrHyde commented Oct 12, 2017

FYI I just ran into this (#14475 is my ticket). Some Homebrew formulae accept a --with-python argument, but youtube-dl doesn't. Unfortunately they don't accept requests for changes to formulae, only pull requests, so I can't even ask them to support using Homebrew's own build of python instead of the system default, and I can't do it myself as I am not skilled in ruby or python.

So for now I've just created this shell script that comes first in my $PATH: DrHyde/shellscripts@1130d7a

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests