Non-ASCII character '\x91' in file /usr/pkg/bin/youtube-dl on line 3

[mmayer]

I am getting the above error on Mac OS X 10.9 using python 2.7.

$ head /usr/pkg/bin/youtube-dl

!/usr/pkg/bin/python2.7

PK [...zipped data follows...]

$ youtube-dl
File "/usr/pkg/bin/youtube-dl", line 2
SyntaxError: Non-ASCII character '\x91' in file /usr/pkg/bin/youtube-dl on line 3, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details

$ /usr/pkg/bin/python2.7 --version
Python 2.7.3

Once I unzip it, I can call it okay.

$ python ./main.py
Usage: main.py [options] url [url...]

main.py: error: you must provide at least one URL

$ python ./main.py --version
2014.04.11.2

I installed it using pkgsrc (http://www.pkgsrc.org). This is what it downloads and extracts:

$ ls -l youtube-dl
-rwxr-xr-x 1 markus staff 444556 11 Apr 00:44 youtube-dl

$ gmd5sum youtube-dl
000d7a9c3b42ce319ce338379b40d897 youtube-dl

Any tips what else I can try to figure out what it is doing?

Thanks,
-Markus

[phihag]

It looks like pkg modifies the binaries for another Python path. Can you try

curl https://yt-dl.org/latest/youtube-dl`-o ./youtube-dl
python --version
python ./youtube-dl -v
/usr/pkg/bin/python2.7 ./youtube-dl -v

and post the output here? I'll try to reproduce your setup in the meantime.

[mmayer]

Alright. Here is what I did:

$ wget https://yt-dl.org/latest/youtube-dl
--2014-05-14 10:35:27-- https://yt-dl.org/latest/youtube-dl
Resolving yt-dl.org (yt-dl.org)... 95.143.172.170, 2001:1a50:11::5f:8f:acaa:177
Connecting to yt-dl.org (yt-dl.org)|95.143.172.170|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 469808 (459K) [application/octet-stream]
Saving to: ‘youtube-dl’

100%[================================================================================>] 469,808 167KB/s in 2.7s

2014-05-14 10:35:31 (167 KB/s) - ‘youtube-dl’ saved [469808/469808]

$ python --version
Python 2.7.5

$ python ./youtube-dl -v
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-v']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2014.05.13
[debug] Python version 2.7.5 - Darwin-13.1.0-x86_64-i386-64bit
[debug] Proxy map: {}
Usage: youtube-dl [options] url [url...]

youtube-dl: error: you must provide at least one URL

$ /usr/pkg/bin/python2.7 --version
Python 2.7.3

$ /usr/pkg/bin/python2.7 ./youtube-dl -v
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-v']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2014.05.13
[debug] Python version 2.7.3 - Darwin-13.1.0-x86_64-i386-64bit
[debug] Proxy map: {}
Usage: youtube-dl [options] url [url...]

youtube-dl: error: you must provide at least one URL

So it does work with both Python interpreters, but it does not like the first line of the script changed. (When I manually change "#!/usr/bin/env python" in an editor, it starts failing again.)

Thanks for your help.

I'll poke around a bit more to see if there's an easy way to make this work again with pkgsrc. (It tries to ensure its own Python interpreter is being used, because that way it can ensure it's the right version rather than relying on the system to provide it. Of course that doesn't work so well here.)

Regards,
-Markus

[phihag]

The problem seems to be that editors start convertin line endings, and that doesn't make any sense in a binary zip file.

[mmayer]

Whatever is happening, it doesn't mess with the line endings of the ZIP portion. I can remove the first line and unzip the archive without issues (even the archives Python refused to run).

[phihag]

Can you upload the file generated by pkg_src somewhere? Note that when some bytes (e.g. EOLs) are changed, the result may be a valid zip file, but (some) its contents may be gibberish, and therefore it won't be valid Python code anymore.

[mmayer]

I put the ZIP file up here: https://www.dropbox.com/s/bmjnm3p0xz955a8/youtube-dl

This was created by pkgsrc from youtube-dl-2014.04.11.2.tar.gz.

Here's the original before pkgsrc replaces the interpreter: https://www.dropbox.com/s/sj0xco08ysqxqy7/youtube-dl.orig