Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Passing binary files as 'scripts' causes errors at develop and install. #210

Closed
ghost opened this issue May 28, 2014 · 21 comments
Closed

Passing binary files as 'scripts' causes errors at develop and install. #210

ghost opened this issue May 28, 2014 · 21 comments
Labels

Comments

@ghost
Copy link

@ghost ghost commented May 28, 2014

Originally reported by: davidoff (Bitbucket: davidoff, GitHub: davidoff)


If you specify a binary file to the scripts= parameter of setup, the setup.py develop command copies the script with bytes 0D converted to 0A, garbling the binary data.

I have traced this to the text mode file opening in setuptools/command/develop.py:165 (introduced in c9e11fdff773).

If I change from open(script_path, 'rU') to open(script_path, 'rb') the binary files are copied correctly.


@ghost
Copy link
Author

@ghost ghost commented May 28, 2014

Original comment by davidoff (Bitbucket: davidoff, GitHub: davidoff):


Test all pass except SVN related tests.

@ghost
Copy link
Author

@ghost ghost commented May 28, 2014

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


What is the behavior of scripts copied during 'install'? Are installed scripts not also affected?

@ghost
Copy link
Author

@ghost ghost commented May 29, 2014

Original comment by davidoff (Bitbucket: davidoff, GitHub: davidoff):


Just tested, and no, the behavior is as expected with the install command,
The binary files are identical.

@ghost
Copy link
Author

@ghost ghost commented Jun 1, 2014

Original comment by davidoff (Bitbucket: davidoff, GitHub: davidoff):


Resolved by this pull request #56.

@ghost
Copy link
Author

@ghost ghost commented Jun 1, 2014

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


Merged in davidoff/setuptools (pull request #56)

Script files to be copied in the develop command should be open in binary mode. Fixes #210.

1 similar comment
@ghost
Copy link
Author

@ghost ghost commented Jun 1, 2014

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


Merged in davidoff/setuptools (pull request #56)

Script files to be copied in the develop command should be open in binary mode. Fixes #210.

@ghost
Copy link
Author

@ghost ghost commented Jun 6, 2014

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


This issue has caused a regression reported in #218

@ghost
Copy link
Author

@ghost ghost commented Jun 6, 2014

Original comment by davidoff (Bitbucket: davidoff, GitHub: davidoff):


Do you have a suggestion to fix it so that binaries and text files are properly copied? I will try to add a test for this.

@ghost
Copy link
Author

@ghost ghost commented Jun 14, 2014

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


I'm re-opening this ticket, as 3.8.1 is now the stable release and 4.x is currently broken.

@ghost
Copy link
Author

@ghost ghost commented Jun 14, 2014

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


David, I'm not sure distutils/setuptools should necessarily support arbitrary binaries as 'scripts'. In the Python documentation, it defines scripts as "Scripts are files containing Python source code". Under this assumption, it's reasonable for setuptools to open these files as text.

I acknowledge that you've created pull request 60 to address the regression, but I'm reluctant to accept that pull request as it adds quite a bit more undocumented special casing, creating more untested, implicit requirements on setuptools.

@ghost
Copy link
Author

@ghost ghost commented Jun 14, 2014

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


What I'm failing to understand is why easy_install.install_egg_scripts doesn't encounter this same error, since it apparently uses dist.get_metadata to load the script as a binary blob.

Can you provide a reference to a project (preferably one with minimal dependencies and system requirements) that experiences this failure?

@ghost
Copy link
Author

@ghost ghost commented Jun 15, 2014

Original comment by davidoff (Bitbucket: davidoff, GitHub: davidoff):


As I understood the feature, and the usage we have in a normal install, is that scripts are copied to the python Scripts installation directory. On windows, this brings a fine way to include things in a package that can be bundled and used with the package, binary or not.

For example, we bundle a custom version of the python interpreter built with another build system (but it could be any other auxiliary binary support file). Let's call it myPython.exe (with it's runtime dependencies (dll etc)). The package to be installed is ourPythonDriver.py. Now when people install ourPythonDriver.py, they get also get myPython.exe installed, and it is available in the environment.

It is very practical that this feature works also with the develop command, and having a different semantics from the install and the develop command seems like a bug. Even if binary support is not explicitly mentioned in the docs, using binary files specified as scripts break only on windows because of the line ending translation caused by text mode opening. This happens only with the develop command.

@ghost
Copy link
Author

@ghost ghost commented Jun 23, 2014

Original comment by davidoff (Bitbucket: davidoff, GitHub: davidoff):


You can easily clone a sample project I created on GitHub to see the problem

https://github.com/davidovich/develop_breakage_on_windows

With pull request #60, this does not crash.

Without it, it crashes.

@jaraco
Copy link
Member

@jaraco jaraco commented May 3, 2016

Even not on Windows, the package presented fails to build on Python 3 on Unix:

$ hg clone gh://davidovich/develop_breakage_on_windows
destination directory: develop_breakage_on_windows
importing git objects into hg
updating to branch default
5 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cd develop_breakage_on_windows 
$ python setup.py develop --user
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py:261: UserWarning: Unknown distribution option: 'pacakges'
  warnings.warn(msg)
running develop
running egg_info
creating setuptools_develop_breaker.egg-info
writing setuptools_develop_breaker.egg-info/PKG-INFO
writing dependency_links to setuptools_develop_breaker.egg-info/dependency_links.txt
writing top-level names to setuptools_develop_breaker.egg-info/top_level.txt
writing manifest file 'setuptools_develop_breaker.egg-info/SOURCES.txt'
reading manifest file 'setuptools_develop_breaker.egg-info/SOURCES.txt'
writing manifest file 'setuptools_develop_breaker.egg-info/SOURCES.txt'
running build_ext
Creating /Users/jaraco/Library/Python/3.5/lib/python/site-packages/setuptools-develop-breaker.egg-link (link to .)
Adding setuptools-develop-breaker 0.0.0 to easy-install.pth file
Traceback (most recent call last):
  File "setup.py", line 6, in <module>
    scripts=['binary_hello_world.exe']
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/command/develop.py", line 34, in run
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/command/develop.py", line 133, in install_for_development
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/command/easy_install.py", line 697, in process_distribution
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/command/develop.py", line 168, in install_egg_scripts
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x90 in position 2: invalid start byte
$ python setup.py sdist
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py:261: UserWarning: Unknown distribution option: 'pacakges'
  warnings.warn(msg)
running sdist
running egg_info
writing dependency_links to setuptools_develop_breaker.egg-info/dependency_links.txt
writing top-level names to setuptools_develop_breaker.egg-info/top_level.txt
writing setuptools_develop_breaker.egg-info/PKG-INFO
reading manifest file 'setuptools_develop_breaker.egg-info/SOURCES.txt'
writing manifest file 'setuptools_develop_breaker.egg-info/SOURCES.txt'
warning: sdist: standard file not found: should have one of README, README.rst, README.txt

running check
warning: check: missing required meta-data: url

warning: check: missing meta-data: either (author and author_email) or (maintainer and maintainer_email) must be supplied

creating setuptools_develop_breaker-0.0.0
creating setuptools_develop_breaker-0.0.0/setuptools_develop_breaker.egg-info
making hard links in setuptools_develop_breaker-0.0.0...
hard linking binary_hello_world.exe -> setuptools_develop_breaker-0.0.0
hard linking setup.py -> setuptools_develop_breaker-0.0.0
hard linking setuptools_develop_breaker.egg-info/PKG-INFO -> setuptools_develop_breaker-0.0.0/setuptools_develop_breaker.egg-info
hard linking setuptools_develop_breaker.egg-info/SOURCES.txt -> setuptools_develop_breaker-0.0.0/setuptools_develop_breaker.egg-info
hard linking setuptools_develop_breaker.egg-info/dependency_links.txt -> setuptools_develop_breaker-0.0.0/setuptools_develop_breaker.egg-info
hard linking setuptools_develop_breaker.egg-info/top_level.txt -> setuptools_develop_breaker-0.0.0/setuptools_develop_breaker.egg-info
Writing setuptools_develop_breaker-0.0.0/setup.cfg
creating dist
Creating tar archive
removing 'setuptools_develop_breaker-0.0.0' (and everything under it)
$ ls dist
setuptools_develop_breaker-0.0.0.tar.gz
$ python -m easy_install --user dist/setuptools_develop_breaker-0.0.0.tar.gz 
Processing setuptools_develop_breaker-0.0.0.tar.gz
Writing /var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/easy_install-bffnbre1/setuptools_develop_breaker-0.0.0/setup.cfg
Running setuptools_develop_breaker-0.0.0/setup.py -q bdist_egg --dist-dir /var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/easy_install-bffnbre1/setuptools_develop_breaker-0.0.0/egg-dist-tmp-ouau29u5
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py:261: UserWarning: Unknown distribution option: 'pacakges'
  warnings.warn(msg)
warning: install_lib: 'build/lib' does not exist -- no Python modules to install

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/tokenize.py", line 392, in find_cookie
    line_string = line.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x90 in position 2: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/sandbox.py", line 154, in save_modules
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/sandbox.py", line 195, in setup_context
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/sandbox.py", line 243, in run_setup
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/sandbox.py", line 273, in run
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/sandbox.py", line 242, in runner
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/sandbox.py", line 46, in _execfile
  File "/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/easy_install-bffnbre1/setuptools_develop_breaker-0.0.0/setup.py", line 6, in <module>
    scripts=['binary_hello_world.exe']
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/command/bdist_egg.py", line 191, in run
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/command/bdist_egg.py", line 147, in call_command
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/command/install_scripts.py", line 20, in run
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/command/install_scripts.py", line 43, in run
    self.run_command('build_scripts')
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/command/build_scripts.py", line 50, in run
    self.copy_scripts()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/command/build_scripts.py", line 82, in copy_scripts
    encoding, lines = tokenize.detect_encoding(f.readline)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/tokenize.py", line 433, in detect_encoding
    encoding = find_cookie(first)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/tokenize.py", line 397, in find_cookie
    raise SyntaxError(msg)
SyntaxError: invalid or missing encoding declaration for 'binary_hello_world.exe'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/easy_install.py", line 5, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/command/easy_install.py", line 2241, in main
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/command/easy_install.py", line 391, in run
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/command/easy_install.py", line 621, in easy_install
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/command/easy_install.py", line 670, in install_item
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/command/easy_install.py", line 850, in install_eggs
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/command/easy_install.py", line 1078, in build_and_install
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/command/easy_install.py", line 1064, in run_setup
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/sandbox.py", line 246, in run_setup
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/sandbox.py", line 195, in setup_context
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/sandbox.py", line 166, in save_modules
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/sandbox.py", line 141, in resume
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/pkg_resources/_vendor/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/sandbox.py", line 154, in save_modules
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/sandbox.py", line 195, in setup_context
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/sandbox.py", line 243, in run_setup
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/sandbox.py", line 273, in run
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/sandbox.py", line 242, in runner
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/sandbox.py", line 46, in _execfile
  File "/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/easy_install-bffnbre1/setuptools_develop_breaker-0.0.0/setup.py", line 6, in <module>
    scripts=['binary_hello_world.exe']
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/command/bdist_egg.py", line 191, in run
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/command/bdist_egg.py", line 147, in call_command
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools-20.10.1-py3.5.egg/setuptools/command/install_scripts.py", line 20, in run
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/command/install_scripts.py", line 43, in run
    self.run_command('build_scripts')
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/command/build_scripts.py", line 50, in run
    self.copy_scripts()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/command/build_scripts.py", line 82, in copy_scripts
    encoding, lines = tokenize.detect_encoding(f.readline)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/tokenize.py", line 433, in detect_encoding
    encoding = find_cookie(first)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/tokenize.py", line 397, in find_cookie
    raise SyntaxError(msg)
SyntaxError: invalid or missing encoding declaration for 'binary_hello_world.exe'

Notice that it fails to develop and it fails to install, and the failure in install occurs in distutils, reinforcing the idea that scripts, as the namespace and documentation indicates, is for scripts and not for binary executables. Given the traceback, I suspect that even the patch in the referenced pull request would not correct the issue for installation.

It still feels wrong to me to be hacking scripts to support something which they weren't intended to support, so here are a few options to consider:

  • A new feature, an option that allows installing arbitrary binary files to the bin or Scripts directories.
  • Use the data_files to install the relevant directory (although beware of #130).
  • Store the binary files as package data and have a separate command that links or copies those files into the scripts directory following installation (a custom post-install step).
@jaraco jaraco changed the title On Windows, binary files specified as scripts are garbled by setup.py develop. Passing binary files as 'scripts' causes errors at develop and install. May 3, 2016
@jaraco jaraco added the invalid label May 3, 2016
@jaraco
Copy link
Member

@jaraco jaraco commented May 3, 2016

Accordingly, as a bug, I'm declaring this invalid, though I welcome follow-on tickets to implement one of the suggestions above or perhaps another enhancement to better support this need.

@benfred
Copy link

@benfred benfred commented Sep 25, 2018

I hit this problem too - and implemented the 3rd suggestion by @jaraco (custom post-install script to copy binary over to the scripts directory).

The code is here in case anyone is interested: https://github.com/benfred/py-spy/blob/290584dde76834599d66d74b64165dfe9a357ef5/setup.py#L42

I'm using this to distribute a rust binary with pypi (so you can go pip install py-spy and get a precompiled rust binary installed).

@ntc2
Copy link

@ntc2 ntc2 commented Dec 24, 2018

I just ran into this while porting to Python 3 a setup.py that used scripts to install a compiled binary, and a simple solution was to use data_files instead. I.e. I added

data_files = [ ('bin', [<my binary>]) ]

One important difference vs using scripts is that for data_files the permissions don't get set automatically. So, be sure to chmod a+rx on the <my binary> first!

ntc2 added a commit to LaBatata101/fluxgui that referenced this issue Dec 24, 2018
Apparently you can no longer install arbitrary binaries using
`scripts`. Instead you need to use `data_files` now.

Discussion of error in Python 3 when trying to use `scripts` to
install binaries here:
pypa/setuptools#210. Basically, you get an
error about a failed parse when setup tools tries to rad the binary as
a Python or #! script :P
ntc2 added a commit to LaBatata101/fluxgui that referenced this issue Dec 24, 2018
Apparently you can no longer install arbitrary binaries using
`scripts`. Instead you need to use `data_files` now.

Discussion of error in Python 3 when trying to use `scripts` to
install binaries here:
pypa/setuptools#210. Basically, you get an
error about a failed parse when setup tools tries to rad the binary as
a Python or #! script :P
@atomass
Copy link

@atomass atomass commented Aug 23, 2019

One important difference vs using scripts is that for data_files the permissions don't get set automatically. So, be sure to chmod a+rx on the <my binary> first!

Actually I just tried and it looks that now the permissions are been correctly kept.

@DongqingSun96
Copy link

@DongqingSun96 DongqingSun96 commented Mar 11, 2020

@ntc2 Hi, when I use data_files to add some binary file to bin, it seems that the file is copied to a temporary directory like build/bdist.linux-x86_64/egg/bin rather than installation_prefix/bin. And I use the setup function from setuptools module not distutils.core module. How can I solve this problem? Thanks!

@ntc2
Copy link

@ntc2 ntc2 commented Mar 14, 2020

@DongqingSun96 sorry, I don't remember anything about how this works, and it sounds like it might have changed since my comment above.

@jaraco
Copy link
Member

@jaraco jaraco commented Mar 15, 2020

@DongqingSun96 Please open a new issue, describing your issue - what commands you ran, what versions of Python and Setuptools you were using, what you encountered, and what you expected instead. In particular, if you can describe the problem in a way that the team can replicate, that would go a long way to getting more insight into the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants