Eggs/Wheels have broken shebangs

[snordhausen]

For distribution on Pypi, I wanted to create both Eggs and Wheels with

    project.set_property("distutils_commands", ["sdist", "bdist_egg", "bdist_wheel"])

The project includes a script in src/main/scripts with a shebang of #!/usr/bin/env python. In the "sdist", this shebang remains intact and it works fine when installed from Pypi.

In the Egg and the Wheel, there is a modified shebang that points to the Python interpreter in the virtual environment that I use for building. Consequently, these files do not work when installed via Pypi. The same thing happens when I call the auto-generated setup.py myself.

I was able to work around this issue (after some inspiration) by manually adding

options = {'build_scripts': {'executable': '/usr/bin/env python'}},

into the setup.py, deleting the temporary files and then calling setup.py myself.

[mriehl]

Hi, thanks for opening an issue. I actually already ran into the same problem, investigated, and decided that I will not use wheels for the time being.

In my opinion this is a bug in distutils. The PEP recommends to write #!python in your shebang which I think is stupid beyond belief since the script will not run unless you use some magic installer like pip which replaces the shebang.
Note that I'm not making this up

I guess we can hardcode the executable in the setup script but I'm not sure how this might affect existing users.

[mriehl]

Sorry for the rant... but what do you think? Did you know about the #!python part? (do you still want to use wheels? :-P). Maybe using #!python would be a possibility.. honestly I really like the simplicity of a tarball so apart from huge projects like numpy I never bother with wheels.

Also I'm not able to find documentation on executable.. what does it do? Does it always replace the shebang? e.G. if I have a script with #!/usr/bin/python3 will it rewrite it to #!/usr/bin/env python?

[mriehl]

@snordhausen any update on this? IMHO the wheel spec sucks and should be changed so I'd rather not work around it. What do you think?

[arcivanov]

@mriehl question for you based on this:

which I think is stupid beyond belief since the script will not run unless you use some magic installer like pip which replaces the shebang.

It does assume the use of pip or a similar installer but here's the deal - you can't install a wheel without pip or a similar tool to begin with. And pip already does a proper rewrite, I believe.

Setuptools, however, does rewrite the shebang for wheel only.

We have a bug how we package as the order of invocation of setup tools matters.
If you do in pyb build.py clean package and then run ./setup.py bdist_wheel sdist you'll notice that the shebang in the pyb in wheel will be #!python and in sdist it'll be untouched.

However if you clean again and run ./build.py and then go and run ./setup.py bdist_wheel, the build/scripts-2.7/pyb already has a rewritten shebang and wheel picks that crap up.

[arcivanov]

So either we have to blow away $dist_dir/build or stop generating useless bdist_dumb that rewrites shebang right in the $dist_dir/build/scripts-2.7/pyb.

[arcivanov]

I'm going to implement the following solution:

• Change defaults to ['sdist', 'bdist_wheel'].
• Run clean -all before every command to ensure to insulate the individual builds from each other.

[mriehl]

You're right, bdist_dump is usually not what you want.

It does assume the use of pip or a similar installer but here's the deal - you can't install a wheel without pip or a similar tool to begin with. And pip already does a proper rewrite, I believe.

That's true, but my point is that if I get the project sources and try to run the file (which should always work, IMHO) it won't run because the shebang is broken. So the app is broken unless I actually install it.
I like the "have the cake and eat it too" approach from being able to run the sources or install as a package, which is no longer easy with wheels.

[mriehl]

• Change defaults to ['sdist', 'bdist_wheel'].
• Run clean -all before every command to ensure to insulate the individual builds from each other.

Sounds good.