generate console_scripts wrappers on wheel install (optional)

[dholth]

bdist_wheel will stop putting pre-generated setuptools console_scripts into wheels.

When that happens pip will need to generate them (and it would be cool to be able to tell it it doesn't have to) for example, wheel's own command "wheel install-scripts packagename" does it.

[qwcode]

ok, so pip would need this in 1.5 (a few months away), before you could release this change in bdist_wheel (i.e. wheel)

don't understand this "and it would be cool to be able to tell it it doesn't have to"

[dholth]

I mean if you aren't going to use the console scripts you could avoid writing them out at all.

[qwcode]

in another issue, you mention, that it's a new option --skip-scripts, but I don't see it in your default branch.
so it's a new option, defaulted to false? not just a change in default behavior, period?

[dholth]

I just pushed --skip-scripts to https://bitbucket.org/dholth/wheel/. I decided it was probably better to keep the admittedly less-than-cross-platform scripts wrappers in wheels until pip can do something about them.

[dholth]

In the meantime I'm loving wheel install-scripts pkgname. Every time I get the error "can't find the old version of pip because you're doing development" I can just run that.

[pfmoore]

I've closed #1152 as a duplicate, but there's a bit more analysis in that issue that's worth reviewing (for anyone reading this issue in isolation).

[pfmoore]

I might try to work on this. As the contents of the SCRIPTS section of wheels is typically only setuptools-generated wrapper scripts, I propose that the default behaviour be to ignore the contents of SCRIPTS and generate new wrappers based on entry-points.txt (probably using the embedded distlib's script generation feature). For the (likely rare) cases where SCRIPTS contains things that are needed, add an --include-legacy-scripts flag that copies anything in SCRIPTS that is not already generated as an entry point.

Does that seem reasonable?

As an extra, it may be possible to extend entry_points.txt to say that a wrapper should be "versioned" - i.e., as well as cmd.exe, we also generate cmdX.exe and cmdX.y.exe (where X.Y is the target Python version). This would only be a short-term measure, though - proper metadata for versioned scripts should be a Metadata 2.0 feature.

[dstufft]

We should be able to reliably tell which scripts are entrypoints or not, see if their name matches an entry in entry_points.txt

Your idea about extending made me think though, we can make arbitrary entry points, so why not make just "pip" be the console_entrypoint, and then add a console_version_entrypoint that will do string formatting for like %(py_major)s and %(py_minor)s. For setuptools this would install just a pip binary which isn't the most ideal thing but it would work and we could possibly get this added to setuptools itself. Nick should be consulted as he had an idea along this lines for the new metadata 2.0 so we'd want something that could be easily translated.

[pfmoore]

We should be able to reliably tell which scripts are entrypoints or not, see if their name matches an entry in entry_points.txt

Relatively reliably, yes. For windows, entry point XXX matches XXX.exe and XXX-script.exe. For Unix it matches XXX (I assume - I'd need to check that). So any of the above 3 should be ignored if XXX is a console script entry point (because the wheel could have been generated on Windows or Unix). Are there gui script entry points? I can't remember offhand but if there are, include those too.

Anything else I guess we can just copy across. Although in practice, I suspect there's a good chance they would be relatively unreliable anyway (in my experience, anything else is usually a Unix shell script that won't work on Windows, or maybe a bat file that sort-of works). I don't recall ever seeing a custom script that worked the way I expected it to on Windows... So I tend to prefer omitting them by default as a way of pushing people towards using entry points rather than rolling their own solution. It's just an idea at the moment, so let's wait to see how the code looks before worrying too much.

[dstufft]

Yes there is gui entry points I forget what they are called.

It's perfectly reasonable to have a linux only (or Windows only) wheel that contains hand crafted scripts, I don't think we should break that.

[qwcode]

I vote for not omitting custom scripts by default if possible. That seems like too much of a change. Ditto on trying to determine what's custom based on matching with the entry points.

As for using distlib's script support, recall this thread: https://mail.python.org/pipermail/distutils-sig/2013-July/021872.html

I don't think distlib wrappers enforce the script to have properly fulfilled dependencies, where setuptools wrappers do. That's a difference to be aware of, but not sure if the implications are that important.

[pfmoore]

Happy to skip the "don't emit custom scripts by default" bit. But if we include pregenerated entry points and try to write our own, we'll get in a mess unless wheel building tools are converted to skip entry point scripts (that's pip wheel/bdist_wheel, wheel convert and distlib). That's an option, as there aren't that many wheels "in the wild" yet, but it makes a subtly incompatible wheel, and I thought that was deferred for Wheel 1.1.

Yeah, distlib script support would have to be thought through.

@dstufft I'm not saying it's not possible to have hand crafted scripts, just that in my experience it doesn't happen now and this is a short-term proposal pending Metadata 2.0/sdist 2.0/etc. So "good enough" may be reasonable for now.

[pfmoore]

I don't think distlib wrappers enforce the script to have properly fulfilled dependencies, where setuptools wrappers do. That's a difference to be aware of, but not sure if the implications are that important.

Personally, I consider that a feature not an issue. The dependency checking means that setuptools wrappers are dependent on pkg_resources at runtime, which I view as a significant downside.

[dstufft]

I don't think we need to include the pre generated scripts, infact I think we shouldn't. I just know if we skip custom scripts we're going to break things for people. Openstack for instance use a custom script on Linux (but entrypoints on Windows) in order to cut out pkg_resources. Importing pkg_resource was causing enough delay on start up that they felt that was a better option.

I agree on getting rid of the pkg_resources runtime checks being a feature not an issue. It would greatly speed up start up time on those scripts and enable better -e usage. If someone really wants it they could simply implement the check themselves.

[pfmoore]

Note that the fun bit about not including the pregenerated scripts will be wheel convert (a critical use case for me and possibly many Windows users for migrating wininst installers). The wininst does include the wrappers (and that won't change) so all we're doing is moving the "ignore wrappers" logic from wheel installs to wheel convert. Still a good idea, certainly, but it needs doing.

I see another chain of prerequisites starting up. I'll raise a feature request for wheel for now. I can get away for the initial stages by hand-hacking wheels, but ultimately this will need to be part of the wheel spec. (Actually, @dholth didn't this come up before and did you say that omitting script wrappers should be left for Wheel 1.1? If so, we're hitting a bit of a chicken and egg issue here...)

[dstufft]

The wheel library currently has an option where you can ask it not to generate the scripts (it was on by default for one version). Theoretically once a version of pip that generates scripts is available it could turn the generating of scripts off by default.

[pfmoore]

OK, cool. That still leaves wheel convert (and less importantly wheel install) needing to change, correct?

[dstufft]

Uhh, Yea wheel convert would want to have the same thing as we're doing in pip where it excludes any entry points. For wheel install it would want to generate scripts as well (however it has it's own command to generate them already).

[pfmoore]

See #1251 for work in progress on this

[dstufft]

Fixed in #1251