using setuptools installs every script twice #335

Closed
minrk opened this Issue Apr 7, 2011 · 16 comments

Comments

Projects
None yet
4 participants
Owner

minrk commented Apr 7, 2011

Scripts are registered as setuptools entry_points in setup.py, but also with the find_scripts() function in setupbase.py. When setup is run with setuptools (via setupegg.py), then every script is installed twice.

This duplicate listing also makes it easier for new scripts to not be properly added to setup, but appear so for developers.

I don't know enough about these things, but what is the point of having both?

output of python setupegg.py install:

...
Installing ipcluster script to /usr/local/bin
Installing ipcontroller script to /usr/local/bin
Installing ipengine script to /usr/local/bin
Installing iplogger script to /usr/local/bin
Installing iptest script to /usr/local/bin
Installing ipython script to /usr/local/bin
Installing ipython-qtconsole script to /usr/local/bin
Installing irunner script to /usr/local/bin
Installing pycolor script to /usr/local/bin
Installing ipcontroller script to /usr/local/bin
Installing iptest script to /usr/local/bin
Installing ipcluster script to /usr/local/bin
Installing ipython script to /usr/local/bin
Installing pycolor script to /usr/local/bin
Installing iplogger script to /usr/local/bin
Installing irunner script to /usr/local/bin
Installing ipengine script to /usr/local/bin
Installing ipython-qtconsole script to /usr/local/bin

Obviously half of those are the auto-generated scripts from setuptools, and the other half are the explicit scripts in IPython/scripts and elsewhere.

Owner

fperez commented Apr 7, 2011

On Thu, Apr 7, 2011 at 12:22 PM, minrk
reply@reply.github.com
wrote:

I don't know enough about these things, but what is the point of having both?

I'm not sure how to get rid of the duplication right now (it's been a
long time since I dug into distutils/setuptools). But one key point
is that the installation MUST always work without setuptools in the
picture. So the scripts do need to be registered in normal
non-setuptools fashion.

Now, perhaps what we should do is, in find_scripts, have something like

if 'setuptools' in sys.modules:
 return []

so that a setuptools-driven installation doesn't produce duplicate scripts.

f

Owner

ellisonbg commented Apr 7, 2011

On Thu, Apr 7, 2011 at 12:22 PM, minrk
reply@reply.github.com
wrote:

Scripts are registered as setuptools entry_points in setup.py, but also with the find_scripts() function in setupbase.py.  When setup is run with setuptools (via setupegg.py), then every script is installed twice.

This duplicate listing also makes it easier for new scripts to not be properly added to setup, but appear so for developers.

I don't know enough about these things, but what is the point of having both?

You want to have setuptools know about them because them enables the
scripts to be intalled properly on Windows (I think as .bat files).
But we should only do one or the other and should add logic to detect
that. We just need to see if setuptools has been imported or not when
the main setup.py is being run. If not, then we add the script the
normal (setup.py) way, if setuptools is detected, we add them using
the setuptools way.

Brian

output of python setupegg.py install:

...
Installing ipcluster script to /usr/local/bin
Installing ipcontroller script to /usr/local/bin
Installing ipengine script to /usr/local/bin
Installing iplogger script to /usr/local/bin
Installing iptest script to /usr/local/bin
Installing ipython script to /usr/local/bin
Installing ipython-qtconsole script to /usr/local/bin
Installing irunner script to /usr/local/bin
Installing pycolor script to /usr/local/bin
Installing ipcontroller script to /usr/local/bin
Installing iptest script to /usr/local/bin
Installing ipcluster script to /usr/local/bin
Installing ipython script to /usr/local/bin
Installing pycolor script to /usr/local/bin
Installing iplogger script to /usr/local/bin
Installing irunner script to /usr/local/bin
Installing ipengine script to /usr/local/bin
Installing ipython-qtconsole script to /usr/local/bin

Obviously half of those are the auto-generated scripts from setuptools, and the other half are the explicit scripts in IPython/scripts and elsewhere.

Reply to this email directly or view it on GitHub:
#335

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

Owner

fperez commented Apr 7, 2011

On Thu, Apr 7, 2011 at 12:36 PM, ellisonbg
reply@reply.github.com
wrote:

 If not, then we add the script the
normal (setup.py) way, if setuptools is detected, we add them using
the setuptools way.

Yup, I posted the code in my reply; Min, if you're working on this now
I think you can go ahead and merge it in, just run a quick test
install with/without setuptools just to be sure (using some junk
--prefix you can nuke or a virtualenv). But that two-line fix should
do the trick.

cheers,

f

Owner

minrk commented Apr 7, 2011

There is already an if 'setuptools' in sys.modules check in setup.py, so I was making the switch there: https://gist.github.com/908567, but the result is the same.

I know Python package management gets mocked pretty mercilessly for being so terrible, but is it really bad enough that we have to define all our scripts twice?

Contributor

rkern commented Apr 7, 2011

Works for me. Did you remove your build/ directory? The scripts may have already been staged there.

Contributor

rkern commented Apr 7, 2011

I'm also using distribute rather than setuptools proper. Maybe they've fixed something there.

Owner

fperez commented Apr 7, 2011

On Thu, Apr 7, 2011 at 12:58 PM, minrk
reply@reply.github.com
wrote:

There is already an if 'setuptools' in sys.modules check in setup.py, so I was making the switch there: https://gist.github.com/908567, but the result is the same.

Yes, that's cleaner. But where you remove the scripts call at the
top, please fix the comment above that mentions scripts, indicating
those get defined later on with a setuptools conditional. Otherwise
the code and comment will be out of sync.

I know Python package management gets mocked pretty mercilessly for being so terrible, but is it really bad enough that we have to define all our scripts twice?

Well, now they will only be defined once :)

f

Owner

minrk commented Apr 7, 2011

No, they are still defined twice (in setup.py and setupbase.py), but only installed once after the fix. It's pretty pathetic that we have to specify it in both setuptools and non-setuptools ways. Is it really not possible for setuptools entry points to use existing scripts?

Robert: I, too, am using Distribute (0.6.15), and it still duplicates scripts with a fresh clone of IPython, if setuptools is used.

Owner

fperez commented Apr 7, 2011

On Thu, Apr 7, 2011 at 1:17 PM, minrk
reply@reply.github.com
wrote:

No, they are still defined twice (in setup.py and setupbase.py), but only installed once after the fix.  It's pretty pathetic that we have to specify it in both setuptools and non-setuptools ways.  Is it really not possible for setuptools entry points to use existing scripts?

Ah, yes, I missed the point about the definition duplication in the
'if setuptools' block and find_scripts(). I would love it if
setuptools could use existing script definitions, but as far as I know
it does its own thing separately.

There is a cleaner solution, perhaps, but it requires a little bit
more work. Easy work, though: we make a base find_scripts() function
that returns more information; instead of a simple list of pathds, it
returns paths, package names and function entry points, in some
suitable data structure (a dict seems the obvious choice).

And we have find_scripts_distutils and find_scripts_setuptools that
get called in the branches of the "if setuptools" block. These two
call the same base find_scripts(), but they each parse it differently,
to make distutils/setuptools happy.

How does that sound? It's easy to implement.

Cheers,

f

Contributor

rkern commented Apr 7, 2011

Uh, setuptools does work with the regular distutils scripts data just fine.

Owner

ellisonbg commented Apr 7, 2011

Robert,

On windows will setuptools create .bat or .exe files as needed when
scripts are specified in the distutils manner?

Brian

On Thu, Apr 7, 2011 at 2:10 PM, rkern
reply@reply.github.com
wrote:

Uh, setuptools does work with the regular distutils scripts data just fine.

Reply to this email directly or view it on GitHub:
#335 (comment)

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

Owner

minrk commented Apr 7, 2011

Brian,

No, setuptools does not create executables from distutils scripts.

Contributor

rkern commented Apr 7, 2011

No, it doesn't.

Owner

ellisonbg commented Apr 7, 2011

Then this is why they are double listed...

Brian

On Thu, Apr 7, 2011 at 3:59 PM, rkern
reply@reply.github.com
wrote:

No, it doesn't.

Reply to this email directly or view it on GitHub:
#335 (comment)

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

Owner

minrk commented Apr 7, 2011

Even better, after checking this on Windows:

setuptools installs entry points as 'ipython.exe' and 'ipython-script.py', so currently we are installing 3 of every script on Windows since the names don't collide: 'ipython', 'ipython.exe', and 'ipython-script.py'.

Awesome.

Owner

ellisonbg commented Apr 7, 2011

setuptools installs entry points as 'ipython.exe' and 'ipython-script.py', so currently we are installing 3 of every script on Windows since the names don't collide: 'ipython', 'ipython.exe', and 'ipython-script.py'.

Awesome.

Yes, truly...

The problem with Windows is that the hacks that get the scripts
installed when we build the MSI installer and super fragile. At times
setuptools has been the only way of getting these scripts installed
properly on Windows.

Brian

Reply to this email directly or view it on GitHub:
#335 (comment)

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

minrk closed this in 87cd993 Apr 8, 2011

@mattvonrocketstein mattvonrocketstein pushed a commit to mattvonrocketstein/ipython that referenced this issue Nov 3, 2014

@minrk minrk prevent duplicate script installs in setuptools
closes gh-335
8359bb2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment