New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Can't install astropy automatically as a dependency #3541
Comments
Hmm, this is really a thing with astropy-helpers so I'll open a corresponding issue there. Do poppy and/or webbpsf use astropy-helpers? Otherwise this shouldn't happen (not that it should at all, but it especially shouldn't happen in those cases). |
What happens if astropy is already installed first? |
Also do you have astropy-helpers installed in your environment? If so, make sure to uninstall it first, what ever else you do. |
If I install the latest astropy-helpers, then install poppy into an environment that doesn't have astropy, then I get
which is slightly different, but similar problem. Obviously this shouldn't happen. It doesn't seem to be an issue at all if I uninstall astropy-helpers first. |
Oh that's weird. Actually I uninstalled astropy-helpers and then I got the same exception you reported. But there weren't any issues at all before I installed astropy-helpers. And yet after installing and and subsequently uninstalling it it started this... |
I'm also getting
this is unrelated but a bit odd. It might be that when the source file are unpacked it isn't preserving the timestamps, and unpacking them in such an order. I think that warning should just be squelched entirely. |
Starting from a fresh Python 2.7 virtualenv, I install numpy with Not much installed:
I made sure to cd to my home directory before trying to install.. since where I was running it before (~/dev/webbpsf) had astropy_helpers in that folder (and might have been imported at some point in the build process). Didn't seem to make a difference. |
What if astropy is manually installed first too? |
That works. If I do |
I think this might have something to do with the new astropy-helpers command hooks interfering with the build of other packages even if they don't use astropy-helpers. Or maybe even something more generally when multiple packages are using astropy-helpers during the same install session and from within the same process. There may be some globals that don't get reset. |
I think I can reproduce this now, but it's weird that it was working for a bit. |
I think I know what's probably happening here, in particular. I don't think it has anything specifically to do with the command-hooks, at least. It seems that poppy has astropy listed in setup_requires, which is fine. And it's also using astropy_helpers, albeit a previous version. Depending on the exact order in which things happen, if the astropy_helpers shipped with poppy first gets loaded that gets used first. However, when the install of poppy then tries to get astropy, astropy's install runs (via easy_install, since pip doesn't properly handle ordering for setup_requires yet), and when it runs it sees that astropy_helpers is already available on the path, rather then use the newer version of astropy_helpers shipped with astropy it goes on using the old one used by poppy. Furthermore, the old one already has some global variables set in it that are associated with poppy, resulting in further havoc (though it might break later for other reasons as well). I think I can make a workaround to this and make a patch release of astropy-helpers that will address it without requiring any changes in any other packages affected by this issue, since the astropy-helpers auto-upgrade is working fine in this case. It needs to allow for multiple different versions of astropy-helpers to be swapped in and out when installing multiple packages within the same Python session. I think that the sandboxing already provided by |
Wow, that's pretty arcane. I'm glad you figured it out! We'll have to update astropy_helpers for POPPY and WebbPSF in our next point releases. Can you comment here or send me an email when that feature lands? |
Yeah it would be worth updating anyways. I believe there was a package-template update that coincided with the astropy 1.0 release. But in principle it shouldn't be necessary to switch to it (otherwise we of course would have given people more time!) In the meantime the simplest workaround is to install astropy first. When the patch to astropy-helpers lands it should just work. |
Is there a low-traffic announcement list for astropy helpers and package template changes? It seems like I just updated those recently, but evidently not recently enough. |
@josePhoenix That sounds like a good idea. My preference would be to have an all encompassing list that includes updates to affiliated packages and Astropy itself. But what do you think? Either way I think there should definitely be a better channel for that. |
That would be fine too. If there were an announce-only list that's ≤1 email/week of things of interest to affiliated package maintainers, I'd read it. (Minor wrinkle: ST's new mail filtering system means we can't participate in Google Groups under our work emails. Bah.) |
Didn't quite get a chance to fix this today. |
@josePhoenix I think under the present circumstances there is a chicken/egg problem that makes it almost impossible to fix this entirely, at least not without egregious monkeying around. Would it be possible to just put out patch releases for webbpsf and poppy that include an upgraded astropy-helpers? With the changes I have been able to make this won't be a problem in the future. But in the meantime it's really hard since it needs a package to be able to replace itself with a newer version of the package. Except only the newer version of the package knows how to actually do that :/ Alternatively (or additionally) I can make a patch release of Astropy that is able to handle this. |
I guess the immediate course of action depends on what's higher priority. |
Ideally, Astropy helpers would function in such a way that it doesn't have to make any assumptions about what other packages are doing. We can certainly put out a bugfix release of POPPY and WebbPSF (cc @cslocum, but this probably doesn't affect the stable Ureka release timeline?). It sounds like a bugfix release of Astropy is in order as well? |
That would be fine except when two different packages are depending on two different versions of astropy-helpers that need to be used simultaneously in the same Python process because the installer for one of those packages is trying to also bootstrap the other package :) This is a problem far beyond just astropy-helpers. |
To clarify, also, if I release a new patch release of Astropy the issue will be essentially resolved, as neither WebbPSF or POPPY seem to be pinning their |
Didn't really intend to close this, since it won't really be fixed until the new releases are made. |
Confirmed that this is now fixed with the just released Astropy v1.0.1. @josePhoenix, could you confirm that it works for you? |
(I do seem to be having some problems running |
If I pre-install NumPy, the issue is fixed with no additional changes in POPPY. We'll put out patch releases to POPPY and WebbPSF requiring As far as the second issue, being unable to pre-install NumPy, it does indeed seem to be something to do with astropy. If I remove all the setup-related That is, at least, a separate issue. I could open an issue on the package template or helpers, whichever you think is more appropriate. |
Perhaps, though I can't think of any reason it would interfere with Numpy installation. Numpy does some pretty wild stuff in its setup too though. |
I've traced the problems with Numpy to this: https://github.com/numpy/numpy/blob/master/numpy/distutils/core.py#L92 There's several levels of wrong with this, not all of which are Numpy's fault. A lot of it goes to the heart of the badness of distutils. Should be interesting to see if I can come up with a workaround... |
See here for an explanation I left on how/why this is affecting Numpy. I believe this may actually be fixed if using the latest version of setuptools, but I need to confirm this... |
Confirmed--the issue with Numpy should be worked around if you have setuptools >= 10.0 (although in practice >= 11.0 since astropy_helpers is broken with setuptools 10.x, as reported in #3576). |
… installs another package that uses astropy-helpers the second package's astropy-helpers can override the first if they are different/incompatible versions. This should address astropy#3541. In order for this to work there will have be patch releases for both the 0.4.x and 1.0.x versions. Then, through use of the auto-upgrade process, this should all work. Without auto-upgrade, of course, it will still break under the old versions of astropy-helpers. No way around that. Fortunately, most of the time the only context where auto-upgrade is disabled is for OS package builds, where build requirements will already be installed anyways (and hence no packages installing other packages).
Since Astropy 1.0 was released, I've been seeing problems installing POPPY and WebbPSF with pip. (It's possible this was true with the preceding Astropy release as well. The last time I went through this process of installing "from scratch" was Fall 2014.) They both refer to Astropy through
setup_requires
andinstall_requires
in theirsetup.py
, and the error seems to implicate astropy in both cases. (It could be something to do with the helpers/package template too, but I don't know enough to say.)The relevant error:
distutils.errors.DistutilsError: Setup script exited with error: [Errno 2] No such file or directory: 'poppy/_compiler.c'
(or likewise with s/poppy/webbpsf/)Output from
pip install poppy
Output from
pip install webbpsf
The text was updated successfully, but these errors were encountered: