Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make install directory if it doesn't exist yet #1431

Merged
merged 2 commits into from Mar 21, 2020
Merged

Conversation

hroncok
Copy link
Contributor

@hroncok hroncok commented Jul 20, 2018

Problem: In Fedora, sudo python3 setup.py install installs to /usr/local/lib/pythonX.Y/site-packages. However that directory is not always there as it is not installed together with the python3 package (Fedora packages should not install stuff to /usr/local).

When the directory does not exist an installation attempt fails with:

[Errno 2] No such file or directory: '/usr/local/lib/python3.7/site-packages/test-easy-install-11875.write-test'

Solution: Make the directory if it doesn't exists yet. If user has no permissions, the above error is preserved, yet with root (sudo) it works now.

I realize that this is not usefull outside Fedora's specific setup, yet I don't like downstream only patches and this breaks nothing.

  • Changes have tests - not sure how to do that, as this is fairly Fedora specific
  • News fragment added in changelog.d

Problem: In Fedora, `sudo python3 setup.py install` installs to
/usr/local/lib/pythonX.Y/site-packages. However that directory is
not always there as it is not installed together with the python3
package (Fedora packages should not install stuff to /usr/local).

When the directory does not exist an installation attempt fails with:

  [Errno 2] No such file or directory:
  '/usr/local/lib/python3.7/site-packages/test-easy-install-11875.write-test'

Solution: Make the directory if it doesn't exists yet. If user has
no permissions, the above error is preserved, yet with root (sudo)
it works now.
@hroncok hroncok changed the title Make install directory if it doesn't exist yet WIP: Make install directory if it doesn't exist yet Jul 20, 2018
hroncok added a commit to hroncok/setuptools that referenced this issue Jul 20, 2018
@hroncok
Copy link
Contributor Author

@hroncok hroncok commented Jul 20, 2018

Trying to add tests for this. Locally I fail to run them, so let's see what travis says.

hroncok added a commit to hroncok/setuptools that referenced this issue Jul 20, 2018
@hroncok
Copy link
Contributor Author

@hroncok hroncok commented Jul 20, 2018

9683b77 this seems to pass even without my patch.

@pganssle
Copy link
Member

@pganssle pganssle commented Jul 20, 2018

I'm not really sure we want to support sudo python3.6 setup.py install at all, but if this isn't installed as part of python3, are you even sure its in the PYTHONPATH?

I am -1 on this for the following reasons:

  1. setup.py install should be deprecated in favor of pip install
  2. Using sudo to install things is not even supported in pip - you should use the system package manager for system packages, and pip install --user for user installs, or use virtualenvs.
  3. If we did want to support this, I don't think creating the directory is the right way to do it. If the directory we're trying to install it in doesn't exist, it seems very likely that we're installing it in the wrong place. The "right place" should probably be inferred from the system configuration.

If we're already inferring the right place to install packages from the system configuration (haven't checked), then Fedora probably needs to be fixed to properly advertise its non-standard package deployment location (if, again, Fedora wants to support this very dangerous workflow).

@hroncok
Copy link
Contributor Author

@hroncok hroncok commented Jul 20, 2018

  1. agreed. yet people use this and report this error to us. sudo pip works in this scenario.
  2. agreed. yet people still use sudo here and report problems.
  3. pip supports it. this is the right place form our system configuration.

Fedora properly advertises this at https://fedoraproject.org/wiki/Changes/Making_sudo_pip_safe - this regression was a direct result of this change.

That said, I understand if you won't accept this, yet we'll still apply it downstream, becasue otherwise users are rightfully angry.

@pganssle
Copy link
Member

@pganssle pganssle commented Jul 20, 2018

@jaraco @benoit-pierre At this point we can safely deprecated setup.py install in favor of pip, like we've deprecated upload and register, right?

@pradyunsg @dstufft Thoughts?

@pganssle
Copy link
Member

@pganssle pganssle commented Jul 20, 2018

If we can deprecate setup.py install, then I think that can at least partially help with the "people are complaining that this doesn't work" issue.

That said, if Fedora actually intends for people to deploy things to /usr/lib/python3.6/site-packages, why isn't the solution to create that directory as part of the python3 package?

@hroncok
Copy link
Contributor Author

@hroncok hroncok commented Jul 20, 2018

It has two point of views. We don't support it, but we are trying to prevent sudo pip installers to nuke their systems. Yet they complain we broke their workflow and we try to make it work even when we say: don't do this. We cannot install rpm packaged stuff into /usr/local.

@dstufft
Copy link
Member

@dstufft dstufft commented Jul 20, 2018

This isn't really specific to /usr/local/lib/python3.7/site-packages/, it just so happens that's the directory where Fedora ran into it. I think it makes sense for setuptools to make sure that the directory it's trying to write to exists.

@pganssle
Copy link
Member

@pganssle pganssle commented Jul 20, 2018

@dstufft In what other situations would it come up? I'm not deeply opposed to having setuptools create the directory it's trying to write to, but my worry is that the only reason this would happen is if there's been a horrible mistake (in which case you may end up installing somewhere that won't get picked up - or worse, you'll mess up some user's file system), in which case it makes sense to throw an error instead.

If this only affects people using setuptools in a dangerous way, I'm not sure we should be facilitating this at the - admittedly small - risk of failing to report an actual error condition.

@dstufft
Copy link
Member

@dstufft dstufft commented Jul 20, 2018

@pganssle Not sure, I'm pretty sure that pip has this code though, so I assume it came up in some other context at some point.

@hroncok
Copy link
Contributor Author

@hroncok hroncok commented Jul 20, 2018

pip install creates the directory, yes

@silkentrance
Copy link
Contributor

@silkentrance silkentrance commented Aug 14, 2018

@hroncok have you tried installing pyenv + tox in the various environments, say 2.7.x etc.
Then simply run tox from inside the setuptools clone and you will be able to run the tests locally.

@hroncok
Copy link
Contributor Author

@hroncok hroncok commented Aug 14, 2018

I've used tox, I don't remember what was the problem, will try now.

@hroncok
Copy link
Contributor Author

@hroncok hroncok commented Aug 14, 2018

The tests work for me now. However I'd rather see an approval of action before spending time figuring out how to test this if it gets rejected at the end.

@jaraco
Copy link
Member

@jaraco jaraco commented Sep 16, 2018

At this point we can safely deprecated setup.py install in favor of pip, like we've deprecated upload and register, right?

Correct. setup.py install is only recommended with --single-version-externally-managed. However, setup.py develop is still supported (as pip relies on it for pip install -e), and that might also hit this code path.

@jaraco
Copy link
Member

@jaraco jaraco commented Sep 16, 2018

@hroncok Can you confirm this issue also affects setup.py develop?

@hroncok
Copy link
Contributor Author

@hroncok hroncok commented Sep 16, 2018

# python3 setup.py develop
running develop
error: can't create or remove files in install directory

The following error occurred while trying to add or remove files in the
installation directory:

    [Errno 2] No such file or directory: '/usr/local/lib/python3.7/site-packages/test-easy-install-37.write-test'

The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

    /usr/local/lib/python3.7/site-packages/

This directory does not currently exist.  Please create it and try again, or
choose a different installation directory (using the -d or --install-dir
option).

@jaraco jaraco changed the title WIP: Make install directory if it doesn't exist yet Make install directory if it doesn't exist yet Mar 21, 2020
@jaraco jaraco merged commit 42d0d03 into pypa:master Mar 21, 2020
19 of 21 checks passed
@hroncok hroncok deleted the fedora-makedirs branch Mar 21, 2020
@hroncok
Copy link
Contributor Author

@hroncok hroncok commented Mar 21, 2020

Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants