Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pip install -U . fails when there's a UNIX domain socket in the current directory #5306

Closed
gvanrossum opened this issue Apr 20, 2018 · 5 comments · Fixed by #6844

Comments

@gvanrossum
Copy link

@gvanrossum gvanrossum commented Apr 20, 2018

  • Pip version: 10.0.1
  • Python version: 3.6
  • Operating system: macOS Sierra

Description:

It seems pip 10.0.1, when installing from a directory, makes a copy of that directory before doing anything else (?). This causes problems for mypy developers. We have a daemon process that creates a UNIX domain socket named dmypy.sock in the current directory. When the mypy daemon is running in the mypy directory, and we try to install from there, this copy fails with the following error:

$ pip3 install -U .
Processing /Users/guido/src/mypy
Could not install packages due to an EnvironmentError: [('/Users/guido/src/mypy/dmypy.sock', '/private/var/folders/63/czkyq6090dd0t157zhx54xvhrdlybt/T/pip-req-build-yucol_r3/dmypy.sock', "[Errno 102] Operation not supported on socket: '/Users/guido/src/mypy/dmypy.sock'")]

Using -v we get the following traceback:

$ pip3 install -U . -v
Created temporary directory: /private/var/folders/63/czkyq6090dd0t157zhx54xvhrdlybt/T/pip-ephem-wheel-cache-z05l_dty
Created temporary directory: /private/var/folders/63/czkyq6090dd0t157zhx54xvhrdlybt/T/pip-install-pdgnid0m
Processing /Users/guido/src/mypy
  Created temporary directory: /private/var/folders/63/czkyq6090dd0t157zhx54xvhrdlybt/T/pip-req-build-cykzr90d
Could not install packages due to an EnvironmentError.
Traceback (most recent call last):
  File "/Users/guido/v36/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 291, in run
    resolver.resolve(requirement_set)
  File "/Users/guido/v36/lib/python3.6/site-packages/pip/_internal/resolve.py", line 103, in resolve
    self._resolve_one(requirement_set, req)
  File "/Users/guido/v36/lib/python3.6/site-packages/pip/_internal/resolve.py", line 257, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/Users/guido/v36/lib/python3.6/site-packages/pip/_internal/resolve.py", line 210, in _get_abstract_dist_for
    self.require_hashes
  File "/Users/guido/v36/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 310, in prepare_linked_requirement
    progress_bar=self.progress_bar
  File "/Users/guido/v36/lib/python3.6/site-packages/pip/_internal/download.py", line 824, in unpack_url
    unpack_file_url(link, location, download_dir, hashes=hashes)
  File "/Users/guido/v36/lib/python3.6/site-packages/pip/_internal/download.py", line 700, in unpack_file_url
    shutil.copytree(link_path, location, symlinks=True)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/shutil.py", line 359, in copytree
    raise Error(errors)
shutil.Error: [('/Users/guido/src/mypy/dmypy.sock', '/private/var/folders/63/czkyq6090dd0t157zhx54xvhrdlybt/T/pip-req-build-cykzr90d/dmypy.sock', "[Errno 102] Operation not supported on socket: '/Users/guido/src/mypy/dmypy.sock'")]
Cleaning up...

A simple workaround is to stop the daemon (or if it's been killed, rm dmypy.sock).

(First reported for mypy: python/mypy#4945, for @ilevkivsky.)

@pradyunsg

This comment has been minimized.

Copy link
Member

@pradyunsg pradyunsg commented Apr 21, 2018

A fix for this would be that pip tries to be smarter about what it's copying.

Honestly though, I have a feeling that adding support for a .pipignore will let people work around edge cases and that would be cleaner than adding support for all kinds of edge cases in pip.

Unrelated: pip will always reinstall from a local directory -- pip install . should just work for you. :)

@gvanrossum

This comment has been minimized.

Copy link
Author

@gvanrossum gvanrossum commented Apr 22, 2018

It's also arguably a bug in shutil.copytree(), which contains this comment:

# XXX What about other special files? (sockets, devices...)
@gvanrossum

This comment has been minimized.

Copy link
Author

@gvanrossum gvanrossum commented Apr 22, 2018

(And thanks for the tip about not needing -U! I learned something. :-)

@piotr-dobrogost

This comment has been minimized.

Copy link

@piotr-dobrogost piotr-dobrogost commented Apr 23, 2018

For some background see issue #2195pip install of a directory is super slow.
Problem with source directory containing sockets was raised in issue #2974Wheel command does not work if the directory contains a socket.

@pradyunsg

This comment has been minimized.

Copy link
Member

@pradyunsg pradyunsg commented Aug 21, 2019

Thanks to @chrahunt and @cjerdonek, there's a Python-3-only fix for this which skips copying socket files when copying the source directory. 🎉

@lock lock bot added the S: auto-locked label Sep 20, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Sep 20, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
4 participants
You can’t perform that action at this time.