Using --target with --editable results in "internal" error #562

Open
piotr-dobrogost opened this Issue Jun 3, 2012 · 19 comments

Projects

None yet
@piotr-dobrogost

It seems like --target is not supported when using --editable at the same time. It would be nice to have it working. Meanwhile there should be some better diagnostic as

c:\>pip install -t z:\1 -e git+https://github.com/kennethreitz/requests.git#egg=requests
results in

Obtaining requests from git+https://github.com/kennethreitz/requests.git#egg=requests
  Updating c:\python\virtualenv\test\src\requests clone
  Running setup.py egg_info for package requests

Downloading/unpacking certifi>=0.0.7 (from requests)
  Running setup.py egg_info for package certifi

Downloading/unpacking oauthlib>=0.1.0,<0.2.0 (from requests)
  Running setup.py egg_info for package oauthlib

Downloading/unpacking chardet>=1.0.0 (from requests)
  Running setup.py egg_info for package chardet

Downloading/unpacking rsa (from oauthlib>=0.1.0,<0.2.0->requests)
  Running setup.py egg_info for package rsa

    warning: no files found matching 'README'
Downloading/unpacking pyasn1>=0.0.13 (from rsa->oauthlib>=0.1.0,<0.2.0->requests)
  Running setup.py egg_info for package pyasn1

Installing collected packages: requests, certifi, oauthlib, chardet, rsa, pyasn1
  Running setup.py develop for requests
    usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
       or: -c --help [cmd1 cmd2 ...]
       or: -c --help-commands
       or: -c cmd --help

    error: option --home not recognized
    Complete output from command c:\python\virtualenv\test\Scripts\python.exe -c "import setuptools; __file__='c:\\python\\virtualenv\\test\\src\\requests\\setup.py'; exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" develop --no-deps --home=c:\users\piotr\appdata\local\temp\tmp3abskl:
    usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]

   or: -c --help [cmd1 cmd2 ...]

   or: -c --help-commands

   or: -c cmd --help



error: option --home not recognized

----------------------------------------
Command c:\python\virtualenv\test\Scripts\python.exe -c "import setuptools; __file__='c:\\python\\virtualenv\\test\\src\\requests\\setup.py'; exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" develop --no-deps --home=c:\users\piotr\appdata\local\temp\tmp3abskl failed with error code 1 in c:\python\virtualenv\test\src\requests
Storing complete log in C:\Users\Piotr\AppData\Roaming\pip\pip.log
@sprt
sprt commented Jan 19, 2013

Experiencing this error with pip install -t dir -e git+git://github.com/shazow/urllib3.git@f088037#egg=urllib3 as well.

@henzk
henzk commented Feb 5, 2013

i also ran into this error right now.
This seems to happen on any package if --target and --editable are combined:
pip calls setup.py develop --home ..., but --home is only applicable with setup.py install.

So, in the end i found out that using the --src option with editable packages instead of --target does what i want.

Maybe --target should have the same effect as --src when --editable is specified or it could present the user with an error message and maybe point the user to --src.

@schlamar
Contributor
schlamar commented Apr 3, 2013

Maybe --target should have the same effect as --src when --editable is specified

IMO the expected behavior would create / update the easy_install.pth in the target directory.

@YorickPeterse

I just experienced the same issue. As suggested by @jezdez you can work around this by doing the following (without using --editable that is):

git+ssh://user@github.com/some-user/some-repo.git#egg=Foo
@tima
tima commented Jun 27, 2013

I'm seeing a similar issue here myself:

Cleaning up...
Exception:
Traceback (most recent call last):
  File "/efs/dev/bti/pip/1.3.1-build001/install/exec/2.7/lib/python/pip-1.3.1-py2.7.egg/pip/basecommand.py", line 139, in main
  status = self.run(options, args)
  File "/efs/dev/bti/pip/1.3.1-build001/install/exec/2.7/lib/python/pip-1.3.1-py2.7.egg/pip/commands/install.py", line 291, in run
    for item in os.listdir(lib_dir):
OSError: [Errno 2] No such file or directory: '/tmp/tmppjdGHI/lib/python'

line 290 in pip/commands/install.py is:

     lib_dir = home_lib(temp_target_dir)

From what I can trace, pip has already cleaned this path up in pip/req.py line 1194 where it removes the temporary source.

     requirement.remove_temporary_source()

I can see leaving that clean-up in there as is but wrapping it with an exists or try block so the install can continue on. Does anyone see that as a problem?

@YorickPeterse

@tima I had the same problem with the lib directory. Pip HEAD supposedly fixed this but at least on CentOS the issue still persists. In this particular case there is a lib64 directory instead of a lib one.

@beaumartinez

I have the same issue using --target (but not --editable).

Here's my traceback—

Exception:
Traceback (most recent call last):
  File "/Users/beaum/homebrew/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg/pip/basecommand.py", line 139, in main
    status = self.run(options, args)
  File "/Users/beaum/homebrew/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg/pip/commands/install.py", line 294, in run
    for item in os.listdir(lib_dir):
OSError: [Errno 2] No such file or directory: '/var/folders/00/1hyxr000h01000cxqpysvccm0063vq/T/tmpc_E_Bl/lib/python'
@satels
satels commented Nov 29, 2013

+1,

Downloading PyYAML-3.10.tar.gz (241kB): 241kB downloaded
  Running setup.py egg_info for package pyyaml
    skipping 'ext/_yaml.c' Cython extension (up-to-date)
Installing collected packages: krcore, sympy, pyparsing, pyyaml
  Running setup.py develop for krcore
    usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
       or: -c --help [cmd1 cmd2 ...]
       or: -c --help-commands
       or: -c cmd --help
    error: option --home not recognized
    Complete output from command /usr/bin/python -c "import setuptools; __file__='/tmp/krapp/src/krcore/setup.py'; exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" develop --no-deps --home=/tmp/tmpvKaRYp:
    usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: -c --help [cmd1 cmd2 ...]
   or: -c --help-commands
   or: -c cmd --help
error: option --home not recognized
----------------------------------------
Cleaning up...
Command /usr/bin/python -c "import setuptools; __file__='/tmp/krapp/src/krcore/setup.py'; exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" develop --no-deps --home=/tmp/tmpvKaRYp failed with error code 1 in /tmp/krapp/src/krcore
@radzhome
radzhome commented Aug 5, 2014

I get the 'error: option --home not recognized' when using --target with -r (--requirements)

@tdavis
tdavis commented Apr 1, 2015

👍

@eramirem
eramirem commented Aug 4, 2015

Following @YorickPeterse and @jezdez workaround caused:
error: must supply either home or prefix/exec-prefix -- not both

--editable and --target options don't work together

@asmodehn
asmodehn commented Apr 4, 2016

I have been hitting the same issue...
I am trying to use pip to install python packages in a custom location (not a virtualenv) and I need one of them (the one I'm working on) to be editable.

This is likely related to pypa/setuptools#392

And since one pip command can trigger both setup.py develop (for editable pkg) and setup.py install (dependencies), the only (very ugly) workaround that I can think about is to issue 2 commands :

  • one for pkg with --no-deps
  • one for dependencies only (no simple way here...)

It would be much cleaner if there was no need to change other pip command line options whether --editable is passed or not.
So I guess there are two ways to get this working :

  • making setuptools support --home ie. fixing pypa/setuptools#392. probably tricky, reading the details of that issue ?
  • having pip abstract setuptools details and implement --target behavior differently depending if it calls setup.py develop or setup.py install. maybe simpler ?
@pfmoore
Member
pfmoore commented Apr 4, 2016

having pip abstract setuptools details and implement --target behavior differently depending if it calls setup.py develop or setup.py install. maybe simpler ?

The biggest difficulty with this option is that pip is working to abstract away the details of the build system (setuptools) so special-case workarounds for setuptools issues work against that goal.

@asmodehn
asmodehn commented Apr 5, 2016

In the end I managed to do what I wanted, using --prefix, and without using any custom --install-option, so I can finally use the same options whether I pass --editable or not.

However I had to somehow adapt my existing (debian) layout to match pip default (site-packages), since there is no pip option to specify the layout...

Maybe a feature to consider adding?

@piotr-dobrogost
piotr-dobrogost commented May 8, 2016 edited

@xavfernandez

This needs --target option label.

@markfink
markfink commented Aug 28, 2016 edited

could someone please share what the best workaround is to use both -e and -t options? Are you suggesting to use distutils since it supports the '--home' option?

@markfink

any news?

@asmodehn
asmodehn commented Feb 1, 2017 edited

I am still using --editable with --prefix (instead of --target) which does the job for me. But I am stuck at pip<9.0.0 because of the difference between --prefix and --target. more details in #4243

@thisch
thisch commented Mar 15, 2017

if you want to use the --prefix option instead of --target you need to have the PYTHONPATH (pointing to the to-be-created site-packages dir) set correctly before you can call pip -t . --prefix myprefix. Is there an elegant way to circumvent this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment