Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using --target with --editable results in "internal" error #562

Closed
piotr-dobrogost opened this issue Jun 3, 2012 · 22 comments
Closed

Using --target with --editable results in "internal" error #562

piotr-dobrogost opened this issue Jun 3, 2012 · 22 comments

Comments

@piotr-dobrogost
Copy link

@piotr-dobrogost piotr-dobrogost commented Jun 3, 2012

It seems like --target is not supported when using --editable at the same time. It would be nice to have it working. Meanwhile there should be some better diagnostic as

c:\>pip install -t z:\1 -e git+https://github.com/kennethreitz/requests.git#egg=requests
results in

Obtaining requests from git+https://github.com/kennethreitz/requests.git#egg=requests
  Updating c:\python\virtualenv\test\src\requests clone
  Running setup.py egg_info for package requests

Downloading/unpacking certifi>=0.0.7 (from requests)
  Running setup.py egg_info for package certifi

Downloading/unpacking oauthlib>=0.1.0,<0.2.0 (from requests)
  Running setup.py egg_info for package oauthlib

Downloading/unpacking chardet>=1.0.0 (from requests)
  Running setup.py egg_info for package chardet

Downloading/unpacking rsa (from oauthlib>=0.1.0,<0.2.0->requests)
  Running setup.py egg_info for package rsa

    warning: no files found matching 'README'
Downloading/unpacking pyasn1>=0.0.13 (from rsa->oauthlib>=0.1.0,<0.2.0->requests)
  Running setup.py egg_info for package pyasn1

Installing collected packages: requests, certifi, oauthlib, chardet, rsa, pyasn1
  Running setup.py develop for requests
    usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
       or: -c --help [cmd1 cmd2 ...]
       or: -c --help-commands
       or: -c cmd --help

    error: option --home not recognized
    Complete output from command c:\python\virtualenv\test\Scripts\python.exe -c "import setuptools; __file__='c:\\python\\virtualenv\\test\\src\\requests\\setup.py'; exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" develop --no-deps --home=c:\users\piotr\appdata\local\temp\tmp3abskl:
    usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]

   or: -c --help [cmd1 cmd2 ...]

   or: -c --help-commands

   or: -c cmd --help



error: option --home not recognized

----------------------------------------
Command c:\python\virtualenv\test\Scripts\python.exe -c "import setuptools; __file__='c:\\python\\virtualenv\\test\\src\\requests\\setup.py'; exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" develop --no-deps --home=c:\users\piotr\appdata\local\temp\tmp3abskl failed with error code 1 in c:\python\virtualenv\test\src\requests
Storing complete log in C:\Users\Piotr\AppData\Roaming\pip\pip.log
@sprt
Copy link

@sprt sprt commented Jan 19, 2013

Experiencing this error with pip install -t dir -e git+git://github.com/shazow/urllib3.git@f088037#egg=urllib3 as well.

@henzk
Copy link

@henzk henzk commented Feb 5, 2013

i also ran into this error right now.
This seems to happen on any package if --target and --editable are combined:
pip calls setup.py develop --home ..., but --home is only applicable with setup.py install.

So, in the end i found out that using the --src option with editable packages instead of --target does what i want.

Maybe --target should have the same effect as --src when --editable is specified or it could present the user with an error message and maybe point the user to --src.

@schlamar
Copy link
Contributor

@schlamar schlamar commented Apr 3, 2013

Maybe --target should have the same effect as --src when --editable is specified

IMO the expected behavior would create / update the easy_install.pth in the target directory.

@YorickPeterse
Copy link

@YorickPeterse YorickPeterse commented Jun 11, 2013

I just experienced the same issue. As suggested by @jezdez you can work around this by doing the following (without using --editable that is):

git+ssh://user@github.com/some-user/some-repo.git#egg=Foo
@tima
Copy link

@tima tima commented Jun 27, 2013

I'm seeing a similar issue here myself:

Cleaning up...
Exception:
Traceback (most recent call last):
  File "/efs/dev/bti/pip/1.3.1-build001/install/exec/2.7/lib/python/pip-1.3.1-py2.7.egg/pip/basecommand.py", line 139, in main
  status = self.run(options, args)
  File "/efs/dev/bti/pip/1.3.1-build001/install/exec/2.7/lib/python/pip-1.3.1-py2.7.egg/pip/commands/install.py", line 291, in run
    for item in os.listdir(lib_dir):
OSError: [Errno 2] No such file or directory: '/tmp/tmppjdGHI/lib/python'

line 290 in pip/commands/install.py is:

     lib_dir = home_lib(temp_target_dir)

From what I can trace, pip has already cleaned this path up in pip/req.py line 1194 where it removes the temporary source.

     requirement.remove_temporary_source()

I can see leaving that clean-up in there as is but wrapping it with an exists or try block so the install can continue on. Does anyone see that as a problem?

@YorickPeterse
Copy link

@YorickPeterse YorickPeterse commented Jun 28, 2013

@tima I had the same problem with the lib directory. Pip HEAD supposedly fixed this but at least on CentOS the issue still persists. In this particular case there is a lib64 directory instead of a lib one.

@beaumartinez
Copy link

@beaumartinez beaumartinez commented Jul 3, 2013

I have the same issue using --target (but not --editable).

Here's my traceback—

Exception:
Traceback (most recent call last):
  File "/Users/beaum/homebrew/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg/pip/basecommand.py", line 139, in main
    status = self.run(options, args)
  File "/Users/beaum/homebrew/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg/pip/commands/install.py", line 294, in run
    for item in os.listdir(lib_dir):
OSError: [Errno 2] No such file or directory: '/var/folders/00/1hyxr000h01000cxqpysvccm0063vq/T/tmpc_E_Bl/lib/python'
@satels
Copy link

@satels satels commented Nov 29, 2013

+1,

Downloading PyYAML-3.10.tar.gz (241kB): 241kB downloaded
  Running setup.py egg_info for package pyyaml
    skipping 'ext/_yaml.c' Cython extension (up-to-date)
Installing collected packages: krcore, sympy, pyparsing, pyyaml
  Running setup.py develop for krcore
    usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
       or: -c --help [cmd1 cmd2 ...]
       or: -c --help-commands
       or: -c cmd --help
    error: option --home not recognized
    Complete output from command /usr/bin/python -c "import setuptools; __file__='/tmp/krapp/src/krcore/setup.py'; exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" develop --no-deps --home=/tmp/tmpvKaRYp:
    usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: -c --help [cmd1 cmd2 ...]
   or: -c --help-commands
   or: -c cmd --help
error: option --home not recognized
----------------------------------------
Cleaning up...
Command /usr/bin/python -c "import setuptools; __file__='/tmp/krapp/src/krcore/setup.py'; exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" develop --no-deps --home=/tmp/tmpvKaRYp failed with error code 1 in /tmp/krapp/src/krcore
@radzhome
Copy link

@radzhome radzhome commented Aug 5, 2014

I get the 'error: option --home not recognized' when using --target with -r (--requirements)

@tdavis
Copy link

@tdavis tdavis commented Apr 1, 2015

👍

@eramirem
Copy link

@eramirem eramirem commented Aug 4, 2015

Following @YorickPeterse and @jezdez workaround caused:
error: must supply either home or prefix/exec-prefix -- not both

--editable and --target options don't work together

@asmodehn
Copy link

@asmodehn asmodehn commented Apr 4, 2016

I have been hitting the same issue...
I am trying to use pip to install python packages in a custom location (not a virtualenv) and I need one of them (the one I'm working on) to be editable.

This is likely related to pypa/setuptools#392

And since one pip command can trigger both setup.py develop (for editable pkg) and setup.py install (dependencies), the only (very ugly) workaround that I can think about is to issue 2 commands :

  • one for pkg with --no-deps
  • one for dependencies only (no simple way here...)

It would be much cleaner if there was no need to change other pip command line options whether --editable is passed or not.
So I guess there are two ways to get this working :

  • making setuptools support --home ie. fixing pypa/setuptools#392. probably tricky, reading the details of that issue ?
  • having pip abstract setuptools details and implement --target behavior differently depending if it calls setup.py develop or setup.py install. maybe simpler ?
@pfmoore
Copy link
Member

@pfmoore pfmoore commented Apr 4, 2016

having pip abstract setuptools details and implement --target behavior differently depending if it calls setup.py develop or setup.py install. maybe simpler ?

The biggest difficulty with this option is that pip is working to abstract away the details of the build system (setuptools) so special-case workarounds for setuptools issues work against that goal.

@asmodehn
Copy link

@asmodehn asmodehn commented Apr 5, 2016

In the end I managed to do what I wanted, using --prefix, and without using any custom --install-option, so I can finally use the same options whether I pass --editable or not.

However I had to somehow adapt my existing (debian) layout to match pip default (site-packages), since there is no pip option to specify the layout...

Maybe a feature to consider adding?

@piotr-dobrogost
Copy link
Author

@piotr-dobrogost piotr-dobrogost commented May 8, 2016

@xavfernandez

This needs --target option label.

@markfink
Copy link

@markfink markfink commented Aug 28, 2016

could someone please share what the best workaround is to use both -e and -t options? Are you suggesting to use distutils since it supports the '--home' option?

@markfink
Copy link

@markfink markfink commented Nov 15, 2016

any news?

@asmodehn
Copy link

@asmodehn asmodehn commented Feb 1, 2017

I am still using --editable with --prefix (instead of --target) which does the job for me. But I am stuck at pip<9.0.0 because of the difference between --prefix and --target. more details in #4243

@thisch
Copy link

@thisch thisch commented Mar 15, 2017

if you want to use the --prefix option instead of --target you need to have the PYTHONPATH (pointing to the to-be-created site-packages dir) set correctly before you can call pip -t . --prefix myprefix. Is there an elegant way to circumvent this?

@dstufft
Copy link
Member

@dstufft dstufft commented Mar 30, 2017

Closing to move a bunch of related issues to a single issue: #4390.

@dstufft dstufft closed this Mar 30, 2017
@blainegarrett
Copy link

@blainegarrett blainegarrett commented Jun 22, 2017

I'd add that this is an issue for Google Appengine developers who need to install their dependencies in a directory in the app root but also need to deploy an editable dependency in a local checkout as part of a QA process. As it sits, the editable would need to be manually symlinked which is cumbersome.

@lock
Copy link

@lock lock bot commented Jun 2, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot added the S: auto-locked label Jun 2, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Jun 2, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet