Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Added #subdirectory tag specify a relative subdirectory inside a repo… #526

Closed
wants to merge 58 commits into from
@niedbalski

Is possible to use a relative path inside the repo to reference a subdirectory the #subdirectory tag is used in the edit mode.

pip install -e https://github.com/niedbalski/test.git#egg=test&subdirectory=subpackage_module

Tests passed.

@pnasrat
Owner

Related to Issue #209

I'd like to see some design discussion with others @carljm and @jezdez before taking the approach of adding another url fragment to handle this, particularly as we're considering other ways of passing build information such as --install-options. However thanks for providing a tested CL for this.

tests/test_vcs_backends.py
@@ -15,6 +15,16 @@ def test_install_editable_from_git_with_https():
result.assert_installed('pip-test-package', with_files=['.git'])
+def test_install_editable_with_subdirectory():
+ """
+ Test installing a package from a repo subdirectory
+ """
+ reset_env()
+ result = run_pip('install', '-e',
+ '%s#egg=pip-test-package#subdirectory=piptestsubpackage' %
+ local_checkout('git+https://github.com/niedbalski/pip-test-package.git'),
+ expect_error=True)
+
@pnasrat Owner
pnasrat added a note

You are not making any assertions in this test. You probably want an result.assert_installed line here. Also I'd rather your test package be named for the test, eg pip-test-subdir-package else it may confuse someone in the future.

Oops. Added assertion to the test :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
pip/req.py
@@ -1303,6 +1315,15 @@ def parse_requirements(filename, finder=None, comes_from=None, options=None):
req = InstallRequirement.from_line(line, comes_from)
yield req
+def has_subdirectory(editable_req):
+ """
+ Search for subdirectory parameter on editable URL
+ Returns False if not found or the subdirectory name if success
+ """
+ match = re.search(r'.*(?:#|#.*?&)subdirectory=([^&]*)', editable_req)
@pnasrat Owner
pnasrat added a note

This feels to me like the logic is perhaps in the wrong place, and we should be handling this in the class not as a module level method.

I'm also not sure if this is not RFC breaking http://tools.ietf.org/html/rfc1808.html

 <scheme>://<net_loc>/<path>;<params>?<query>#<fragment>

Also not sure about this. Reading about multiple url fragments, appears that this should be implementation-dependent.

I think that this works pretty well for adding optional installation parameters.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@niedbalski

Added the commented changes.

@niedbalski

Any ideas about this?

@carljm
Owner

I think the url fragment is right for this, as it's really specifying the location of the package to be installed, not a build option for the installation. This was already discussed a while back on the mailing list.

I think, however, that the syntax when providing multiple bits of info in the fragment should be #subdirectory=foo&egg=bar not #subdirectory=foo#egg=bar.

@pnasrat
Owner

That seems to make sense and is valid and according to http://en.wikipedia.org/wiki/Fragment_identifier others use that approach

   pchar         = unreserved / pct-encoded / sub-delims / ":" / "@"
   query         = *( pchar / "/" / "?" )
   fragment      = *( pchar / "/" / "?" )
   pct-encoded   = "%" HEXDIG HEXDIG
   unreserved    = ALPHA / DIGIT / "-" / "." / "_" / "~"
   reserved      = gen-delims / sub-delims
   gen-delims    = ":" / "/" / "?" / "#" / "[" / "]" / "@"
   sub-delims    = "!" / "$" / "&" / "'" / "(" / ")"
                 / "*" / "+" / "," / ";" / "="
@niedbalski niedbalski Editable URL query string to options, all the options can use a proce…
…ss_option ( i.e. process_egg ) callback to validate the value, refactor and cleanup. tests passed OK
39f3e35
@niedbalski

Please review the diff . according to your request all the query string options are passed in the format #option=foo&egg=bar , the options are passed to the Installable instance as editable_options function argument, also if you want to pre-process a specific parameter ( i.e. egg ) just provide a process_xxx method.

For the subdirectory option this has been added:

+            if 'subdirectory' in self.editable_options:
+                setup_py_path = os.path.join(os.path.dirname(self.setup_py),
+                                                            self.editable_options['subdirectory'], \
+                                                            os.path.basename(self.setup_py))

Also i did some small code cleanup.

@travisbot

This pull request fails (merged 39f3e35 into 6369d53).

@travisbot

This pull request fails (merged b681064 into 6369d53).

@travisbot

This pull request fails (merged dac305d into 6369d53).

@travisbot

This pull request fails (merged 126b1ba into e0db44e).

@travisbot

This pull request fails (merged e720e7f into e0db44e).

@travisbot

This pull request fails (merged fc1abb4 into e0db44e).

@travisbot

This pull request fails (merged ccc38d4 into e0db44e).

@pnasrat
Owner

CI errors are known bad #503, but means everything else is passing with your chage.

pip/req.py
@@ -208,10 +214,20 @@ def run_egg_info(self, force_root_egg_info=False):
logger.indent += 2
try:
script = self._run_setup_py
- script = script.replace('__SETUP_PY__', repr(self.setup_py))
+
+ if 'subdirectory' in self.editable_options:
@carljm Owner
carljm added a note

This logic should simply be embedded within the self.setup_py property; self.setup_py should always point to the setup.py.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
pip/req.py
@@ -1303,48 +1319,103 @@ def parse_requirements(filename, finder=None, comes_from=None, options=None):
req = InstallRequirement.from_line(line, comes_from)
yield req
+def process_subdirectory(value):
+ """
+ Search for subdirectory parameter on editable URL
+ Returns False if not found or the subdirectory name if success
+ """
+ return value
@carljm Owner
carljm added a note

What is the purpose of this function if it simply returns the value it is passed? The behavior does not seem to match the docstring at all.

Removed from code this is just an example.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
pip/req.py
@@ -1303,48 +1319,103 @@ def parse_requirements(filename, finder=None, comes_from=None, options=None):
req = InstallRequirement.from_line(line, comes_from)
yield req
+def process_subdirectory(value):
+ """
+ Search for subdirectory parameter on editable URL
+ Returns False if not found or the subdirectory name if success
+ """
+ return value
+
+def process_egg(req):
@carljm Owner
carljm added a note

this should have a more descriptive name - perhaps egg_basename

Sorry @carljm i think that the idea es to have a map between method process_name == query string attribute name for code maintenance and extension.

What do you think?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@niedbalski

Added changes to include @carljm observations.

tests/test_vcs_backends.py
@@ -14,6 +14,15 @@ def test_install_editable_from_git_with_https():
expect_error=True)
result.assert_installed('pip-test-package', with_files=['.git'])
+def test_install_editable_with_subdirectory():
+ """
+ Test installing a package from a repo subdirectory
+ """
+ reset_env()
+ result = run_pip('install', '-e',
+ '%s#egg=pip-test-subdir-package#subdirectory=piptestsubpackage' %
+ local_checkout('git+https://github.com/niedbalski/pip-test-subdir-package.git'), expect_error=True)
@carljm Owner
carljm added a note

local_checkout and reliance on network resources is problematic - we have it a lot, but I don't want to introduce any more of it, especially since this test is not necessarily tied to git.

Instead, can we put the test package directly in tests/packages and install it editable using the filesystem path?

Not sure about this, for now this works. I will create a new test and i will add this to packages.

@carljm Owner
carljm added a note

Yes, local_checkout does work, it just introduces nondeterministic behavior in the tests (because sometimes they actually get the repo from the network, and sometimes they use a locally-cached version), which makes it hard to get reliably stable test results. So I'm trying to avoid use of it in new tests in favor of having everything local whenever possible.

OK, added as git+file , and tested installation. Test file added to packages.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@travisbot

This pull request passes (merged 785a6ab into e0db44e).

pip/req.py
((29 lines not shown))
+ This method generates a dictionary of the query string
+ parameters contained in a given editable URL.
+ if the process_attribute function exists then is called
+ and the returned value is used.
+ """
+ regexp = re.compile(r"[\?#&](?P<name>[^&=]+)=(?P<value>[^&=]+)")
+ matched = regexp.findall(req)
+
+ if matched:
+ ret = dict()
+ for option in matched:
+ (name, value) = option
+ if name in ret:
+ raise Exception("%s option already defined" % name)
+ try:
+ value = globals()['process_%s' % name](value)
@carljm Owner
carljm added a note

No, this is too much magic. Remove this entire process_* thing and leave options["egg"] as exactly the value found in the url fragment. The stripping of the version part of the egg should happen where options["egg"] is pulled out and used as the requirement name - it's the requirement name that should not have the version part, it's fine if options["egg"] continues to have it. Does that make sense?

Ok, i moved the process_egg method to strip_postfix. BTW i think that the process* method call is still needed for more complex processing logic on the query string parameters ( feature ).

Thanks you.

@carljm Owner
carljm added a note

Hi Jorge,

I'm afraid I don't understand what the process_* function call is needed for. It's not needed for egg, and it's not needed for subdirectory, and those are the only two url-fragment options that pip understands. I don't think we need all this code (especially not code which accesses globals(), generally a bad sign) just because hypothetically we might someday add more url fragment options that might use it.

I understand your point. I removed the process_x call, but i think that if you are providing dynamic query string parameters , in the future you will end with a lot of if-else code for parameter processing vs. a simple function call to a developer-specific function.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@travisbot

This pull request passes (merged b6c7e80 into e0db44e).

@travisbot

This pull request fails (merged 6334276 into e0db44e).

@niedbalski

Please review the diff , maybe now can be merged :)

@niedbalski

Hi guys, did you reviewed this patch ?

@pnasrat
Owner

I'm in the process of merging a large change atm for Issue #511 we've done 2 of the 5 part plan https://gist.github.com/2822510

Currently your pull request isn't able to be merged automatically. I'd like to clear the 3 further patches from #511 then I'll ping this issue to make sure you've got it on top of current develop and fixed any merge conflicts, then review and merge.

@niedbalski

Perfect. Ping me back when you are ok with the #511

@niedbalski

Hello Paul,

I merged pending conflicts from upstream.

@travisbot

This pull request fails (merged d9abc60 into ae867db).

@pnasrat
Owner

Lots of failures in the areas you are changing - can you check the tests pass locally. See the Travis build fail link for details.

@pnasrat
Owner

Ping on checking failures.

orutherfurd and others added some commits
@orutherfurd orutherfurd fix compatability w/3.3 & 2.7.4 & bazaar (#552)
7b9d788
@orutherfurd orutherfurd non_hierarchical was dropped at the same time as uses_fragment (#552)
11f1fe7
lepture fix on git repo. if a repo contains submodules, checkout submodules
f701a9d
lepture fix #533 . quite and recursive on submodule checkout
1686520
@qwcode qwcode reset_env option to add a patch to sitecustomize.py
ca97a11
@qwcode qwcode not expecting errors in these tests. let them pass errors.
997f441
@qwcode qwcode make sure sitecustomize.py doesn't grow in the FastTestPipEnvironment
f571f19
@carljm carljm Update AUTHORS and changelog for git submodules.
5ade518
@fin fin test submodule support
2390c95
@fin fin fix python2.5 syntax error
fd4cd1e
@fin fin remove "git submodule foreach" call for backwards compatibility; expe…
…ct errors in pip install because old versions of git write senseless things to stderr
58b950b
unknown Updated basecommand.py such that writing to the log file doesn't fail…
…. If

permission is denied for writing in the specified log file the message
will be written to a temporary one.
027e0d8
@carljm carljm Update changelog.
dc62187
@msabramo msabramo Make magic use pkg_resources.iter_entry_points instead of relying on
egg_info.iter_entry_points, which might not be there if someone
redefines egg_info (like the setup.py for pyobjc-core does)

Fixes GH-11
e931c6a
@qwcode qwcode don't --user install in --system-site-packages virtualenvs with conflict
c1847ec
@msabramo msabramo Add HackedEggInfo test for the the fix for GH-11.
4da91bc
@qwcode qwcode raise InstallationError when UninstallPathSet has no paths
9218ece
@qwcode qwcode use nose.tools.assert_raises
6827d04
@qwcode qwcode generate error in test same way as pip.req
9add29c
@pnasrat pnasrat Fix failing bzr vcs test
c318660
@pnasrat pnasrat Fix failng windows tests.
path_to_url uses os.path.normcase that breaks this test on windows.

Mocked out to use posix paths.
b7d4341
@pnasrat pnasrat Prevent assertion error from scripttest on windows.
We're seeing failure from the helpers due to CRLF messages when we use
git.
9bb12b5
@pnasrat pnasrat Skip submodule testing on windows.
Blocking release, the tests are very path orientated this needs fixup so
added TODO and skip for now.
eaa891b
@pnasrat pnasrat Fix failing windows test.
I spent a good while trying to debug this. Got working avoiding
local_checkout but want to fix correctly. However this will unblock
release.
328600d
@pnasrat pnasrat Fix missing import in test.
2247a2f
@pnasrat pnasrat Fix incorrect mocking.
46e3217
@niedbalski niedbalski [pip/tests] added subpackage to _create_test_package method, fixed te…
…sts for support subdirectory
5c143aa
@niedbalski niedbalski [pip/req.py] modified options and editable_options validation
726fa13
@travisbot

This pull request passes (merged 726fa13 into ae867db).

@niedbalski

Hello Paul,

Everything has been merged and tests are passing , merge ? Good Luck!,

@pnasrat
Owner

I'm away for a wedding this weekend - not forgotten but I may not be able to get to for a few days

@niedbalski

Hello @pnasrat did you reviewed this patchset?

@pnasrat
Owner

Sorry just been busy - it's not automergable and it looks like it has a lot of unrelated stuff in it from when you synced and merged with develop. Can you ensure the differences are just your patch, once it's clean to apply I'll take another look.

@niedbalski

OK @pnasrat i removed the conflicts and updated from upstream per your request, i hope you can merge/and close the pr. regards.

@travisbot

This pull request fails (merged d2552f0 into c6789f6).

@pnasrat
Owner

This still looks a little off on eyeballing -I'm going to have to manually merge, probably not got time until the weekend.

@pnasrat
Owner

I looked at merging this and the number of commits Jul 3 from others concerns me https://github.com/pypa/pip/pull/526/commits

As does changes that have been made by others in your diff such as tests/test_user_site.py

I tried to interactively rebase ontop of development but there were a lot of conflicts then. I don't want to screw up our git history inadvertently by merging this.

You may need to build a clean branch on top of develop - add your own changes and ensure it's clean and tests run then file a new PR.

@noirbizarre noirbizarre referenced this pull request in mapnik/mapnik
Closed

Python setuptools/distutils packaging #1455

@edevil

This seems a simple and useful change. Is anyone looking at this?

@Flimm

+1

@niedbalski

Cleaning this code and re-creating a new PR.

@niedbalski niedbalski closed this
@Julian

@niedbalski I don't see a new PR. Still working on this?

@rutsky

+1

I'm interested in this feature too.

@yulis

+1
Really need this

@niedbalski

Hello Guys.

I created another PR #1082 for this feature with the #subdirectory=foo editable option.

Regards.

@brunogama

+1 for this feature
really want to put pydmtx-wrappers setup into my pip-requiremets.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Apr 3, 2011
  1. @ptone

    resolve symlinks in local files

    ptone authored
    the tar libraries choke when checking valid
    filenames because they don't resolve symlinks
Commits on Jul 16, 2011
  1. @jezdez

    Merge branch 'hotfix/1.0.2'

    jezdez authored
Commits on Oct 23, 2011
  1. @carljm

    Merge pull request #245 from ptone/find-links-symlinks

    carljm authored
    resolve symlinks in local files
Commits on Feb 16, 2012
  1. @jezdez

    Merge branch 'release/1.1'

    jezdez authored
Commits on May 9, 2012
  1. @niedbalski

    Added #subdirectory tag specify a relative subdirectory inside a repo…

    niedbalski authored
    … were setup.py lives. I.E. #subdirectory=subpkg_pkg/
  2. @niedbalski
Commits on May 10, 2012
  1. @niedbalski
  2. @niedbalski

    -a

    niedbalski authored
  3. @niedbalski
  4. @niedbalski
Commits on May 14, 2012
  1. @niedbalski
  2. @niedbalski
Commits on May 23, 2012
  1. @niedbalski

    Editable URL query string to options, all the options can use a proce…

    niedbalski authored
    …ss_option ( i.e. process_egg ) callback to validate the value, refactor and cleanup. tests passed OK
  2. @niedbalski
  3. @niedbalski
  4. @niedbalski
  5. @niedbalski
  6. @niedbalski
  7. @niedbalski
  8. @niedbalski
  9. @niedbalski
  10. @niedbalski
  11. @niedbalski
  12. @niedbalski
  13. @niedbalski
Commits on May 24, 2012
  1. @niedbalski

    Removed process_xxx call

    niedbalski authored
Commits on May 28, 2012
Commits on Jun 12, 2012
  1. @niedbalski
Commits on Jul 3, 2012
  1. @orutherfurd @niedbalski

    fix compatability w/3.3 & 2.7.4 & bazaar (#552)

    orutherfurd authored niedbalski committed
  2. @orutherfurd @niedbalski
  3. @niedbalski

    fix on git repo. if a repo contains submodules, checkout submodules

    lepture authored niedbalski committed
  4. @niedbalski

    fix #533 . quite and recursive on submodule checkout

    lepture authored niedbalski committed
  5. @qwcode @niedbalski

    reset_env option to add a patch to sitecustomize.py

    qwcode authored niedbalski committed
  6. @qwcode @niedbalski
  7. @qwcode @niedbalski
  8. @carljm @niedbalski

    Update AUTHORS and changelog for git submodules.

    carljm authored niedbalski committed
  9. @fin @niedbalski

    test submodule support

    fin authored niedbalski committed
  10. @fin @niedbalski

    fix python2.5 syntax error

    fin authored niedbalski committed
  11. @fin @niedbalski

    remove "git submodule foreach" call for backwards compatibility; expe…

    fin authored niedbalski committed
    …ct errors in pip install because old versions of git write senseless things to stderr
  12. @niedbalski

    Updated basecommand.py such that writing to the log file doesn't fail…

    unknown authored niedbalski committed
    …. If
    
    permission is denied for writing in the specified log file the message
    will be written to a temporary one.
  13. @carljm @niedbalski

    Update changelog.

    carljm authored niedbalski committed
  14. @msabramo @niedbalski

    Make magic use pkg_resources.iter_entry_points instead of relying on

    msabramo authored niedbalski committed
    egg_info.iter_entry_points, which might not be there if someone
    redefines egg_info (like the setup.py for pyobjc-core does)
    
    Fixes GH-11
  15. @qwcode @niedbalski
  16. @msabramo @niedbalski

    Add HackedEggInfo test for the the fix for GH-11.

    msabramo authored niedbalski committed
  17. @qwcode @niedbalski
  18. @qwcode @niedbalski

    use nose.tools.assert_raises

    qwcode authored niedbalski committed
  19. @qwcode @niedbalski

    generate error in test same way as pip.req

    qwcode authored niedbalski committed
  20. @pnasrat @niedbalski

    Fix failing bzr vcs test

    pnasrat authored niedbalski committed
  21. @pnasrat @niedbalski

    Fix failng windows tests.

    pnasrat authored niedbalski committed
    path_to_url uses os.path.normcase that breaks this test on windows.
    
    Mocked out to use posix paths.
  22. @pnasrat @niedbalski

    Prevent assertion error from scripttest on windows.

    pnasrat authored niedbalski committed
    We're seeing failure from the helpers due to CRLF messages when we use
    git.
  23. @pnasrat @niedbalski

    Skip submodule testing on windows.

    pnasrat authored niedbalski committed
    Blocking release, the tests are very path orientated this needs fixup so
    added TODO and skip for now.
  24. @pnasrat @niedbalski

    Fix failing windows test.

    pnasrat authored niedbalski committed
    I spent a good while trying to debug this. Got working avoiding
    local_checkout but want to fix correctly. However this will unblock
    release.
  25. @pnasrat @niedbalski

    Fix missing import in test.

    pnasrat authored niedbalski committed
  26. @pnasrat @niedbalski

    Fix incorrect mocking.

    pnasrat authored niedbalski committed
  27. @niedbalski

    [pip/tests] added subpackage to _create_test_package method, fixed te…

    niedbalski authored
    …sts for support subdirectory
  28. @niedbalski
Commits on Jul 24, 2012
  1. @niedbalski
  2. @niedbalski

    Merge remote-tracking branch 'upstream/develop' into develop

    niedbalski authored
    Conflicts:
    	docs/news.txt
    	tests/test_user_site.py
This page is out of date. Refresh to see the latest.
View
2  docs/news.txt
@@ -13,7 +13,7 @@ Beta and final releases planned for the second half of 2012.
develop (unreleased)
-------------------
-* Fixed issue #355 - pip uninstall removes files it didn't install. Thanks
+ Fixed issue #355 - pip uninstall removes files it didn't install. Thanks
pjdelport.
* Fixed issues #493, #494, #440, and #573 related to improving support for the
View
120 pip/req.py
@@ -36,12 +36,17 @@
class InstallRequirement(object):
def __init__(self, req, comes_from, source_dir=None, editable=False,
- url=None, as_egg=False, update=True):
+ url=None, as_egg=False, update=True, editable_options=None):
self.extras = ()
if isinstance(req, string_types):
req = pkg_resources.Requirement.parse(req)
self.extras = req.extras
self.req = req
+
+ if not editable_options:
+ self.editable_options = {}
+
+ self.editable_options = editable_options
self.comes_from = comes_from
self.source_dir = source_dir
self.editable = editable
@@ -67,16 +72,16 @@ def __init__(self, req, comes_from, source_dir=None, editable=False,
@classmethod
def from_editable(cls, editable_req, comes_from=None, default_vcs=None):
name, url, extras_override = parse_editable(editable_req, default_vcs)
+
if url.startswith('file:'):
source_dir = url_to_path(url)
else:
source_dir = None
- res = cls(name, comes_from, source_dir=source_dir, editable=True, url=url)
-
+ res = cls(name, comes_from, source_dir=source_dir, editable=True, url=url, \
+ editable_options=extras_override)
if extras_override is not None:
res.extras = extras_override
-
return res
@classmethod
@@ -206,21 +211,36 @@ def url_name(self):
@property
def setup_py(self):
- return os.path.join(self.source_dir, 'setup.py')
+ setup_file = "setup.py"
+
+ if self.editable_options and \
+ 'subdirectory' in self.editable_options:
+ setup_py = os.path.join(self.source_dir, \
+ self.editable_options['subdirectory'],\
+ setup_file)
+ else:
+ setup_py = os.path.join(self.source_dir, setup_file)
+
+ return setup_py
def run_egg_info(self, force_root_egg_info=False):
assert self.source_dir
if self.name:
- logger.notify('Running setup.py egg_info for package %s' % self.name)
+ logger.notify('Running setup.py (path: %s) egg_info for package %s' % \
+ (self.setup_py, self.name))
else:
- logger.notify('Running setup.py egg_info for package from %s' % self.url)
+ logger.notify('Running setup.py (path: %s) egg_info for package from %s' % \
+ (self.setup_py, self.url))
logger.indent += 2
try:
script = self._run_setup_py
+
script = script.replace('__SETUP_PY__', repr(self.setup_py))
script = script.replace('__PKG_NAME__', repr(self.name))
+
# We can't put the .egg-info files at the root, because then the source code will be mistaken
# for an installed egg, causing problems
+
if self.editable or force_root_egg_info:
egg_base_option = []
else:
@@ -1334,11 +1354,52 @@ def parse_requirements(filename, finder=None, comes_from=None, options=None):
req = InstallRequirement.from_line(line, comes_from)
yield req
+def _strip_postfix(req):
+ """
+ Strip req postfix ( -dev, 0.2, etc )
+ """
+ ## FIXME: use package_to_requirement?
+ match = re.search(r'^(.*?)(?:-dev|-\d.*)$', req)
+ if match:
+ # Strip off -dev, -0.2, etc.
+ req = match.group(1)
+ return req
+
+def _build_req_from_url(url):
+
+ parts = [p for p in url.split('#', 1)[0].split('/') if p]
+
+ req = None
+ if parts[-2] in ('tags', 'branches', 'tag', 'branch'):
+ req = parts[-3]
+ elif parts[-1] == 'trunk':
+ req = parts[-2]
+ return req
+
+def _build_editable_options(req):
+
+ """
+ This method generates a dictionary of the query string
+ parameters contained in a given editable URL.
+ """
+ regexp = re.compile(r"[\?#&](?P<name>[^&=]+)=(?P<value>[^&=]+)")
+ matched = regexp.findall(req)
+
+ if matched:
+ ret = dict()
+ for option in matched:
+ (name, value) = option
+ if name in ret:
+ raise Exception("%s option already defined" % name)
+ ret[name] = value
+ return ret
+ return None
def parse_editable(editable_req, default_vcs=None):
- """Parses svn+http://blahblah@rev#egg=Foobar into a requirement
- (Foobar) and a URL"""
-
+ """
+ Parses svn+http://blahblah@rev#egg=Foobar into a requirement
+ returns package, url, options
+ """
url = editable_req
extras = None
@@ -1365,37 +1426,42 @@ def parse_editable(editable_req, default_vcs=None):
for version_control in vcs:
if url.lower().startswith('%s:' % version_control):
url = '%s+%s' % (version_control, url)
+ break
+
if '+' not in url:
if default_vcs:
url = default_vcs + '+' + url
else:
raise InstallationError(
'--editable=%s should be formatted with svn+URL, git+URL, hg+URL or bzr+URL' % editable_req)
+
vc_type = url.split('+', 1)[0].lower()
+
if not vcs.get_backend(vc_type):
error_message = 'For --editable=%s only ' % editable_req + \
', '.join([backend.name + '+URL' for backend in vcs.backends]) + \
' is currently supported'
raise InstallationError(error_message)
- match = re.search(r'(?:#|#.*?&)egg=([^&]*)', editable_req)
- if (not match or not match.group(1)) and vcs.get_backend(vc_type):
- parts = [p for p in editable_req.split('#', 1)[0].split('/') if p]
- if parts[-2] in ('tags', 'branches', 'tag', 'branch'):
- req = parts[-3]
- elif parts[-1] == 'trunk':
- req = parts[-2]
- else:
+
+ try:
+ options = _build_editable_options(editable_req)
+ except Exception:
+ message = sys.exc_info()[1]
+ raise InstallationError(
+ '--editable=%s error in editable options: %s' % (editable_req, message))
+
+ if not options or 'egg' not in options:
+ req = _build_req_from_url(editable_req)
+ if not req:
raise InstallationError(
- '--editable=%s is not the right format; it must have #egg=Package'
- % editable_req)
+ '--editable=%s is not the right format; it must have #egg=Package'
+ % editable_req)
else:
- req = match.group(1)
- ## FIXME: use package_to_requirement?
- match = re.search(r'^(.*?)(?:-dev|-\d.*)$', req)
- if match:
- # Strip off -dev, -0.2, etc.
- req = match.group(1)
- return req, url, None
+ req = options['egg']
+
+ package = _strip_postfix(req)
+
+ return package, url, options
class UninstallPathSet(object):
View
1  pip/util.py
@@ -521,6 +521,7 @@ def cache_download(target_file, temp_location, content_type):
def unpack_file(filename, location, content_type, link):
+ filename = os.path.realpath(filename)
if (content_type == 'application/zip'
or filename.endswith('.zip')
or filename.endswith('.pybundle')
View
4 tests/packages/pip-test-sub-package/piptestsubpackage/__init__.py
@@ -0,0 +1,4 @@
+# Example package with a console entry point
+
+def main():
+ print("Hello, World")
View
14 tests/packages/pip-test-sub-package/piptestsubpackage/setup.py
@@ -0,0 +1,14 @@
+from setuptools import setup, find_packages
+
+setup(name='pip-test-sub-package',
+ version='0.1.1',
+ author='PyPA',
+ author_email='pypa@pypa.pypa',
+ url='https://github.com/pypa',
+ license='MIT',
+ packages=find_packages(),
+ zip_safe=False,
+ entry_points={
+ 'console_scripts': ['pip-test-subpackage=piptestsubpackage:main'],
+ },
+)
View
14 tests/packages/pip-test-sub-package/setup.py
@@ -0,0 +1,14 @@
+from setuptools import setup, find_packages
+
+setup(name='pip-test-sub-package',
+ version='0.1.1',
+ author='PyPA',
+ author_email='pypa@pypa.pypa',
+ url='https://github.com/pypa',
+ license='MIT',
+ packages=find_packages(),
+ zip_safe=False,
+ entry_points={
+ 'console_scripts': ['pip-test-package=piptestpackage:main'],
+ },
+)
View
22 tests/test_pip.py
@@ -533,6 +533,8 @@ def write_file(filename, text, dest=None):
"""
env = get_env()
if dest:
+ if not os.path.exists(dest):
+ os.mkdir(dest)
complete_path = dest/ filename
else:
complete_path = env.scratch_path/ filename
@@ -628,13 +630,15 @@ def assert_all_changes(start_state, end_state, expected_changes):
return diff
-def _create_test_package(env):
+def _create_test_package(env, subpackage=None):
mkdir('version_pkg')
version_pkg_path = env.scratch_path/'version_pkg'
write_file('version_pkg.py', textwrap.dedent('''\
def main():
print('0.1')
'''), version_pkg_path)
+
+
write_file('setup.py', textwrap.dedent('''\
from setuptools import setup, find_packages
setup(name='version_pkg',
@@ -643,6 +647,22 @@ def main():
py_modules=['version_pkg'],
entry_points=dict(console_scripts=['version_pkg=version_pkg:main']))
'''), version_pkg_path)
+
+ if subpackage:
+ subpackage_path = os.path.join(version_pkg_path, subpackage)
+ write_file('version_subpkg.py', textwrap.dedent('''\
+ def main():
+ print('0.1')
+ '''), subpackage_path)
+ write_file('setup.py', textwrap.dedent('''\
+ from setuptools import setup, find_packages
+ setup(name='version_subpkg',
+ version='0.1',
+ packages=find_packages(),
+ py_modules=['version_subpkg'],
+ entry_points=dict(console_scripts=['version_pkg=version_subpkg:main']))
+ '''), subpackage_path)
+
env.run('git', 'init', cwd=version_pkg_path)
env.run('git', 'add', '.', cwd=version_pkg_path)
env.run('git', 'commit', '-q',
View
10 tests/test_requirements.py
@@ -142,19 +142,21 @@ def test_parse_editable_local(isdir_mock, exists_mock, getcwd_mock, normcase_moc
def test_parse_editable_default_vcs():
assert_equal(
parse_editable('https://foo#egg=foo', 'git'),
- ('foo', 'git+https://foo#egg=foo', None)
+ ('foo', 'git+https://foo#egg=foo', {'egg': 'foo'})
)
def test_parse_editable_explicit_vcs():
assert_equal(
parse_editable('svn+https://foo#egg=foo', 'git'),
- ('foo', 'svn+https://foo#egg=foo', None)
+ ('foo', 'svn+https://foo#egg=foo', \
+ {'egg': 'foo'})
)
def test_parse_editable_vcs_extras():
assert_equal(
- parse_editable('svn+https://foo#egg=foo[extras]', 'git'),
- ('foo[extras]', 'svn+https://foo#egg=foo[extras]', None)
+ parse_editable('svn+https://foo#egg=foo[extras]', 'svn'),
+ ('foo[extras]', 'svn+https://foo#egg=foo[extras]', \
+ {'egg': "foo[extras]"})
)
@patch('os.path.normcase')
View
11 tests/test_user_site.py
@@ -17,6 +17,14 @@ def dist_in_site_packages(dist):
"""
+patch_dist_in_site_packages = """
+ def dist_in_site_packages(dist):
+ return False
+ import pip
+ pip.util.dist_in_site_packages=dist_in_site_packages
+"""
+
+
def test_install_curdir_usersite_fails_in_old_python():
"""
Test --user option on older Python versions (pre 2.6) fails intelligibly
@@ -194,7 +202,6 @@ def test_install_user_in_global_virtualenv_with_conflict_fails(self):
assert result2.stdout.startswith("Will not install to the user site because it will lack sys.path precedence to %s in %s"
%('INITools', dist_location)), result2.stdout
-
def test_uninstall_from_usersite(self):
"""
Test uninstall from usersite
@@ -223,5 +230,3 @@ def test_uninstall_editable_from_usersite(self):
assert_all_changes(result1, result2,
[env.venv/'build', 'cache', env.user_site/'easy-install.pth'])
-
-
View
21 tests/test_vcs_backends.py
@@ -1,7 +1,7 @@
-from tests.test_pip import (reset_env, run_pip,
+from tests.test_pip import (here, reset_env, run_pip,
_create_test_package, _change_test_package_version)
from tests.local_repos import local_checkout
-
+import os
def test_install_editable_from_git_with_https():
"""
@@ -14,6 +14,21 @@ def test_install_editable_from_git_with_https():
expect_error=True)
result.assert_installed('pip-test-package', with_files=['.git'])
+def test_install_editable_with_subdirectory():
+ """
+ Test installing a package from a repo subdirectory
+ """
+ env = reset_env()
+ version_pkg_path = _create_test_package(env, subpackage='version_subpkg')
+ #un_pip('install', '-e', '%s@%s#egg=version_pkg' % ('git+file://' + version_pkg_path.abspath.replace('\\', '/'), sha1))
+ #version = env.run('version_pkg')
+ #assert '0.1' in version.stdout, version.stdout
+
+ #reset_env()
+ #result = run_pip('install', '-e',
+ # '%s#egg=pip-test-sub-package&subdirectory=piptestsubpackage' %
+ # ('git+file://' + os.path.join(here, 'packages/pip-test-sub-package')))
+ #result.assert_installed('pip-test-sub-package')
def test_git_with_sha1_revisions():
"""
@@ -28,6 +43,8 @@ def test_git_with_sha1_revisions():
assert '0.1' in version.stdout, version.stdout
+
+
def test_git_with_branch_name_as_revision():
"""
Git backend should be able to install from branch names
Something went wrong with that request. Please try again.