Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix --generate-hashes with bare VCS URLs #812

Merged
merged 1 commit into from May 11, 2019

Conversation

jcushman
Copy link
Contributor

@jcushman jcushman commented May 10, 2019

#807 introduced an unhandled edge case when bare VCS URLs like git+git://github.com/jazzband/pip-tools@7d86c8d3ecd1faa6be11c7ddc6b29a30ffd1dae3 are used with pip-compile --generate-hashes:

$ cat requirements.in
git+https://github.com/jazzband/pip-tools@7d86c8d3ecd1faa6be11c7ddc6b29a30ffd1dae3
$ pip-compile --generate-hashes
Traceback (most recent call last):
  File "/Users/jcwork/.pyenv/versions/pip-tools/bin/pip-compile", line 11, in <module>
    load_entry_point('pip-tools', 'console_scripts', 'pip-compile')()
  File "/Users/jcwork/.pyenv/versions/3.5.3/envs/pip-tools/lib/python3.5/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/Users/jcwork/.pyenv/versions/3.5.3/envs/pip-tools/lib/python3.5/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/Users/jcwork/.pyenv/versions/3.5.3/envs/pip-tools/lib/python3.5/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/jcwork/.pyenv/versions/3.5.3/envs/pip-tools/lib/python3.5/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/Users/jcwork/.pyenv/versions/3.5.3/envs/pip-tools/lib/python3.5/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/jcwork/Documents/pip-tools/piptools/scripts/compile.py", line 357, in cli
    hashes = resolver.resolve_hashes(results)
  File "/Users/jcwork/Documents/pip-tools/piptools/resolver.py", line 88, in resolve_hashes
    return {ireq: self.repository.get_hashes(ireq) for ireq in ireqs}
  File "/Users/jcwork/Documents/pip-tools/piptools/resolver.py", line 88, in <dictcomp>
    return {ireq: self.repository.get_hashes(ireq) for ireq in ireqs}
  File "/Users/jcwork/Documents/pip-tools/piptools/repositories/local.py", line 78, in get_hashes
    return self.repository.get_hashes(ireq)
  File "/Users/jcwork/Documents/pip-tools/piptools/repositories/pypi.py", line 299, in get_hashes
    return {self._get_file_hash(cached_link)}
  File "/Users/jcwork/Documents/pip-tools/piptools/repositories/pypi.py", line 323, in _get_file_hash
    with open_local_or_remote_file(location, self.session) as f:
  File "/Users/jcwork/.pyenv/versions/3.5.3/lib/python3.5/contextlib.py", line 59, in __enter__
    return next(self.gen)
  File "/Users/jcwork/Documents/pip-tools/piptools/repositories/pypi.py", line 397, in open_local_or_remote_file
    response = session.get(url, headers=headers, stream=True)
  File "/Users/jcwork/.pyenv/versions/3.5.3/envs/pip-tools/lib/python3.5/site-packages/pip/_vendor/requests/sessions.py", line 546, in get
    return self.request('GET', url, **kwargs)
  File "/Users/jcwork/.pyenv/versions/3.5.3/envs/pip-tools/lib/python3.5/site-packages/pip/_internal/download.py", line 439, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/Users/jcwork/.pyenv/versions/3.5.3/envs/pip-tools/lib/python3.5/site-packages/pip/_vendor/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/jcwork/.pyenv/versions/3.5.3/envs/pip-tools/lib/python3.5/site-packages/pip/_vendor/requests/sessions.py", line 640, in send
    adapter = self.get_adapter(url=request.url)
  File "/Users/jcwork/.pyenv/versions/3.5.3/envs/pip-tools/lib/python3.5/site-packages/pip/_vendor/requests/sessions.py", line 731, in get_adapter
    raise InvalidSchema("No connection adapters were found for '%s'" % url)
pip._vendor.requests.exceptions.InvalidSchema: No connection adapters were found for 'git+https://github.com/jazzband/pip-tools@7d86c8d3ecd1faa6be11c7ddc6b29a30ffd1dae3'

This is because PyPIRepository.get_hashes incorrectly detects all non-editable URLs as hashable, instead of only artifact URLs.

This PR does two things in separate commits (so you can confirm the failing behavior and test case if you want):

  • Add a failing test case by having test_url_package try all URLs with and without --generate-hashes
  • Fix the test case by updating get_hashes to use a better hashability check, modeled on the hashability logic in pip's RequirementPreparer.prepare_linked_requirement

Changelog-friendly one-liner: Fix --generate-hashes with bare VCS URLs

Contributor checklist
  • Provided the tests for the changes.
  • Requested a review from another contributor.
  • Gave a clear one-line description in the PR (that the maintainers can add to CHANGELOG.md on release).
  • Assign the PR to an existing or new milestone for the target version (following Semantic Versioning).

@codecov
Copy link

codecov bot commented May 10, 2019

Codecov Report

Merging #812 into master will increase coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #812      +/-   ##
==========================================
+ Coverage   98.83%   98.83%   +<.01%     
==========================================
  Files          36       36              
  Lines        2147     2150       +3     
  Branches      278      279       +1     
==========================================
+ Hits         2122     2125       +3     
  Misses         15       15              
  Partials       10       10
Impacted Files Coverage Δ
piptools/repositories/pypi.py 93.84% <100%> (+0.06%) ⬆️
tests/test_cli_compile.py 100% <100%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e6bdfa7...d755c60. Read the comment docs.

Copy link
Member

@atugushev atugushev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@atugushev atugushev added this to the 3.7.1 milestone May 10, 2019
@atugushev
Copy link
Member

@jcushman squash please commits.

@jcushman
Copy link
Contributor Author

Squashed!

@atugushev atugushev merged commit 56bc1dd into jazzband:master May 11, 2019
@atugushev atugushev modified the milestones: 3.7.1, 3.8.0 May 30, 2019
@atugushev
Copy link
Member

pip-tools v3.8.0 is released

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants