Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pipenv fails to install from hg+https at a given revision #3090

Closed
eolo999 opened this issue Oct 24, 2018 · 7 comments
Closed

Pipenv fails to install from hg+https at a given revision #3090

eolo999 opened this issue Oct 24, 2018 · 7 comments
Assignees

Comments

@eolo999
Copy link

@eolo999 eolo999 commented Oct 24, 2018

Issue description

Trying to install a package from hg+https fails with AttributeError: 'Mercurial' object has no attribute 'get_revision_sha'

Steps to replicate

cd /tmp
mkdir pipenv_hg_fail
cd pipenv_hg_fail
pipenv install -e "hg+https://www.mercurial-scm.org/repo/hg/@stable#egg=hg"

results in the following stacktrace:

Traceback (most recent call last):
File "/home/xxx/.local/bin/pipenv", line 11, in
sys.exit(cli())
File "/home/xxx/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 764, in call
return self.main(*args, **kwargs)
File "/home/xxx/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/home/xxx/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/xxx/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/xxx/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/home/xxx/.local/lib/python3.6/site-packages/pipenv/vendor/click/decorators.py", line 64, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "/home/xxx/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/home/xxx/.local/lib/python3.6/site-packages/pipenv/vendor/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/xxx/.local/lib/python3.6/site-packages/pipenv/cli/command.py", line 249, in install
editable_packages=state.installstate.editables,
File "/home/xxx/.local/lib/python3.6/site-packages/pipenv/core.py", line 1976, in do_install
skip_lock=skip_lock,
File "/home/xxx/.local/lib/python3.6/site-packages/pipenv/core.py", line 1274, in do_init
pypi_mirror=pypi_mirror,
File "/home/xxx/.local/lib/python3.6/site-packages/pipenv/core.py", line 1058, in do_lock
dev=settings["dev"],
File "/home/xxx/.local/lib/python3.6/site-packages/pipenv/utils.py", line 1154, in get_vcs_deps
with locked_repository(requirement) as repo:
File "/home/xxx/.pyenv/versions/3.6.4/lib/python3.6/contextlib.py", line 81, in enter
return next(self.gen)
File "/home/xxx/.local/lib/python3.6/site-packages/pipenv/utils.py", line 1288, in locked_repository
with requirement.req.locked_vcs_repo(src_dir=src_dir) as repo:
File "/home/xxx/.pyenv/versions/3.6.4/lib/python3.6/contextlib.py", line 81, in enter
return next(self.gen)
File "/home/xxx/.local/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 621, in locked_vcs_repo
vcsrepo = self.get_vcs_repo(src_dir=src_dir)
File "/home/xxx/.local/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 599, in get_vcs_repo
vcsrepo.obtain()
File "/home/xxx/.local/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/vcs.py", line 37, in obtain
self.checkout_ref(self.ref)
File "/home/xxx/.local/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/vcs.py", line 45, in checkout_ref
self.checkout_directory, self.get_commit_hash(ref)
File "/home/xxx/.local/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/vcs.py", line 63, in get_commit_hash
return self.repo_instance.get_revision_sha(self.checkout_directory, target_ref.arg_rev)
AttributeError: 'Mercurial' object has no attribute 'get_revision_sha'


$ pipenv --support

Pipenv version: '2018.10.13'

Pipenv location: '/home/xxx/.local/lib/python3.6/site-packages/pipenv'

Python location: '/home/xxx/.pyenv/versions/3.6.4/bin/python3.6'

Python installations found:

  • 3.6.6: /usr/bin/python3.6
  • 3.6.6: /usr/bin/python3.6m
  • 3.6.2: /usr/local/bin/python3.6
  • 3.6.2: /usr/local/bin/python3.6m
  • 2.7.15rc1: /usr/bin/python2.7

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.6.4',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '4.15.0-36-generic',
 'platform_system': 'Linux',
 'platform_version': '#39-Ubuntu SMP Mon Sep 24 16:19:09 UTC 2018',
 'python_full_version': '3.6.4',
 'python_version': '3.6',
 'sys_platform': 'linux'}

System environment variables:

  • XDG_CONFIG_DIRS
  • LC_TELEPHONE
  • LANG
  • DISPLAY
  • SHLVL
  • LOGNAME
  • XDG_VTNR
  • MANDATORY_PATH
  • LC_NAME
  • PWD
  • XAUTHORITY
  • QT_QPA_PLATFORMTHEME
  • GTK_IM_MODULE
  • COLORTERM
  • XDG_SESSION_ID
  • GTK2_MODULES
  • GNOME_TERMINAL_SCREEN
  • DESKTOP_SESSION
  • XDG_SESSION_DESKTOP
  • GDMSESSION
  • USERNAME
  • TEXTDOMAINDIR
  • WINDOWPATH
  • LC_MEASUREMENT
  • LC_PAPER
  • LC_NUMERIC
  • LC_MONETARY
  • IM_CONFIG_PHASE
  • DEFAULTS_PATH
  • DBUS_SESSION_BUS_ADDRESS
  • VTE_VERSION
  • CLUTTER_IM_MODULE
  • LC_IDENTIFICATION
  • TEXTDOMAIN
  • QT4_IM_MODULE
  • GNOME_TERMINAL_SERVICE
  • XDG_DATA_DIRS
  • S_COLORS
  • SHELL
  • XDG_SESSION_TYPE
  • QT_IM_MODULE
  • LC_TIME
  • TERM
  • SSH_AGENT_PID
  • GTK_MODULES
  • XDG_CURRENT_DESKTOP
  • PATH
  • SSH_AUTH_SOCK
  • LIBVIRT_DEFAULT_URI
  • HOME
  • XDG_SEAT
  • XMODIFIERS
  • XDG_RUNTIME_DIR
  • LC_ADDRESS
  • GPG_AGENT_INFO
  • USER
  • QT_ACCESSIBILITY
  • OLDPWD
  • LS_COLORS
  • PYTHONSTARTUP
  • DEBFULLNAME
  • DEBEMAIL
  • LESS
  • MANPAGER
  • EDITOR
  • GIT_EDITOR
  • NOSE_REDNOSE
  • N_PREFIX
  • GOROOT
  • GOPATH
  • AUTOJUMP_SOURCED
  • AUTOJUMP_ERROR_PATH
  • _
  • PYTHONDONTWRITEBYTECODE
  • PIP_SHIMS_BASE_MODULE
  • PIP_PYTHON_PATH

Pipenv–specific environment variables:

Debug–specific environment variables:

  • PATH: /home/xxx/code/bin:/home/xxx/code/n/n/versions/node/6.10.3/bin:/home/xxx/.local/bin:/home/xxx/bin:/home/xxx/.cargo/bin:/home/xxx/xiki-master/bin:/home/xxx/code/bin:/home/xxx/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/xxx/code/go/bin
  • SHELL: /usr/bin/zsh
  • EDITOR: /home/xxx/code/bin/vim
  • LANG: en_US.UTF-8
  • PWD: /tmp/test_pipenv_hg

Contents of Pipfile ('/tmp/test_pipenv_hg/Pipfile'):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
hg = {editable = true, ref = "stable", hg = "https://www.mercurial-scm.org/repo/hg/"}

[dev-packages]

[requires]
python_version = "3.6"
@kr1
Copy link

@kr1 kr1 commented Oct 25, 2018

I have the same issue.
It appears to be caused by vcs in requirementslib calling get_revision_sha whereas mercurial in patched/notpip/_internal/vcs has get_revision_hash, one possible workaround is here:

kr1@518176b#diff-e8a9d126ae6283d8364c7fe4853401bdL81

Should this be addressed in requirementslib?

@techalchemy
Copy link
Member

@techalchemy techalchemy commented Oct 27, 2018

Yes most likely, although it’s also possible we are using this wrong somewhere (some of the git API didn’t behave quite how I expected so I wound up just experimenting until o got the results I wanted).

Possibly @cjerdonek can speak to whether this is a change that should be reflected back up to pip or whether we can just fix the call in requirementslib.

@cjerdonek
Copy link
Member

@cjerdonek cjerdonek commented Oct 27, 2018

Currently, get_revision() is the only method along those lines meant to be implemented in all the VCS-specific subclasses (which is why it's in the VersionControl base class). There are a couple related methods in certain subclasses that are for use only within those subclasses (e.g. get_revision_sha in the Git class and get_revision_hash for Mercurial). So pipenv should probably be using get_revision(). It would be good to know if get_revision() isn't doing what pipenv wants for some VCS class though. Perhaps the method could be adjusted to be more useful.

@techalchemy
Copy link
Member

@techalchemy techalchemy commented Oct 28, 2018

Hmmm my implementation provides for two possibilities but I only seem to use one of them myself in most cases, so by default I actually do just call get_revision(checkout_directory) which may or may not be correct. I have an implementation which relies on get_revision_sha but only if you ask explicitly for a ref lookup--

This lives in requirementslib.models.requirements.VCSRequirement

    def get_vcs_repo(self, src_dir=None):
        from .vcs import VCSRepository
        checkout_dir = self.get_checkout_dir(src_dir=src_dir)
        url = "{0}#egg={1}".format(self.vcs_uri, self.name)
        vcsrepo = VCSRepository(
            url=url,
            name=self.name,
            ref=self.ref if self.ref else None,
            checkout_directory=checkout_dir,
            vcs_type=self.vcs
        )
        if not self.is_local:
            vcsrepo.obtain()
        return vcsrepo

    @contextmanager
    def locked_vcs_repo(self, src_dir=None):
        vcsrepo = self.get_vcs_repo(src_dir=src_dir)
        if self.ref and not self.is_local:
            vcsrepo.checkout_ref(self.ref)
        self.ref = self.get_commit_hash()
        self.req.revision = self.ref

and it creates a new requirementslib.models.VCSRepository instance which will obtain a copy of the repository which is secretly a wrapper around pip's implementation:

VCS_SUPPORT = VcsSupport()


@attr.s
class VCSRepository(object):
...
    @repo_instance.default
    def get_repo_instance(self):
        backend = VCS_SUPPORT._registry.get(self.vcs_type)
        return backend(url=self.url)

    def obtain(self):
        if not os.path.exists(self.checkout_directory):
            self.repo_instance.obtain(self.checkout_directory)
        if self.ref:
            self.checkout_ref(self.ref)
            self.commit_sha = self.get_commit_hash(self.ref)
        else:
            if not self.commit_sha:
                self.commit_sha = self.get_commit_hash()

    def checkout_ref(self, ref):
        if not self.repo_instance.is_commit_id_equal(
            self.checkout_directory, self.get_commit_hash(ref)
        ) and not self.repo_instance.is_commit_id_equal(self.checkout_directory, ref):
            if not self.is_local:
                self.update(ref)

    def get_commit_hash(self, ref=None):
        if ref:
            target_ref = self.repo_instance.make_rev_options(ref)
            return self.repo_instance.get_revision_sha(self.checkout_directory, target_ref.arg_rev)
        return self.repo_instance.get_revision(self.checkout_directory)
@techalchemy
Copy link
Member

@techalchemy techalchemy commented Oct 28, 2018

ah but in update I do use it:

    def update(self, ref):
        target_ref = self.repo_instance.make_rev_options(ref)
        sha = self.repo_instance.get_revision_sha(self.checkout_directory, target_ref.arg_rev)
        if parse_version(pip_version) > parse_version("18.0"):
            self.repo_instance.update(self.checkout_directory, self.url, target_ref)
        else:
            self.repo_instance.update(self.checkout_directory, target_ref)
        self.commit_hash = self.get_commit_hash(ref)
@cjerdonek
Copy link
Member

@cjerdonek cjerdonek commented Oct 28, 2018

By the way, one observation: in the case of not os.path.exists(self.checkout_directory), you shouldn't ever need to call update() after obtain() because the first obtain() should be putting you in the right state after a fresh clone. (If something's not right about that state, I'd like to know.)

@techalchemy
Copy link
Member

@techalchemy techalchemy commented Oct 28, 2018

Oh perfect. That simplified things considerably

@eolo999 eolo999 changed the title Pipev fails to install from hg+https at a given revision Pipenv fails to install from hg+https at a given revision Oct 29, 2018
@techalchemy techalchemy self-assigned this Oct 29, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.