Skip to content

Pipenv lock doesn't take Pipfile into account when processing package sub-dependencies #2666

@ghost

Description

Issue description

After the release of pylint 2, I discovered this issue trying to install plugins such as pylint-quotes and pytest-pylint:

Pipenv seems to ignore dependency versions in Pipfile when looking through sub-dependencies (dependencies of packages specified in Pipfile).

Expected result

Pipenv should account for Pipfile when resolving sub-depencendies to avoid reporting erroneous conflicts

Actual result

Pipenv incorrectly reports dependency conflicts when none actually exist.

Steps to replicate

Example Pipfile:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
pylint = "==1.9"
pylint-quotes = "==0.1.9"
$ pipenv lock -v
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
using sources: [{'url': 'https://pypi.org/simple', 'verify_ssl': True, 'name': 'pypi'}]
Using pip: -i https://pypi.org/simple

                          ROUND 1
Current constraints:
  pylint==1.9 (from -r /var/folders/rj/210lgzrd4q5ccsyby0rdxg0924842d/T/pipenv-vkxnjh1h-requirements/pipenv-2cbbt1ga-constraints.txt (line 2))
  pylint-quotes==0.1.9 (from -r /var/folders/rj/210lgzrd4q5ccsyby0rdxg0924842d/T/pipenv-vkxnjh1h-requirements/pipenv-2cbbt1ga-constraints.txt (line 3))

Finding the best candidates:
  found candidate pylint==1.9 (constraint was ==1.9)
  found candidate pylint-quotes==0.1.9 (constraint was ==0.1.9)

Finding secondary dependencies:
  pylint-quotes==0.1.9      requires astroid>=2.0.1, isort>=4.2.5, lazy-object-proxy, mccabe, pylint-quotes==0.1.9, pylint>=1.7.6, six, wrapt
  pylint==1.9               requires astroid<2.0,>=1.6, isort>=4.2.5, lazy-object-proxy, mccabe, pylint==1.9, six, wrapt

New dependencies found in this round:
  adding ['astroid', '<2.0,>=1.6,>=2.0.1', '[]']
  adding ['isort', '>=4.2.5', '[]']
  adding ['lazy-object-proxy', '', '[]']
  adding ['mccabe', '', '[]']
  adding ['pylint', '==1.9,>=1.7.6', '[]']
  adding ['pylint-quotes', '==0.1.9', '[]']
  adding ['six', '', '[]']
  adding ['wrapt', '', '[]']
Removed dependencies in this round:
Unsafe dependencies in this round:
------------------------------------------------------------
Result of round 1: not stable

                          ROUND 2
Current constraints:
  astroid<2.0,>=1.6,>=2.0.1
  isort>=4.2.5
  lazy-object-proxy
  mccabe
  pylint==1.9,>=1.7.6 (from -r /var/folders/rj/210lgzrd4q5ccsyby0rdxg0924842d/T/pipenv-vkxnjh1h-requirements/pipenv-2cbbt1ga-constraints.txt (line 2))
  pylint-quotes==0.1.9 (from -r /var/folders/rj/210lgzrd4q5ccsyby0rdxg0924842d/T/pipenv-vkxnjh1h-requirements/pipenv-2cbbt1ga-constraints.txt (line 3))
  six
  wrapt

Finding the best candidates:
Using pip: -i https://pypi.org/simple

                          ROUND 1
Current constraints:
  pylint==1.9 (from -r /var/folders/rj/210lgzrd4q5ccsyby0rdxg0924842d/T/pipenv-vkxnjh1h-requirements/pipenv-eulvhsyk-constraints.txt (line 2))
  pylint-quotes==0.1.9 (from -r /var/folders/rj/210lgzrd4q5ccsyby0rdxg0924842d/T/pipenv-vkxnjh1h-requirements/pipenv-eulvhsyk-constraints.txt (line 3))

Finding the best candidates:
  found candidate pylint==1.9 (constraint was ==1.9)
  found candidate pylint-quotes==0.1.9 (constraint was ==0.1.9)

Finding secondary dependencies:
  pylint-quotes==0.1.9      requires astroid>=2.0.1, isort>=4.2.5, lazy-object-proxy, mccabe, pylint-quotes==0.1.9, pylint>=1.7.6, six, wrapt
  pylint==1.9               requires astroid<2.0,>=1.6, isort>=4.2.5, lazy-object-proxy, mccabe, pylint==1.9, six, wrapt

New dependencies found in this round:
  adding ['astroid', '<2.0,>=1.6,>=2.0.1', '[]']
  adding ['isort', '>=4.2.5', '[]']
  adding ['lazy-object-proxy', '', '[]']
  adding ['mccabe', '', '[]']
  adding ['pylint', '==1.9,>=1.7.6', '[]']
  adding ['pylint-quotes', '==0.1.9', '[]']
  adding ['six', '', '[]']
  adding ['wrapt', '', '[]']
Removed dependencies in this round:
Unsafe dependencies in this round:
------------------------------------------------------------
Result of round 1: not stable

                          ROUND 2
Current constraints:
  astroid<2.0,>=1.6,>=2.0.1
  isort>=4.2.5
  lazy-object-proxy
  mccabe
  pylint==1.9,>=1.7.6 (from -r /var/folders/rj/210lgzrd4q5ccsyby0rdxg0924842d/T/pipenv-vkxnjh1h-requirements/pipenv-eulvhsyk-constraints.txt (line 2))
  pylint-quotes==0.1.9 (from -r /var/folders/rj/210lgzrd4q5ccsyby0rdxg0924842d/T/pipenv-vkxnjh1h-requirements/pipenv-eulvhsyk-constraints.txt (line 3))
  six
  wrapt

Finding the best candidates:

Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
Could not find a version that matches astroid<2.0,>=1.6,>=2.0.1
Tried: 1.0.0, 1.0.1, 1.1.0, 1.1.1, 1.2.0, 1.2.1, 1.3.0, 1.3.0, 1.3.1, 1.3.1, 1.3.2, 1.3.2, 1.3.3, 1.3.3, 1.3.4, 1.3.4, 1.3.5, 1.3.5, 1.3.6, 1.3.6, 1.3.7, 1.3.7, 1.3.8, 1.3.8, 1.4.0, 1.4.0, 1.4.1, 1.4.1, 1.4.2, 1.4.2, 1.4.3, 1.4.3, 1.4.4, 1.4.4, 1.4.5, 1.4.5, 1.4.6, 1.4.6, 1.4.7, 1.4.7, 1.4.8, 1.4.8, 1.4.9, 1.4.9, 1.5.0, 1.5.0, 1.5.1, 1.5.1, 1.5.2, 1.5.2, 1.5.3, 1.5.3, 1.6.0, 1.6.0, 1.6.1, 1.6.1, 1.6.2, 1.6.2, 1.6.3, 1.6.3, 1.6.4, 1.6.4, 1.6.5, 1.6.5, 2.0, 2.0, 2.0.1, 2.0.1
Skipped pre-versions: 2.0.0.dev0, 2.0.0.dev0, 2.0.0.dev1, 2.0.0.dev1, 2.0.0.dev2, 2.0.0.dev2, 2.0.0.dev3, 2.0.0.dev3, 2.0.0.dev4, 2.0.0.dev4
There are incompatible versions in the resolved dependencies.
Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
Could not find a version that matches astroid<2.0,>=1.6,>=2.0.1
Tried: 1.0.0, 1.0.1, 1.1.0, 1.1.1, 1.2.0, 1.2.1, 1.3.0, 1.3.0, 1.3.1, 1.3.1, 1.3.2, 1.3.2, 1.3.3, 1.3.3, 1.3.4, 1.3.4, 1.3.5, 1.3.5, 1.3.6, 1.3.6, 1.3.7, 1.3.7, 1.3.8, 1.3.8, 1.4.0, 1.4.0, 1.4.1, 1.4.1, 1.4.2, 1.4.2, 1.4.3, 1.4.3, 1.4.4, 1.4.4, 1.4.5, 1.4.5, 1.4.6, 1.4.6, 1.4.7, 1.4.7, 1.4.8, 1.4.8, 1.4.9, 1.4.9, 1.5.0, 1.5.0, 1.5.1, 1.5.1, 1.5.2, 1.5.2, 1.5.3, 1.5.3, 1.6.0, 1.6.0, 1.6.1, 1.6.1, 1.6.2, 1.6.2, 1.6.3, 1.6.3, 1.6.4, 1.6.4, 1.6.5, 1.6.5, 2.0, 2.0, 2.0.1, 2.0.1
Skipped pre-versions: 2.0.0.dev0, 2.0.0.dev0, 2.0.0.dev1, 2.0.0.dev1, 2.0.0.dev2, 2.0.0.dev2, 2.0.0.dev3, 2.0.0.dev3, 2.0.0.dev4, 2.0.0.dev4
There are incompatible versions in the resolved dependencies.

pylint 1.9 requires astroid<2.0,>=1.6
pylint-quotes 0.1.9's only dependency is pylint>=1.7.6 which can be seen in its setup.cfg (or by running pip show after installing it):

...
install_requires=[
        'pylint>=1.7.6',
    ],
...

However, Pipenv incorrectly reports that pylint-quotes requires astroid>=2.0.1.

Further investigation revealed that there is one related package that does require astroid>=2.0.1, pylint==2.0.1 which we are not installing. It looks to me like Pipenv is seeing the pylint requirement of pylint-quotes and then checking the dependencies of the latest version of pylint rather than the one actually specified in the pipfile.


$ pipenv --support

Pipenv version: '2018.7.1'

Pipenv location: '/usr/local/Cellar/pipenv/2018.7.1/libexec/lib/python3.7/site-packages/pipenv'

Python location: '/usr/local/Cellar/pipenv/2018.7.1/libexec/bin/python3.7'

Other Python installations in PATH:

  • 2.7: /usr/local/bin/python2.7

  • 2.7: /usr/local/bin/python2.7

  • 2.7: /Users/<user>/.pyenv/shims/python2.7

  • 2.7: /usr/local/bin/python2.7

  • 2.7: /usr/bin/python2.7

  • 3.6: /Users/<user>/.pyenv/shims/python3.6m

  • 3.6: /Users/<user>/.pyenv/shims/python3.6

  • 3.7: /Users/<user>/.pyenv/shims/python3.7

  • 2.7.15: /usr/local/bin/python

  • 3.6.5: /Users/<user>/.pyenv/shims/python

  • 2.7.15: /usr/local/bin/python

  • 2.7.10: /usr/bin/python

  • 2.7.15: /usr/local/bin/python2

  • 2.7.15: /Users/<user>/.pyenv/shims/python2

  • 2.7.15: /usr/local/bin/python2

  • 3.6.5: /Users/<user>/.pyenv/shims/python3

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.7.0',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '17.7.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT '
                     '2018; root:xnu-4570.71.2~1/RELEASE_X86_64',
 'python_full_version': '3.7.0',
 'python_version': '3.7',
 'sys_platform': 'darwin'}

System environment variables:

  • PATH
  • LDFLAGS
  • MANPATH
  • _fzf_orig_completion_tee
  • TERM_PROGRAM
  • _fzf_orig_completion_find
  • _fzf_orig_completion_diff
  • _fzf_orig_completion_javac
  • PYENV_ROOT
  • TERM
  • SHELL
  • _fzf_orig_completion_curl
  • CPPFLAGS
  • _fzf_orig_completion_mv
  • TMPDIR
  • _fzf_orig_completion_patch
  • Apple_PubSub_Socket_Render
  • _fzf_orig_completion_perl
  • TERM_PROGRAM_VERSION
  • _fzf_orig_completion_python
  • _fzf_orig_completion_du
  • _fzf_orig_completion_bunzip2
  • TERM_SESSION_ID
  • _fzf_orig_completion_less
  • _fzf_orig_completion_rmdir
  • _fzf_orig_completion_tail
  • _fzf_orig_completion_head
  • PYENV_VERSION
  • _fzf_orig_completion_jar
  • _fzf_orig_completion_svn
  • _fzf_orig_completion_telnet
  • USER
  • _fzf_orig_completion_g__
  • _fzf_orig_completion_wc
  • _fzf_orig_completion_ftp
  • _fzf_orig_completion_gzip
  • PYENV_DIR
  • SSH_AUTH_SOCK
  • _fzf_orig_completion_view
  • _fzf_orig_completion_export
  • __CF_USER_TEXT_ENCODING
  • PYENV_VIRTUALENV_INIT
  • _fzf_orig_completion_grep
  • _fzf_orig_completion_gvim
  • _fzf_orig_completion_java
  • _fzf_orig_completion_unzip
  • _fzf_orig_completion_sftp
  • PWD
  • _fzf_orig_completion_rm
  • _fzf_orig_completion_ls
  • _fzf_orig_completion_uniq
  • EDITOR
  • _fzf_orig_completion_cat
  • _fzf_orig_completion_chown
  • _fzf_orig_completion_bzip2
  • LANG
  • ITERM_PROFILE
  • PYENV_HOOK_PATH
  • XPC_FLAGS
  • _fzf_orig_completion_cd
  • _fzf_orig_completion_vi
  • _fzf_orig_completion_tar
  • XPC_SERVICE_NAME
  • _fzf_orig_completion_kill
  • PYENV_SHELL
  • SHLVL
  • COLORFGBG
  • HOME
  • ITERM_SESSION_ID
  • LOGNAME
  • _fzf_orig_completion_vim
  • VISUAL
  • _fzf_orig_completion_awk
  • _fzf_orig_completion_ld
  • _fzf_orig_completion_sort
  • _fzf_orig_completion_ssh
  • _fzf_orig_completion_gunzip
  • PKG_CONFIG_PATH
  • GOPATH
  • _fzf_orig_completion_rsync
  • _fzf_orig_completion_gcc
  • _fzf_orig_completion_emacs
  • DISPLAY
  • _fzf_orig_completion_cp
  • _fzf_orig_completion_scp
  • _fzf_orig_completion_ln
  • _fzf_orig_completion_sed
  • _fzf_orig_completion_git
  • COLORTERM
  • PYTHONDONTWRITEBYTECODE
  • PIP_PYTHON_PATH

Pipenv–specific environment variables:

Debug–specific environment variables:

  • PATH: /usr/local/Cellar/pipenv/2018.7.1/libexec/tools:/usr/local/bin:/Users/<user>/.pyenv/libexec:/Users/<user>/.pyenv/plugins/python-build/bin:/Users/<user>/.pyenv/plugins/pyenv-virtualenv/bin:/Users/<user>/.pyenv/plugins/pyenv-update/bin:/Users/<user>/.pyenv/plugins/pyenv-installer/bin:/Users/<user>/.pyenv/plugins/pyenv-doctor/bin:/Users/<user>/.local/bin:/usr/local/opt/coreutils/libexec/gnubin:/usr/local/opt/libarchive/bin:/usr/local/sbin:/usr/local/opt/go/libexec/bin:/Users/<user>/go/bin:/Users/<user>/.pyenv/plugins/pyenv-virtualenv/shims:/Users/<user>/.pyenv/shims:/Users/<user>/.pyenv/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/VMware Fusion.app/Contents/Public:/usr/local/MacGPG2/bin:/opt/X11/bin:/Users/<user>/.vim/plugged/fzf/bin
  • SHELL: /usr/local/bin/bash
  • EDITOR: vim
  • LANG: en_US.UTF-8
  • PWD: /Users/<user>/Documents/Source/pipenvdeps

Contents of Pipfile ('/Users//Documents/Source/pipenvdeps/Pipfile'):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
pylint = "==1.9"
pylint-quotes = "==0.1.9"

[dev-packages]

[requires]
python_version = "3.7"

Contents of Pipfile.lock ('/Users//Documents/Source/pipenvdeps/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "8401a941de091e385a3679312b16f3d165c68c9f8d79c6963b1d3e3a741dbe53"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.7"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "astroid": {
            "hashes": [
                "sha256:0ef2bf9f07c3150929b25e8e61b5198c27b0dca195e156f0e4d5bdd89185ca1a",
                "sha256:fc9b582dba0366e63540982c3944a9230cbc6f303641c51483fa547dcc22393a"
            ],
            "version": "==1.6.5"
        },
        "isort": {
            "hashes": [
                "sha256:1153601da39a25b14ddc54955dbbacbb6b2d19135386699e2ad58517953b34af",
                "sha256:b9c40e9750f3d77e6e4d441d8b0266cf555e7cdabdcff33c4fd06366ca761ef8",
                "sha256:ec9ef8f4a9bc6f71eec99e1806bfa2de401650d996c59330782b89a5555c1497"
            ],
            "markers": "python_version != '3.3.*' and python_version >= '2.7' and python_version != '3.1.*' and python_version != '3.0.*' and python_version != '3.2.*'",
            "version": "==4.3.4"
        },
        "lazy-object-proxy": {
            "hashes": [
                "sha256:0ce34342b419bd8f018e6666bfef729aec3edf62345a53b537a4dcc115746a33",
                "sha256:1b668120716eb7ee21d8a38815e5eb3bb8211117d9a90b0f8e21722c0758cc39",
                "sha256:209615b0fe4624d79e50220ce3310ca1a9445fd8e6d3572a896e7f9146bbf019",
                "sha256:27bf62cb2b1a2068d443ff7097ee33393f8483b570b475db8ebf7e1cba64f088",
                "sha256:27ea6fd1c02dcc78172a82fc37fcc0992a94e4cecf53cb6d73f11749825bd98b",
                "sha256:2c1b21b44ac9beb0fc848d3993924147ba45c4ebc24be19825e57aabbe74a99e",
                "sha256:2df72ab12046a3496a92476020a1a0abf78b2a7db9ff4dc2036b8dd980203ae6",
                "sha256:320ffd3de9699d3892048baee45ebfbbf9388a7d65d832d7e580243ade426d2b",
                "sha256:50e3b9a464d5d08cc5227413db0d1c4707b6172e4d4d915c1c70e4de0bbff1f5",
                "sha256:5276db7ff62bb7b52f77f1f51ed58850e315154249aceb42e7f4c611f0f847ff",
                "sha256:61a6cf00dcb1a7f0c773ed4acc509cb636af2d6337a08f362413c76b2b47a8dd",
                "sha256:6ae6c4cb59f199d8827c5a07546b2ab7e85d262acaccaacd49b62f53f7c456f7",
                "sha256:7661d401d60d8bf15bb5da39e4dd72f5d764c5aff5a86ef52a042506e3e970ff",
                "sha256:7bd527f36a605c914efca5d3d014170b2cb184723e423d26b1fb2fd9108e264d",
                "sha256:7cb54db3535c8686ea12e9535eb087d32421184eacc6939ef15ef50f83a5e7e2",
                "sha256:7f3a2d740291f7f2c111d86a1c4851b70fb000a6c8883a59660d95ad57b9df35",
                "sha256:81304b7d8e9c824d058087dcb89144842c8e0dea6d281c031f59f0acf66963d4",
                "sha256:933947e8b4fbe617a51528b09851685138b49d511af0b6c0da2539115d6d4514",
                "sha256:94223d7f060301b3a8c09c9b3bc3294b56b2188e7d8179c762a1cda72c979252",
                "sha256:ab3ca49afcb47058393b0122428358d2fbe0408cf99f1b58b295cfeb4ed39109",
                "sha256:bd6292f565ca46dee4e737ebcc20742e3b5be2b01556dafe169f6c65d088875f",
                "sha256:cb924aa3e4a3fb644d0c463cad5bc2572649a6a3f68a7f8e4fbe44aaa6d77e4c",
                "sha256:d0fc7a286feac9077ec52a927fc9fe8fe2fabab95426722be4c953c9a8bede92",
                "sha256:ddc34786490a6e4ec0a855d401034cbd1242ef186c20d79d2166d6a4bd449577",
                "sha256:e34b155e36fa9da7e1b7c738ed7767fc9491a62ec6af70fe9da4a057759edc2d",
                "sha256:e5b9e8f6bda48460b7b143c3821b21b452cb3a835e6bbd5dd33aa0c8d3f5137d",
                "sha256:e81ebf6c5ee9684be8f2c87563880f93eedd56dd2b6146d8a725b50b7e5adb0f",
                "sha256:eb91be369f945f10d3a49f5f9be8b3d0b93a4c2be8f8a5b83b0571b8123e0a7a",
                "sha256:f460d1ceb0e4a5dcb2a652db0904224f367c9b3c1470d5a7683c0480e582468b"
            ],
            "version": "==1.3.1"
        },
        "mccabe": {
            "hashes": [
                "sha256:ab8a6258860da4b6677da4bd2fe5dc2c659cff31b3ee4f7f5d64e79735b80d42",
                "sha256:dd8d182285a0fe56bace7f45b5e7d1a6ebcbf524e8f3bd87eb0f125271b8831f"
            ],
            "version": "==0.6.1"
        },
        "pylint": {
            "hashes": [
                "sha256:b719c86a7395ea0c0ec8030c2a7a2b4fad573ee50460f9948fabb1811d72094f",
                "sha256:cf1be367296e9e534a5cb420186ce99f63f17c2b855fcb4321a3e20ce51502cd"
            ],
            "index": "pypi",
            "version": "==1.9"
        },
        "six": {
            "hashes": [
                "sha256:70e8a77beed4562e7f14fe23a786b54f6296e34344c23bc42f07b15018ff98e9",
                "sha256:832dc0e10feb1aa2c68dcc57dbb658f1c7e65b9b61af69048abc87a2db00a0eb"
            ],
            "version": "==1.11.0"
        },
        "wrapt": {
            "hashes": [
                "sha256:d4d560d479f2c21e1b5443bbd15fe7ec4b37fe7e53d335d3b9b0a7b1226fe3c6"
            ],
            "version": "==1.10.11"
        }
    },
    "develop": {}
}

Metadata

Metadata

Assignees

No one assigned
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions