New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unexpected Dependancy Graph Issue #2529

Closed
JBKahn opened this Issue Jul 5, 2018 · 14 comments

Comments

Projects
None yet
8 participants
@JBKahn
Contributor

JBKahn commented Jul 5, 2018

Issue description

I had a requirement pinned to "<4.0.0" and a sub-dependancy that required it to be "==3.3.0". I checked the graph and as expected, that was the only dependancy that required the protobuf package.

That gave me a dependancy resolution error of:
Could not find a version that matches protobuf<4.0.0,==3.3.0,==3.6.0

Expected result

I expected it to resolve to "==3.3.0"

Actual result

I was able to get it to be happy by changing my local "<4.0.0" to "==3.3.0".

Steps to replicate

I will fill this is later today, but wanted to spell out the issue just in case it is known.

Provide the steps to replicate (which usually at least includes the commands and the Pipfile).


Please run $ pipenv --support, and paste the results here. Don't put backticks (`) around it! The output already contains Markdown formatting.

If you're on macOS, run the following:

$ pipenv --support | pbcopy

If you're on Windows, run the following:

> pipenv --support | clip
@erinxocon

This comment has been minimized.

Contributor

erinxocon commented Jul 12, 2018

Hey @JBKahn Any update on this?

@JBKahn

This comment has been minimized.

Contributor

JBKahn commented Jul 13, 2018

I had been slammed at work, it seems to be possibly related to my build environment as this didn't happen on my machine but inside an alpine python docker container. Was going to dig in before I included all the details.

@teeberg

This comment has been minimized.

teeberg commented Jul 26, 2018

I think I'm experiencing the same issue with a different package:

Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
Hint: try $ pipenv lock --pre if it is a pre-release dependency.
Could not find a version that matches botocore<1.11.0,==1.10.26,>=1.10.26,>=1.10.64 (from -r /var/folders/h4/n2jm49b97dx5d89gkkt4yxkm0000gn/T/pipenv-KF21xr-requirements/pipenv-Vk6UhE-constraints.txt (line 34))

and it's true, ==1.10.26 and >=1.10.64 contradict themselves, but no package seems to be pulling in >=1.10.64?!

After running the suggested pipenv install --skip-lock and inspecting the output of pipenv graph, here are botocore's dependency specs:

> pipenv graph | grep -i botocore
  - botocore [required: ==1.10.26, installed: 1.10.26]
    - botocore [required: >=1.3.0,<2.0.0, installed: 1.10.26]
  - botocore [required: >=1.0.1, installed: 1.10.26]
    - botocore [required: >=1.10.26,<1.11.0, installed: 1.10.26]
      - botocore [required: >=1.3.0,<2.0.0, installed: 1.10.26]

I'm just investigating how much work it is to switch over to using pipenv from a workflow using pip-compile, so I have another virtual env set up with exactly these requirements as specified, and using pipdeptree in there doesn't show anything specifying >=1.10.64 either:

> pipdeptree -r -p botocore | grep "requires: botocore"
  - awscli==1.15.26 [requires: botocore==1.10.26]
  - awsebcli==3.14.3 [requires: botocore>=1.0.1]
  - boto3==1.7.26 [requires: botocore>=1.10.26,<1.11.0]
  - s3transfer==0.1.12 [requires: botocore>=1.3.0,<2.0.0]

So, maybe this comes down to, how do I find out where that bogus version spec comes from?

@JBKahn

This comment has been minimized.

Contributor

JBKahn commented Jul 26, 2018

yeah, the real difficulty is that when there is a collision, the output isn't showing where the different requirements come from. I've been super busy these past few days but I'll try to get to this ASAP.

@mr-bo-jangles

This comment has been minimized.

mr-bo-jangles commented Aug 3, 2018

yeah, the real difficulty is that when there is a collision, the output isn't showing where the different requirements come from

Literally dealing with this right now, it's impossible to track down which package is causing issues

@mr-bo-jangles

This comment has been minimized.

mr-bo-jangles commented Aug 3, 2018

$ pipenv --support

Pipenv version: '2018.7.1'

Pipenv location: '/home/josh/.virtualenvs/savings-champion/lib/python3.6/site-packages/pipenv'

Python location: '/home/josh/.virtualenvs/savings-champion/bin/python'

Other Python installations in PATH:

  • 2.7: /usr/bin/python2.7

  • 2.7: /usr/bin/python2.7

  • 3.6: /usr/bin/python3.6m

  • 3.6: /home/josh/.virtualenvs/savings-champion/bin/python3.6

  • 3.6: /usr/bin/python3.6

  • 3.6.6: /home/josh/.virtualenvs/savings-champion/bin/python

  • 3.6.6: /usr/bin/python

  • 2.7.15: /usr/bin/python2

  • 3.6.6: /home/josh/.virtualenvs/savings-champion/bin/python3

  • 3.6.6: /usr/bin/python3

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.6.6',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '4.17.11-arch1',
 'platform_system': 'Linux',
 'platform_version': '#1 SMP PREEMPT Sun Jul 29 10:11:16 UTC 2018',
 'python_full_version': '3.6.6',
 'python_version': '3.6',
 'sys_platform': 'linux'}

System environment variables:

  • PATH
  • XAUTHORITY
  • GDMSESSION
  • XDG_DATA_DIRS
  • ZDOTDIR
  • KDE_SESSION_UID
  • TERM
  • DBUS_SESSION_BUS_ADDRESS
  • LANG
  • XDG_SESSION_TYPE
  • XDG_SESSION_ID
  • XDG_CURRENT_DESKTOP
  • QT_AUTO_SCREEN_SCALE_FACTOR
  • DISPLAY
  • MAIL
  • MOZ_PLUGIN_PATH
  • SESSION_MANAGER
  • USERNAME
  • LOGNAME
  • PWD
  • _
  • XCURSOR_THEME
  • LANGUAGE
  • KDE_SESSION_VERSION
  • SHELL
  • GDM_LANG
  • XCURSOR_SIZE
  • DESKTOP_SESSION
  • OLDPWD
  • USER
  • DESKTOP_STARTUP_ID
  • KDE_FULL_SESSION
  • WINDOWPATH
  • XDG_SEAT
  • GTK_MODULES
  • GS_LIB
  • LC_ALL
  • KWIN_TRIPLE_BUFFER
  • XDG_RUNTIME_DIR
  • XDG_SESSION_DESKTOP
  • XDG_VTNR
  • SHLVL
  • HOME
  • EDITOR
  • VISUAL
  • PAGER
  • LESS
  • LESS_TERMCAP_mb
  • LESS_TERMCAP_md
  • LESS_TERMCAP_me
  • LESS_TERMCAP_se
  • LESS_TERMCAP_so
  • LESS_TERMCAP_ue
  • LESS_TERMCAP_us
  • LS_COLORS
  • GREP_COLOR
  • GREP_COLORS
  • VIRTUAL_ENV
  • PS1
  • PYTHONDONTWRITEBYTECODE
  • PIP_PYTHON_PATH

Pipenv–specific environment variables:

Debug–specific environment variables:

  • PATH: /home/josh/.virtualenvs/savings-champion/bin:/usr/local/bin:/usr/local/sbin:/home/josh/go/bin:/home/josh/.yarn/bin:/usr/bin:/opt/cuda/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl
  • SHELL: /usr/bin/zsh
  • EDITOR: nano
  • LANG: en_GB.UTF-8
  • PWD: /home/josh/PycharmProjects/savings-champion
  • VIRTUAL_ENV: /home/josh/.virtualenvs/savings-champion

Contents of Pipfile ('/home/josh/PycharmProjects/savings-champion/Pipfile'):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
lxml = "*"
nose = "*"
django-extensions = "*"
django-tinymce = "*"
sorl-thumbnail = "*"
"suds-py3" = "*"
feedparser = "*"
python-dateutil = "*"
python-memcached = "*"
django-pipeline = "*"
requests = "*"
django-pipeline-forgiving = "*"
django-ckeditor = "*"
django-filebrowser = "*"
raven = "*"
celery = {version = "*", extras = ["redis"]}
"psycopg2-binary" = "*"
django-crispy-forms = "*"
pyotp = "*"
newrelic = "*"
djangorestframework = "*"
djangorestframework-xml = "*"
djangorestframework-jwt = "*"
django-filter = "*"
parsedatetime = "*"
simple-salesforce = "*"
django-allauth = "*"
apache-libcloud = "*"
django-storages = "*"
"boto3" = "*"
sortedcontainers = "*"
redis = "*"
hiredis = "*"
msgpack-python = "*"
django-waffle = "*"
django-report-builder = "*"
django-markupfield = "*"
django-cors-headers = "*"
arrow = "*"
django-redis = "*"
"mailchimp3" = "*"
dotmailer = {editable = true, git = "https://github.com/savingschampion/dotmailer.git"}
simplejson = "*"
zeep = "*"
djangorestframework-jsonp = "*"
requests-oauthlib = "*"
django-ipware = "*"
hypothesis = {version = "*", extras = ["pytz"]}
drf-yasg = {version = "*", extras = ["validation"]}
Django = ">=2.0"
Pillow = "*"
uWSGI = "*"
"Django-Select2" = "*"
Collectfast = "*"
Markdown = "*"
Faker = "*"
rest_condition = "*"
django-rest-knox = "*"
django-simple-history = "*"
wagtail = "*"
wagtail-metadata-mixin = "*"

[dev-packages]

[requires]
python_version = "3.6"
Locking [dev-packages] dependencies...
Locking [packages] dependencies...

Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
Could not find a version that matches Django<2.1,==2.1,>=1.11,>=1.11.7,>=1.3,>=1.8,>=2.0 (from -r /tmp/pipenv-0s0uam0c-requirements/pipenv-7uweinlt-constraints.txt (line 51))
Tried: 1.1.3, 1.1.4, 1.2, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6, 1.2.7, 1.3, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.3.5, 1.3.6, 1.3.7, 1.4, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.4.7, 1.4.8, 1.4.9, 1.4.10, 1.4.11, 1.4.12, 1.4.13, 1.4.14, 1.4.15, 1.4.16, 1.4.17, 1.4.18, 1.4.19, 1.4.20, 1.4.21, 1.4.22, 1.5, 1.5.1, 1.5.2, 1.5.2, 1.5.3, 1.5.4, 1.5.5, 1.5.6, 1.5.7, 1.5.8, 1.5.8, 1.5.9, 1.5.10, 1.5.11, 1.5.12, 1.5.12, 1.6, 1.6, 1.6.1, 1.6.1, 1.6.2, 1.6.2, 1.6.3, 1.6.3, 1.6.4, 1.6.4, 1.6.5, 1.6.5, 1.6.6, 1.6.6, 1.6.7, 1.6.7, 1.6.8, 1.6.8, 1.6.9, 1.6.9, 1.6.10, 1.6.10, 1.6.11, 1.6.11, 1.7, 1.7, 1.7.1, 1.7.1, 1.7.2, 1.7.2, 1.7.3, 1.7.3, 1.7.4, 1.7.4, 1.7.5, 1.7.5, 1.7.6, 1.7.6, 1.7.7, 1.7.7, 1.7.8, 1.7.8, 1.7.9, 1.7.9, 1.7.10, 1.7.10, 1.7.11, 1.7.11, 1.8, 1.8, 1.8.1, 1.8.1, 1.8.2, 1.8.2, 1.8.3, 1.8.3, 1.8.4, 1.8.4, 1.8.5, 1.8.5, 1.8.6, 1.8.6, 1.8.7, 1.8.7, 1.8.8, 1.8.8, 1.8.9, 1.8.9, 1.8.10, 1.8.10, 1.8.11, 1.8.11, 1.8.12, 1.8.12, 1.8.13, 1.8.13, 1.8.14, 1.8.14, 1.8.15, 1.8.15, 1.8.16, 1.8.16, 1.8.17, 1.8.17, 1.8.18, 1.8.18, 1.8.19, 1.8.19, 1.9, 1.9, 1.9.1, 1.9.1, 1.9.2, 1.9.2, 1.9.3, 1.9.3, 1.9.4, 1.9.4, 1.9.5, 1.9.5, 1.9.6, 1.9.6, 1.9.7, 1.9.7, 1.9.8, 1.9.8, 1.9.9, 1.9.9, 1.9.10, 1.9.10, 1.9.11, 1.9.11, 1.9.12, 1.9.12, 1.9.13, 1.9.13, 1.10, 1.10, 1.10.1, 1.10.1, 1.10.2, 1.10.2, 1.10.3, 1.10.3, 1.10.4, 1.10.4, 1.10.5, 1.10.5, 1.10.6, 1.10.6, 1.10.7, 1.10.7, 1.10.8, 1.10.8, 1.11, 1.11, 1.11.1, 1.11.1, 1.11.2, 1.11.2, 1.11.3, 1.11.3, 1.11.4, 1.11.4, 1.11.5, 1.11.5, 1.11.6, 1.11.6, 1.11.7, 1.11.7, 1.11.8, 1.11.8, 1.11.9, 1.11.9, 1.11.10, 1.11.10, 1.11.11, 1.11.11, 1.11.12, 1.11.12, 1.11.13, 1.11.13, 1.11.14, 1.11.14, 1.11.15, 1.11.15, 2.0, 2.0, 2.0.1, 2.0.1, 2.0.2, 2.0.2, 2.0.3, 2.0.3, 2.0.4, 2.0.4, 2.0.5, 2.0.5, 2.0.6, 2.0.6, 2.0.7, 2.0.7, 2.0.8, 2.0.8, 2.1, 2.1
Skipped pre-versions: 1.8a1, 1.8b1, 1.8b2, 1.8rc1, 1.9a1, 1.9b1, 1.9rc1, 1.9rc2, 1.10a1, 1.10a1, 1.10b1, 1.10b1, 1.10rc1, 1.10rc1, 1.11a1, 1.11b1, 1.11rc1, 1.11rc1, 2.0a1, 2.0b1, 2.0rc1, 2.1a1, 2.1b1, 2.1rc1
There are incompatible versions in the resolved dependencies.

I have no idea where the ==2.1 comes from, listing of all requirements from graph with Django in the name

  - Django [required: >=1.8, installed: 2.0.7]
  - django-storages [required: >=1.6, installed: 1.6.6]
django-allauth==0.36.0
  - Django [required: >=1.11, installed: 2.0.7]
django-ckeditor==5.6.1
  - django-js-asset [required: Any, installed: 1.1.0]
django-cors-headers==2.4.0
django-crispy-forms==1.7.2
django-extensions==2.1.0
django-filebrowser==3.10.1
  - django-grappelli [required: >=2.11,<2.12, installed: 2.11.1]
django-filter==2.0.0
  - Django [required: >=1.11, installed: 2.0.7]
django-ipware==2.1.0
django-markupfield==1.4.3
django-pipeline-forgiving==1.0.0
  - django-pipeline [required: Any, installed: 1.6.14]
django-redis==4.9.0
  - Django [required: >=1.11, installed: 2.0.7]
django-report-builder==6.0.2
  - djangorestframework [required: >=3.1.0, installed: 3.8.2]
django-rest-knox==3.1.5
  - django [required: Any, installed: 2.0.7]
  - djangorestframework [required: Any, installed: 3.8.2]
django-select2==6.1.1
  - django-appconf [required: >=0.6.0, installed: 1.0.2]
django-simple-history==2.3.0
django-tinymce==2.7.0
django-url-filter==0.3.5
  - Django [required: >=1.8, installed: 2.0.7]
django-waffle==0.14.0
djangorestframework-jsonp==1.0.2
djangorestframework-jwt==1.11.0
djangorestframework-xml==1.3.0
  - Django [required: >=1.11.7, installed: 2.0.7]
  - djangorestframework [required: >=3.7.7, installed: 3.8.2]
  - django [required: >=1.3, installed: 2.0.7]
  - djangorestframework [required: Any, installed: 3.8.2]
  - django-meta [required: >=1.0, installed: 1.4.1]
    - Django [required: >=1.11,<2.1, installed: 2.0.7]
    - django-modelcluster [required: >=4.0,<5.0, installed: 4.1]
    - django-taggit [required: >=0.22.2,<1.0, installed: 0.22.2]
    - django-treebeard [required: >=4.2.0,<5.0, installed: 4.3]
      - Django [required: >=1.8, installed: 2.0.7]
    - djangorestframework [required: >=3.7.4,<4.0, installed: 3.8.2]
@uranusjr

This comment has been minimized.

Member

uranusjr commented Aug 3, 2018

It’s from Wagtail, you need to pin django<2.1. (I know this because we are working on this right now and this can be pulled out from the half-down code.)

And yes, I know the resolver should be able to pin this automatically, but right now it’s what it is.

@JBKahn

This comment has been minimized.

Contributor

JBKahn commented Aug 3, 2018

Is it easy to write tests for the resolver. I haven't taken a look at the code but I'd help out if it isn't too bad to get started.

@OliverHofkens

This comment has been minimized.

OliverHofkens commented Aug 29, 2018

I'm running into the same issue with boto3:

$ pipenv lock
Could not find a version that matches botocore<1.11.0,<1.12.0,>=1.10.84,>=1.11.3

Running pipenv install --skip-lock and inspecting the graph shows that no package seems to require <1.11.0

$ pipenv graph | grep botocore
    - botocore [required: >=1.11.3,<1.12.0, installed: 1.11.3]
      - botocore [required: >=1.3.0,<2.0.0, installed: 1.11.3]
        - botocore [required: >=1.11.3,<1.12.0, installed: 1.11.3]
          - botocore [required: >=1.3.0,<2.0.0, installed: 1.11.3]

@mjdunn

This comment has been minimized.

mjdunn commented Aug 29, 2018

@OliverHofkens see #2596. Running pipenv lock --clear helped me.

@OliverHofkens

This comment has been minimized.

OliverHofkens commented Aug 30, 2018

@mjdunn Great, thanks!

techalchemy added a commit that referenced this issue Oct 7, 2018

Add news fragments and final patch for all fixes
- Fixes #2499
- Fixes #2529
- Fixes #2589
- Fixes #2666
- Fixes #2767
- Fixes #2785
- Fixes #2795
- Fixes #2801
- Fixes #2802
- Fixes #2824
- Fixes #2862
- Fixes #2867
- Fixes #2879
- Fixes #2880
- Fixes #2894
- Fixes #2902
- Fixes #2924

Signed-off-by: Dan Ryan <dan@danryan.co>

techalchemy added a commit that referenced this issue Oct 7, 2018

Add news fragments and final patch for all fixes
- Fixes #2499
- Fixes #2529
- Fixes #2589
- Fixes #2666
- Fixes #2767
- Fixes #2785
- Fixes #2795
- Fixes #2801
- Fixes #2802
- Fixes #2824
- Fixes #2862
- Fixes #2867
- Fixes #2879
- Fixes #2880
- Fixes #2894
- Fixes #2902
- Fixes #2924

Signed-off-by: Dan Ryan <dan@danryan.co>
@teeberg

This comment has been minimized.

teeberg commented Oct 11, 2018

still seeing this, but with a different package now:

Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
 Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
Could not find a version that matches html5lib==1.0b10,>=1.0
Tried: 1.0-reupload, 0.2, 0.9, 0.10, 0.11, 0.11.1, 0.90, 0.95, 0.99, 0.999, 0.9999, 0.99999, 0.999999, 0.9999999, 0.99999999, 0.999999999, 0.999999999, 1.0.1, 1.0.1
Skipped pre-versions: 1.0b1, 1.0b2, 1.0b3, 1.0b5, 1.0b6, 1.0b7, 1.0b8, 1.0b9, 1.0b10, 1.0b10
There are incompatible versions in the resolved dependencies.

but according to pipdeptree | grep -i html5lib, nothing pulls in >=1.0:

    - html5lib [required: ==1.0b10, installed: 1.0b10]
  - html5lib [required: Any, installed: 1.0b10]
@techalchemy

This comment has been minimized.

Member

techalchemy commented Oct 11, 2018

With which version of pipenv

@teeberg

This comment has been minimized.

teeberg commented Oct 11, 2018

sorry, missed that! in 2018.10.9. I think that's the latest, right?

GilbertoCS added a commit to GilbertoCS/pipenv that referenced this issue Oct 15, 2018

Add news fragments and final patch for all fixes
- Fixes pypa#2499
- Fixes pypa#2529
- Fixes pypa#2589
- Fixes pypa#2666
- Fixes pypa#2767
- Fixes pypa#2785
- Fixes pypa#2795
- Fixes pypa#2801
- Fixes pypa#2802
- Fixes pypa#2824
- Fixes pypa#2862
- Fixes pypa#2867
- Fixes pypa#2879
- Fixes pypa#2880
- Fixes pypa#2894
- Fixes pypa#2902
- Fixes pypa#2924

Signed-off-by: Dan Ryan <dan@danryan.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment