Pipenv tries to resolve packages where their markers evaluates to false

[gnaggnoyil]

Issue description

In Pipfile, packages that were expected to become a dependency only if marker expressions evaluates to true, has unexpectedly become a dependecy when markers evaluates to false

Expected result

Packages won't be resolved and installed under environment where markers evaluates to false.

Actual result

Pipenv tries to resolve those packages

Steps to replicate

OS is Archlinux

mkdir -p ~/temp
cat > ~/temp/Pipfile <<EOF
[packages]
pywin32 = {version = "==227", markers="sys_platform == 'win32'"}
EOF
pipenv install --verbose

python 3.8 and pipenv are installed through Archlinux offical software repositories and its versions are:

% pacman -Qs | grep -E "(/python-pipenv )|(/python )"
local/python 3.8.3-1
local/python-pipenv 2020.6.2-1

Contents of `~/.pip/pip.conf`

[global]
trusted-host = mirrors.aliyun.com
index-url = https://pypi.doubanio.com/simple/
extra-index-url = 
	https://mirrors.aliyun.com/pypi/simple/
	https://pypi.tuna.tsinghua.edu.cn/simple

$ pipenv --support

Pipenv version: '2020.6.2'

Pipenv location: '/usr/lib/python3.8/site-packages/pipenv'

Python location: '/usr/bin/python'

Python installations found:

• 3.8.3: /usr/bin/python3
• 3.8.3: /usr/bin/python3.8
• 2.7.18: /usr/bin/python2
• 2.7.18: /usr/bin/python2.7

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.8.3',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '5.7.2-arch1-1',
 'platform_system': 'Linux',
 'platform_version': '#1 SMP PREEMPT Wed, 10 Jun 2020 20:36:24 +0000',
 'python_full_version': '3.8.3',
 'python_version': '3.8',
 'sys_platform': 'linux'}

System environment variables:

• INVOCATION_ID
• SSH_AUTH_SOCK
• LANGUAGE
• LANG
• SSH_AGENT_PID
• XDG_CURRENT_DESKTOP
• GRADLE_HOME
• ANDROID_HOME
• WINDOWID
• HG
• CUDA_PATH
• SESSION_MANAGER
• QT_IM_MODULE
• DESKTOP_SESSION
• USER
• XDG_MENU_PREFIX
• HOME
• OLDPWD
• QT4_IM_MODULE
• DBUS_SESSION_BUS_ADDRESS
• XDG_VTNR
• XDG_SEAT
• GTK_MODULES
• XDG_CONFIG_DIRS
• WINDOWPATH
• GTK_IM_MODULE
• LC_TYPE
• VTE_VERSION
• JOURNAL_STREAM
• MAIL
• LOGNAME
• LOCALE
• UBUNTU_MENUPROXY
• XDG_RUNTIME_DIR
• XMODIFIERS
• SHELL
• XDG_SESSION_TYPE
• PANEL_GDK_CORE_DEVICE_EVENTS
• XDG_SESSION_ID
• XDG_DATA_DIRS
• PATH
• QT_QPA_PLATFORMTHEME
• SHLVL
• PWD
• XAUTHORITY
• COLORTERM
• XDG_SESSION_CLASS
• TERM
• DISPLAY
• EDITOR
• PAGER
• LS_COLORS
• LESS_TERMCAP_mb
• LESS_TERMCAP_md
• LESS_TERMCAP_me
• LESS_TERMCAP_se
• LESS_TERMCAP_so
• LESS_TERMCAP_ue
• LESS_TERMCAP_us
• npm_config_prefix
• CONDA_EXE
• _CE_M
• _CE_CONDA
• CONDA_PYTHON_EXE
• CONDA_SHLVL
• _
• PIP_DISABLE_PIP_VERSION_CHECK
• PYTHONDONTWRITEBYTECODE
• PIP_SHIMS_BASE_MODULE
• PIP_PYTHON_PATH
• PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

• PATH: /home/gnaggnoyil/anaconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/android-sdk/emulator:/opt/android-sdk/build-tools/18.0.1/:/opt/android-sdk/platform-tools:/opt/android-sdk/tools:/opt/android-sdk/tools/bin:/opt/cuda/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/home/gnaggnoyil/.dotnet/tools:/home/gnaggnoyil/.npm/node_modules/bin
• SHELL: /bin/zsh
• EDITOR: vim
• LANG: zh_CN.UTF-8
• PWD: /home/gnaggnoyil/temp

---

Contents of Pipfile ('/home/gnaggnoyil/temp/Pipfile'):

[packages]
pywin32 = {version = "==227", markers="sys_platform == 'win32'"}

$ pipenv install --verbose

Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Building requirements...
Resolving dependencies...
⠴ Locking...Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/pipenv/resolver.py", line 807, in <module>
✘ Locking Failed! 
    main()
  File "/usr/lib/python3.8/site-packages/pipenv/resolver.py", line 802, in main
    _main(parsed.pre, parsed.clear, parsed.verbose, parsed.system, parsed.write,
  File "/usr/lib/python3.8/site-packages/pipenv/resolver.py", line 785, in _main
    resolve_packages(pre, clear, verbose, system, write, requirements_dir, packages)
  File "/usr/lib/python3.8/site-packages/pipenv/resolver.py", line 746, in resolve_packages
    results, resolver = resolve(
  File "/usr/lib/python3.8/site-packages/pipenv/resolver.py", line 728, in resolve
    return resolve_deps(
  File "/usr/lib/python3.8/site-packages/pipenv/utils.py", line 1378, in resolve_deps
    results, hashes, markers_lookup, resolver, skipped = actually_resolve_deps(
  File "/usr/lib/python3.8/site-packages/pipenv/utils.py", line 1090, in actually_resolve_deps
    resolver = Resolver.create(
  File "/usr/lib/python3.8/site-packages/pipenv/utils.py", line 647, in create
    constraints, skipped, index_lookup, markers_lookup = cls.get_metadata(
  File "/usr/lib/python3.8/site-packages/pipenv/utils.py", line 462, in get_metadata
    constraint_update, lockfile_update = cls.get_deps_from_req(
  File "/usr/lib/python3.8/site-packages/pipenv/utils.py", line 610, in get_deps_from_req
    best_match = pypi.find_best_match(req.ireq) if pypi else None
  File "/usr/lib/python3.8/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 202, in find_best_match
    raise NoCandidateFound(ireq, all_candidates, self.finder)
pipenv.patched.piptools.exceptions.NoCandidateFound: Could not find a version that matches pywin32==227
No versions found
Were https://pypi.org/simple or https://mirrors.aliyun.com/pypi/simple/ or https://pypi.tuna.tsinghua.edu.cn/simple reachable?
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/pipenv/resolver.py", line 807, in <module>
    main()
  File "/usr/lib/python3.8/site-packages/pipenv/resolver.py", line 802, in main
    _main(parsed.pre, parsed.clear, parsed.verbose, parsed.system, parsed.write,
  File "/usr/lib/python3.8/site-packages/pipenv/resolver.py", line 785, in _main
    resolve_packages(pre, clear, verbose, system, write, requirements_dir, packages)
  File "/usr/lib/python3.8/site-packages/pipenv/resolver.py", line 746, in resolve_packages
    results, resolver = resolve(
  File "/usr/lib/python3.8/site-packages/pipenv/resolver.py", line 728, in resolve
    return resolve_deps(
  File "/usr/lib/python3.8/site-packages/pipenv/utils.py", line 1378, in resolve_deps
    results, hashes, markers_lookup, resolver, skipped = actually_resolve_deps(
  File "/usr/lib/python3.8/site-packages/pipenv/utils.py", line 1090, in actually_resolve_deps
    resolver = Resolver.create(
  File "/usr/lib/python3.8/site-packages/pipenv/utils.py", line 647, in create
    constraints, skipped, index_lookup, markers_lookup = cls.get_metadata(
  File "/usr/lib/python3.8/site-packages/pipenv/utils.py", line 462, in get_metadata
    constraint_update, lockfile_update = cls.get_deps_from_req(
  File "/usr/lib/python3.8/site-packages/pipenv/utils.py", line 610, in get_deps_from_req
    best_match = pypi.find_best_match(req.ireq) if pypi else None
  File "/usr/lib/python3.8/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 202, in find_best_match
    raise NoCandidateFound(ireq, all_candidates, self.finder)
pipenv.patched.piptools.exceptions.NoCandidateFound: Could not find a version that matches pywin32==227
No versions found
Were https://pypi.org/simple or https://mirrors.aliyun.com/pypi/simple/ or https://pypi.tuna.tsinghua.edu.cn/simple reachable?

[gnaggnoyil]

Ping.
Any advices yet?

[alexrkopp]

I, too, am running into this issue... except I have

pywin32 = {version = "*", markers = "platform_system == 'Windows'"}

Same error, though.

[daxaxelrod]

Same here using WLS with sys_platform == 'win32'

[Inveracity]

Exact same issue on Ubuntu 18.04.

My workaround so far have been to do pipenv lock on a Windows machine and then pipenv install from the Linux machine to get it to install a new package.

This is not a viable approach for everyone unless they happen to have Windows readily available

[frostming]

I don't think this is an issue, the lock file should be generated correctly for all possible platforms. If a package can't be resolved on a specific platform, it should throw an error telling users "hey, I can't finish my job, you may need to resolve this on the right platform". Instead of skipping that package and leave an incomplete lock file.

[gnaggnoyil]

@frostming I'm sorry but I don't quite understand your meaning of "generate correctly for all possible platforms". How does it have anything to do with platform-specific markers?