pipenv lock --pre does not create a lockfile with prerelease version

[nid5]

Issue description

When running pipenv lock --pre with a non-specific version in the Pipfile, only release versions are included in the Pipfile.lock

Expected result

Prerelease version in Pipfile.lock

Actual result

Release versions in Pipfile.lock

Steps to replicate

Pipfile containing non-specific version of a package, e.g., mylibrary = ">=1.2.3"

Pipenv lock -pre

Examine Pipfile.lock

In my attempts at debugging this issue I placed breakpoints in a variety of files. It seems that in my case, the issue was occurring around

pipenv/pipenv/utils.py

Lines 774 to 779 in bf72031

	if self._repository is None:
	from pipenv.patched.piptools.repositories.pypi import PyPIRepository
	self._repository = PyPIRepository(
	self.pip_args, use_json=False, session=self.session,
	build_isolation=self.pip_options.build_isolation
	)

In my case self._repository is none, so a new PypiRepository is created. Examining PypiRepository shows that the self.options field, which is later referenced in the package finder when determining whether or not to use prereleases, is set using the 'pip_args' passed when the PypiRepository is initialized. In my case the pip_args passed contained no reference to prereleases, despite the fact that self.pre (referencing the Resolver object) is set to True. Uncertain if there is a case where self._repository already exists at this point, but I couldn't replicate a situation where it existed.

$ pipenv --support

Pipenv version: '2020.11.15'

Pipenv location: '/Users/nid5/.pyenv/versions/3.7.8/lib/python3.7/site-packages/pipenv'

Python location: '/Users/nid5/.pyenv/versions/3.7.8/bin/python3.7'

Python installations found:

• 3.8.2: /usr/bin/python3
• 3.7.8: /Users/nid5/.pyenv/versions/3.7.8/bin/python3
• 3.7.8: /Users/nid5/.pyenv/versions/3.7.8/bin/python3
• 3.7.8: /Users/nid5/.pyenv/versions/3.7.8/bin/python3.7m
• 3.7.8: /Users/nid5/.pyenv/versions/3.7.8/bin/python3.7
• 2.7.16: /usr/bin/python2
• 2.7.16: /usr/bin/python2.7

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.7.8',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '19.6.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 19.6.0: Thu Jun 18 20:49:00 PDT '
                     '2020; root:xnu-6153.141.1~1/RELEASE_X86_64',
 'python_full_version': '3.7.8',
 'python_version': '3.7',
 'sys_platform': 'darwin'}

System environment variables:

• VIRTUALENVWRAPPER_SCRIPT
• TERM_PROGRAM
• PYENV_ROOT
• TERM
• SHELL
• TMPDIR
• TERM_PROGRAM_VERSION
• TERM_SESSION_ID
• PYENV_VERSION
• ZSH
• USER
• SSH_AUTH_SOCK
• PYENV_DIR
• __CF_USER_TEXT_ENCODING
• VIRTUAL_ENV_DISABLE_PROMPT
• PAGER
• LSCOLORS
• PATH
• LaunchInstanceID
• PWD
• EDITOR
• LANG
• ITERM_PROFILE
• PYENV_HOOK_PATH
• XPC_FLAGS
• XPC_SERVICE_NAME
• PYENV_SHELL
• SHLVL
• HOME
• COLORFGBG
• _VIRTUALENVWRAPPER_API
• LC_TERMINAL_VERSION
• ITERM_SESSION_ID
• LESS
• LOGNAME
• LC_CTYPE
• LC_TERMINAL
• SECURITYSESSIONID
• COLORTERM
• PIP_DISABLE_PIP_VERSION_CHECK
• PYTHONDONTWRITEBYTECODE
• PIP_SHIMS_BASE_MODULE
• PIP_PYTHON_PATH
• PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

• PATH:
• SHELL: /bin/zsh
• EDITOR: /Applications/Atom
• LANG: en_US.UTF-8
• PWD: /Users/nid5/work/delete_this_soon_1111111

---

Contents of Pipfile:

[[source]]
name = "pipsource"
verify_ssl = true
url = "some.devpi.com"
# Before running Pipenv install, rename or copy this file to "Pipfile" and
# modify the values and/or set environment variables for development, testing,
# or deployment purposes as needed (see Pandex documentation for details).



[packages]
mylibrary = ">=0.7.2"


[requires]
python_version = "3.7"

Contents of Pipfile.lock :

{
    "_meta": {
        "hash": {
            "sha256": "dc79f482344bacd8eaa7b32d799857c8569136fdda41e3fd13422b407720eaba"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.7"
        },
        "sources": [
            {
                "name": "pipsource",
                "url": "some.devpi.com",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "certifi": {
            "hashes": [
                "sha256:1a4995114262bffbc2413b159f2a1a480c969de6e6eb13ee966d470af86af59c",
                "sha256:719a74fb9e33b9bd44cc7f3a8d94bc35e4049deebe19ba7d8e108280cfd59830"
            ],
            "version": "==2020.12.5"
        },
        "chardet": {
            "hashes": [
                "sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae",
                "sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691"
            ],
            "version": "==3.0.4"
        },
        "idna": {
            "hashes": [
                "sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6",
                "sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
            "version": "==2.10"
        },
        "jsonpickle": {
            "hashes": [
                "sha256:d0c5a4e6cb4e58f6d5406bdded44365c2bcf9c836c4f52910cc9ba7245a59dc2",
                "sha256:d3e922d781b1d0096df2dad89a2e1f47177d7969b596aea806a9d91b4626b29b"
            ],
            "version": "==1.2"
        },
        "prometheus-client": {
            "hashes": [
                "sha256:983c7ac4b47478720db338f1491ef67a100b474e3bc7dafcbaefb7d0b8f9b01c",
                "sha256:c6e6b706833a6bd1fd51711299edee907857be10ece535126a158f911ee80915"
            ],
            "version": "==0.8.0"
        },
        "pydantic": {
            "hashes": [
                "sha256:176885123dfdd8f7ab6e7ba1b66d4197de75ba830bb44d921af88b3d977b8aa5",
                "sha256:2b32a5f14558c36e39aeefda0c550bfc0f47fc32b4ce16d80dc4df2b33838ed8",
                "sha256:2eab7d548b0e530bf65bee7855ad8164c2f6a889975d5e9c4eefd1e7c98245dc",
                "sha256:479ca8dc7cc41418751bf10302ee0a1b1f8eedb2de6c4f4c0f3cf8372b204f9a",
                "sha256:59235324dd7dc5363a654cd14271ea8631f1a43de5d4fc29c782318fcc498002",
                "sha256:87673d1de790c8d5282153cab0b09271be77c49aabcedf3ac5ab1a1fd4dcbac0",
                "sha256:8a8e089aec18c26561e09ee6daf15a3cc06df05bdc67de60a8684535ef54562f",
                "sha256:b60f2b3b0e0dd74f1800a57d1bbd597839d16faf267e45fa4a5407b15d311085",
                "sha256:c0da48978382c83f9488c6bbe4350e065ea5c83e85ca5cfb8fa14ac11de3c296",
                "sha256:cbe284bd5ad67333d49ecc0dc27fa52c25b4c2fe72802a5c060b5f922db58bef",
                "sha256:d03df07b7611004140b0fef91548878c2b5f48c520a8cb76d11d20e9887a495e",
                "sha256:d4bb6a75abc2f04f6993124f1ed4221724c9dc3bd9df5cb54132e0b68775d375",
                "sha256:dacb79144bb3fdb57cf9435e1bd16c35586bc44256215cfaa33bf21565d926ae",
                "sha256:dd9359db7644317898816f6142f378aa48848dcc5cf14a481236235fde11a148"
            ],
            "markers": "python_version >= '3.6'",
            "version": "==1.3"
        },
        "python-dotenv": {
            "hashes": [
                "sha256:440c7c23d53b7d352f9c94d6f70860242c2f071cf5c029dd661ccb22d64ae42b",
                "sha256:f254bfd0c970d64ccbb6c9ebef3667ab301a71473569c991253a481f1c98dddc"
            ],
            "version": "==0.10.5"
        },
        "redis": {
            "hashes": [
                "sha256:0e7e0cfca8660dea8b7d5cd8c4f6c5e29e11f31158c0b0ae91a397f00e5a05a2",
                "sha256:432b788c4530cfe16d8d943a09d40ca6c16149727e4afe8c2c9d5580c59d9f24"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
            "version": "==3.5.3"
        },
        "requests": {
            "hashes": [
                "sha256:b3559a131db72c33ee969480840fff4bb6dd111de7dd27c8ee1f820f4f00231b",
                "sha256:fe75cc94a9443b9246fc7049224f75604b113c36acb93f87b80ed42c44cbb898"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
            "version": "==2.24.0"
        },
        "urllib3": {
            "hashes": [
                "sha256:8d7eaa5a82a1cac232164990f04874c594c9453ec55eef02eab885aa02fc17a2",
                "sha256:f5321fbe4bf3fefa0efd0bfe7fb14e90909eb62a48ccda331726b4319897dd5e"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4' and python_version < '4'",
            "version": "==1.25.11"
        },
        "walrus": {
            "hashes": [
                "sha256:9d257a95259c3662c53d345c8940494c7f1bf84e4dc940364de83b91cfdbfbfb"
            ],
            "version": "==0.8.1"
        }
    },
    "develop": {}
}

[nid5]

Reverting to version 2018.11.26 solves this, it seems quite a bit has changed in the utils.py since then.

[frostming]

Your debug conclusion seems correct to me. Care to file a PR to fix it? I am happy to review.

Uncertain if there is a case where self._repository already exists at this point, but I couldn't replicate a situation where it existed.

No, it is just a common pattern in this codebase that acts like a cached_property, which is evaluated only once throughout the process.

[nid5]

Pull request created here:
#4652

[matteius]

This was merged a while ago and hasn't been an issue on newer versions.