Skip to content

--require-hashes does not correctly handle pinned package with extras #9644

Closed

Description

@tonybajan

What did you want to do?

Install this requirements.txt file with pip 21.0.1 or master in a new virtual environment.

A package is pinned to a non-latest version with an extra (here, requirements[security]) and another dependency requires this package without specifying the extra.

Output

Install fails with:

...
Collecting requests[security]==2.24.0
  Using cached requests-2.24.0-py2.py3-none-any.whl (61 kB)
Collecting six==1.15.0
  Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting urllib3==1.25.11
  Using cached urllib3-1.25.11-py2.py3-none-any.whl (127 kB)
Collecting requests<3
ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
    requests<3 from https://files.pythonhosted.org/packages/29/c1/24814557f1d22c56d50280771a17307e6bf87b70727d975fd6b2ce6b014a/requests-2.25.1-py2.py3-none-any.whl#sha256=c210084e36a42ae6b9219e00e48287def368a26d03a048ddad7bfee44f75871e (from requests-extra==1.0.0b3->-r requirements.txt (line 116))

The resolver does not recognise that requests[security]==2.24.0 fulfils requests<3, and tries to collect latest requests. This fails as it is has no pinned hash.

Additional information

Installation succeeds with --use-deprecated=legacy-resolver.

If the requirements file has no hashes, installation succeeds with the new resolver: Collecting requests<3 resolves to latest requests (2.25.1) in the install output, but the pinned version (2.24.0) is what ends up installed.

If all packages are already installed in the environment, pip install succeeds (with Requirement already satisfied) even with the new resolver.

Activity

  1. maljub01 commented on Mar 1, 2021

    @maljub01

    One workaround for this is to use --no-deps, which is a good idea by itself anyway (See the second warning here).

  2. cas-- commented on Apr 5, 2021

    @cas--

    I think this might be the same issue I have encountered with pinned and hash-checking botframework-connector requirements:

    Broken requirements file: requirements.txt

    Steps to reproduce:

    pip install -U pip pip-tools
    pip -V
    > pip 21.0.1
    echo "botframework-connector==4.12.0" > requirements.in
    pip-compile requirements.in --generate-hashes
    pip install -r requirements.txt
    

    Actual result:

    Collecting adal==1.2.1
      Using cached adal-1.2.1-py2.py3-none-any.whl (52 kB)
    Collecting botbuilder-schema==4.12.0
      Using cached botbuilder_schema-4.12.0-py2.py3-none-any.whl (34 kB)
    Collecting botframework-connector==4.12.0
      Using cached botframework_connector-4.12.0-py2.py3-none-any.whl (67 kB)
    Collecting certifi==2020.12.5
      Using cached certifi-2020.12.5-py2.py3-none-any.whl (147 kB)
    Collecting cffi==1.14.5
      Using cached cffi-1.14.5-cp38-cp38-manylinux1_x86_64.whl (411 kB)
    Collecting chardet==3.0.4
      Using cached chardet-3.0.4-py2.py3-none-any.whl (133 kB)
    Collecting cryptography==3.3.2
      Using cached cryptography-3.3.2-cp36-abi3-manylinux2010_x86_64.whl (2.6 MB)
    Collecting idna==2.10
      Using cached idna-2.10-py2.py3-none-any.whl (58 kB)
    Collecting isodate==0.6.0
      Using cached isodate-0.6.0-py2.py3-none-any.whl (45 kB)
    Collecting msal==1.6.0
      Using cached msal-1.6.0-py2.py3-none-any.whl (50 kB)
    Collecting msrest==0.6.10
      Using cached msrest-0.6.10-py2.py3-none-any.whl (82 kB)
    Collecting oauthlib==3.1.0
      Using cached oauthlib-3.1.0-py2.py3-none-any.whl (147 kB)
    Collecting pycparser==2.20
      Using cached pycparser-2.20-py2.py3-none-any.whl (112 kB)
    Collecting pyjwt[crypto]==1.5.3
      Using cached PyJWT-1.5.3-py2.py3-none-any.whl (17 kB)
    Collecting python-dateutil==2.8.1
      Using cached python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
    Collecting requests-oauthlib==1.3.0
      Using cached requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
    Collecting requests==2.23.0
      Using cached requests-2.23.0-py2.py3-none-any.whl (58 kB)
    Collecting six==1.15.0
      Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
    Collecting urllib3==1.25.11
      Using cached urllib3-1.25.11-py2.py3-none-any.whl (127 kB)
    Collecting PyJWT>=1.0.0
    ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
        PyJWT>=1.0.0 from https://files.pythonhosted.org/packages/b4/9b/8850f99027ed029af6828199cc87179eaccbbf1f9e6e373e7f0177d32dad/PyJWT-2.0.1-py3-none-any.whl#sha256=b70b15f89dc69b993d8a8d32c299032d5355c82f9b5b7e851d1a6d706dffe847 (from adal==1.2.1->-r requirements.txt (line 7))
    
    

    Workaround

    Either install pip <= 20.3.1, use tonybajan --use-deprecated=legacy-resolver option or use maljub01 --no-deps option:

    pip install -r requirements.txt --use-deprecated=legacy-resolver
    > Successfully installed adal-1.2.1 botbuilder-schema-4.12.0 botframework-connector-4.12.0 certifi-2020.12.5 cffi-1.14.5 chardet-3.0.4 cryptography-3.3.2 idna-2.10 isodate-0.6.0 msal-1.6.0 msrest-0.6.10 oauthlib-3.1.0 pycparser-2.20 pyjwt-1.5.3 python-dateutil-2.8.1 requests-2.23.0 requests-oauthlib-1.3.0 six-1.15.0 urllib3-1.25.11
    
    pip install -r requirements.txt --no-deps
    > Successfully installed adal-1.2.1 botbuilder-schema-4.12.0 botframework-connector-4.12.0 certifi-2020.12.5 cffi-1.14.5 chardet-3.0.4 cryptography-3.3.2 idna-2.10 isodate-0.6.0 msal-1.6.0 msrest-0.6.10 oauthlib-3.1.0 pycparser-2.20 pyjwt-1.5.3 python-dateutil-2.8.1 requests-2.23.0 requests-oauthlib-1.3.0 six-1.15.0 urllib3-1.25.11
    

    Edit: I thought I had posted this as off-topic but after looking at the requirements.txt generated by pip-tools I see that it is specifying extras: pyjwt[crypto]

  3. added this to the 21.2 milestone on Apr 17, 2021
  4. di commented on Apr 24, 2021

    @di
    SponsorMember

    If I understand correctly, #9775 did not fix this, but made it possible to fix this?

  5. uranusjr commented on Apr 24, 2021

    @uranusjr
    Member

    More accurately, the basis of #9775 (#9771) makes it possible to fix this.

  6. di commented on Apr 25, 2021

    @di
    SponsorMember

    Thanks @uranusjr. This is currently blocking CI and deployment for pypa/warehouse, is there any workaround? If not, could you explain at a high level what I'd need to do to fix this issue in pip?

  7. uranusjr commented on Apr 25, 2021

    @uranusjr
    Member

    I believe it’s possible to work around this by manually “unpacking” extras, e.g. add a line to also provide hashes to requests in addition to requests[security] (using the example provided by OP).

  8. di commented on May 18, 2021

    @di
    SponsorMember

    I tried to create a failing test for this:

    def test_new_resolver_hash_with_extras(script):
        parent_path = create_basic_wheel_for_package(
            script, "parent", "0.1.0", depends=["child"]
        )
        parent_hash = hashlib.sha256(parent_path.read_bytes()).hexdigest()
    
        child_path = create_basic_wheel_for_package(
            script, "child", "0.1.0", extras={"extra": ["extra"]}
        )
        child_hash = hashlib.sha256(child_path.read_bytes()).hexdigest()
    
        extra_path = create_basic_wheel_for_package(script, "extra", "0.1.0")
        extra_hash = hashlib.sha256(extra_path.read_bytes()).hexdigest()
    
        requirements_txt = script.scratch_path / "requirements.txt"
        requirements_txt.write_text(
            """
            child[extra]==0.1.0 --hash=sha256:{child_hash}
            parent==0.1.0 --hash=sha256:{parent_hash}
            extra==0.1.0 --hash=sha256:{extra_hash}
            """.format(
                child_hash=child_hash,
                parent_hash=parent_hash,
                extra_hash=extra_hash,
            ),
        )
    
        script.pip(
            "install",
            "--no-cache-dir",
            "--no-index",
            "--find-links", script.scratch_path,
            "--requirement", requirements_txt,
        )
    
        assert_installed(script, parent="0.1.0", child="0.1.0", extra="0.1.0")

    but it's passing on the current main branch at e6414d6.

    Is it possible that this has already been fixed, or is my test not properly exercising this bug?

  9. uranusjr commented on May 18, 2021

    @uranusjr
    Member

    I think the failure case should be like this:

    child==0.1.0 --hash=sha256:{child_hash}
    parent==0.1.0 --hash=sha256:{parent_hash}
    extra==0.1.0 --hash=sha256:{extra_hash}
    

    and change parent to depend on child[extra].

  10. di commented on May 18, 2021

    @di
    SponsorMember

    That still passes, but after looking more closely I was able to figure out what will cause it to fail:

    • two parent dependencies
      • one with a dependency on a child with an extra
      • one with a dependency on a child without an extra
    • a newer version of the child dependency available than is pinned

    It seems like what's happening is that pip is not considering the child dependency with the extra to be equivalent to the child dependency without the extra, and attempting to install the latest version of the child dependency, and not using the existing pinned/hashed requirement when doing this, which causes the hash checking to fail.

    I made a draft PR with the failing test here: #9995, but I don't have time at the moment to continue working on it to fix the issue.

  11. cas-- commented on May 18, 2021

    @cas--

    @di I have confirmed your theory by specifying another dependency without the extras package and everything installs

    pyjwt[crypto]==1.5.3 \
        --hash=sha256:500be75b17a63f70072416843dc80c8821109030be824f4d14758f114978bae7 \
        --hash=sha256:a4e5f1441e3ca7b382fd0c0b416777ced1f97c64ef0c33bfa39daf38505cfd2f
    +pyjwt==1.5.3 \
    +    --hash=sha256:500be75b17a63f70072416843dc80c8821109030be824f4d14758f114978bae7 \
    +    --hash=sha256:a4e5f1441e3ca7b382fd0c0b416777ced1f97c64ef0c33bfa39daf38505cfd2f
  12. di commented on May 18, 2021

    @di
    SponsorMember

    @cas-- It may be preferable to update all the dependencies that the extra includes to the latest versions instead. By removing the extra you are possibly removing sub-dependencies your application (or a dependency of your application) needs.

  13. added a commit that references this issue on May 18, 2021
  14. 90 remaining items

  15. anentropic commented on Oct 20, 2023

    @anentropic
  16. merwok commented on Oct 20, 2023

    @merwok
  17. devashish2203 commented on Aug 27, 2024

    @devashish2203

    We started running into this issue yesterday and as far as I know we haven't updated the requirements.txt in over a month.

    The requirements.txt was created with the following command

    > pip-compile --allow-unsafe --generate-hashes --output-file=requirements.txt requirements.in
    

    And then running the following fails with the below error.

    > pip install -r requirements.txt
    
    ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
        googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/b7/14/3f8b5670e7e082a3735dc9a6411b4ba37af7e6662441ee66782906265632/googleapis_common_protos-1.64.0-py2.py3-none-any.whl#sha256=d1bfc569f70ed2e96ccf06ead265c2cf42b5abfc817cda392e3835f3b67b5c59 (from google-api-core[grpc]==2.19.1->-r requirements.txt (line 672))
    

    Interestingly I'm not able to reproduce this on my M1 Mac laptop, but is consistently occuring in our CI which uses a Linux python3.10 docker image. Also can reproduce locally inside the docker image.

  18. jenstroeger commented on Jun 5, 2025

    @jenstroeger
  19. ichard26 commented on Jul 6, 2025

    @ichard26
  20. learnwithsomya commented on Aug 11, 2025

    @learnwithsomya
  21. notatallshaw commented on Aug 11, 2025

    @notatallshaw
    Member

    Any updates on the resolution of this?

    There are many users subscribed to this issue so making a comment makes a lot of noise.

    Please no further requests for updates where there is clearly no status change, you can read if there are updates either here or if there is a newly linked PR in the future.

    Pip continues to only be supported by volunteers, so it is up to someone to spend their spare time on writing up a well thought out design that can be agreed upon or a high quality PR that can be reviewed.

    Useful posts here would be:

    • Novel use cases that have not been discussed before
    • A design proposal to fix
    • A helpful workaround that has not been discussed before

    Please thumbs up the original post to express that you would like this fixed / implemented. I am marking the last few posts as off topic to help a little. If there are repeated further posts asking for an update this thread may be locked to maintainers only (only as last resort).

  22. pradyunsg commented on Apr 5, 2026

    @pradyunsg
    Member

    I attempted to reproduce this on current main (8c5468d) with Python 3.13 and was unable to.

    Reproduction attempt using the original scenario:

    echo 'requests[security]==2.24.0
    requests-extra==1.0.0b3' > requirements.in
    
    pip-compile --no-strip-extras --generate-hashes requirements.in -o requirements.txt
    pip install --dry-run --no-cache-dir -r requirements.txt
    • pip-compile produces requests[security]==2.24.0 with hashes.
      • requests-extra depends on bare requests<3.
    • pip correctly resolves this to requests==2.24.0 from the requests[security]==2.24.0 pin — no hash error.

    We also have a test covering this pattern (added in #9995), which passes.

    If anyone is still hitting this on a recent pip version, please share a minimal reproducer so we can investigate further.

  23. pradyunsg commented on Apr 6, 2026

    @pradyunsg
    Member

    #12095 should have fixed this, 2.5 years ago, if I did this bisect correctly.

  24. pradyunsg commented on Apr 6, 2026

    @pradyunsg
    Member

    Closing this out for now -- let us know if you've managed to get this error with pip 24 or newer.

  25. removed this from the milestone on Apr 12, 2026
  26. locked as resolved and limited conversation to collaborators on May 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    C: dependency resolutionAbout choosing which dependencies to installtype: bugA confirmed bug or unintended behavior

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions