pip_requirements fails to update hashes on dependencies with extras

[chludwig-haufe]

How are you running Renovate?

Self-hosted

If you're self-hosting Renovate, tell us what version of Renovate you run.

32.135.0

Please select which platform you are using if self-hosting.

GitLab self-hosted

If you're self-hosting Renovate, tell us what version of the platform you run.

GitLab CE 15.1.3

Was this something which used to work for you, and then stopped?

I never saw this working

Describe the bug

Given a requirements.txt file with (a.o.) the following entries, pip_requirements successfully updates the hashes for all dependencies except for boto3-stubs. It does create a MR with the updated version, but fails to update the hashes as well.

boto3==1.24.36 \
    --hash=sha256:8844bbcb69ac0afc68225b58abe105852231cf1b562e6c8c9eb6b2b97fd4757a \
    --hash=sha256:b1855ede59e725b968d6336908ffc864b65985ca441d730625b09c43ccd6413b
    # via secretmanagement (./pyproject.toml)
boto3-stubs[iam]==1.24.36.post1 \
    --hash=sha256:39acbbc8c87a101bdf46e058fbb012d044b773b43f7ed02cc4c24192a564411e \
    --hash=sha256:ca3b3066773fc727fea0dbec252d098098e45fe0def011b22036ef674344def2
    # via secretmanagement (./pyproject.toml)
botocore==1.27.36 \
    --hash=sha256:3119ce186053b9bf6bd0bd0ad19a8cedeb626b205ce6ad26ea0894634f702cd5 \
    --hash=sha256:8109526f55742539d2311d742b40c89e65781ad18966e577dda360cd55c9d047
    # via
    #   boto3
    #   s3transfer
botocore-stubs==1.27.36 \
    --hash=sha256:277b62b74784998dadc23fcb28452bb956fe6e4c6b2499e58430c9260da2fc7e \
    --hash=sha256:f257f39308a56c9e9acf9805f0f282a556c0eed27c7d583c24e4a9884110297b
    # via boto3-stubs

I dug into the source code a bit. Just from reading, I think the error is in line 31 of lib/modules/manager/pip_requirements/artifacts.ts: The test for the presence of hashes expects that the dependency name is directly followed by ==. However, line 112 of lib/modules/manager/pip_requirements/extract.ts stores only the package name (here: boto3-stubs) without any extras (here: [iam]) in the dependency name. Therefore, the test evaluates to false even though there are hashes.

Minimal repro is available at chludwig-haufe/renovate-pip-requirements-extras-repro. Therein, renovate created two PRs: chludwig-haufe/renovate-pip-requirements-extras-repro#1 updates the hashes as well, chludwig-haufe/renovate-pip-requirements-extras-repro#2 only updates the version number, not the hashes.

Relevant debug logs

Logs

DEBUG: packageFiles with updates (repository=library/gitlab-templates/secretsmanagement)
       "config": {
        // ..
        "pip_requirements": [
           {
             "packageFile": "build-image/requirements/base.txt",
             "deps": [
               // ...
              {
                 "depName": "boto3",
                 "currentValue": "==1.24.36",
                 "datasource": "pypi",
                 "currentVersion": "1.24.36",
                 "depIndex": 1,
                 "updates": [
                   {
                     "bucket": "non-major",
                     "newVersion": "1.24.41",
                     "newValue": "==1.24.41",
                     "releaseTimestamp": "2022-07-29T19:17:37.000Z",
                     "newMajor": 1,
                     "newMinor": 24,
                     "updateType": "patch",
                     "isRange": true,
                     "branchName": "renovate/boto3-botocore-and-stubs"
                   }
                 ],
                 "warnings": [],
                 "versioning": "pep440",
                 "sourceUrl": "https://github.com/boto/boto3",
                 "isSingleVersion": true,
                 "fixedVersion": "1.24.36"
               },
              {
                 "depName": "boto3-stubs",
                 "currentValue": "==1.24.36.post1",
                 "datasource": "pypi",
                 "currentVersion": "1.24.36.post1",
                 "depIndex": 2,
                 "updates": [
                   {
                     "bucket": "non-major",
                     "newVersion": "1.24.41",
                     "newValue": "==1.24.41",
                     "releaseTimestamp": "2022-07-29T19:24:47.000Z",
                     "newMajor": 1,
                     "newMinor": 24,
                     "updateType": "patch",
                     "isRange": true,
                     "branchName": "renovate/boto3-botocore-and-stubs"
                   }
                 ],
                 "warnings": [],
                 "versioning": "pep440",
                 "sourceUrl": "https://github.com/youtype/mypy_boto3_builder",
                 "isSingleVersion": true,
                 "fixedVersion": "1.24.36.post1"
               },
               // ...
             ]
           }
         ]
       }

Have you created a minimal reproduction repository?

I have linked to a minimal reproduction repository in the bug description

[chludwig-haufe]

I forked the repro into another account. There you can see that #16910 fixes the issue as the generated PR updates the hashes as well.

[renovate-release]

🎉 This issue has been resolved in version 32.165.4 🎉

The release is available on:

• GitHub release
• 32.165.4

Your semantic-release bot 📦🚀