Grouping multiple proxy pypi repositories no longer works as of version 3.65

[yakirm-cr]

• What problem are you trying to solve?
  We are currently using Nexus OSS version 3.63 and we are trying to upgrade to a newer version (currently, 3.67).
  We have a group pypi repository called pypi-all grouping 3 PyPi repositories:

  1. Hosted pypi repository.
  2. Proxy pypi repository for https://pypi.python.org.
  3. Proxy pypi repository for https://piwheels.org.

  Downloading and installing Python libraries using pip via the group pypi repository work as expected in version 3.63.
  After upgrading to version 3.65 (and above) we are getting the following errors when attempting to download and install Python libraries (using pip) from the group pypi repository:

Looking in indexes: https://nexus.my-company.com/repository/pypi-all/simple
Collecting attrs~=23.2.0 (from -r requirements.txt (line 1))
  Using cached https://nexus.my-company.com/repository/pypi-all/packages/attrs/23.2.0/attrs-23.2.0-py3-none-any.whl (60 kB)
 ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
     attrs~=23.2.0 from https://nexus.my-company.com/repository/pypi-all/packages/attrs/23.2.0/attrs-23.2.0-py3-none-any.whl#sha256=706ae337ee9e0276de51a670b0343ff60e106388771922ff5db5e640cb20c838 (from -r requirements.txt (line 1)):
         Expected sha256 706ae337ee9e0276de51a670b0343ff60e106388771922ff5db5e640cb20c838
              Got        99b87a485a5820b23b879f04c2305b44b951b502fd64be915879d77a7e8fc6f1

• Do you have a workaround you are using at present?
  Remain in version 3.63.

• What feature or behavior is this required for?
  Grouping multiple pypi repositories together as a single point of access.

• How could we solve this issue? (Not knowing is okay!)
  Unfortunately, I do not know. 🙁

• Tell us about your Nexus Repository deployment: what version, operating system, and database are you using?
  We are running version 3.63 via Docker image sonatype/nexus3:3.63.0 (running on Ubuntu 22.04).
  In addition we are using the default internal OrientDB.

• Anything else?
  Nope.

[nblair]

Hi @yakirm-cr thanks for opening an issue. I'm unable to replicate the error you've described with a similar setup; I was able to configure the same group and pull the attrs 23.2.0 through the group (it looks like the source was pulled from the piwheels proxy). Are there other aspects to your environment that could be contributing to the error?

[nblair]

That warning is pretty specific and severe:

ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE....Otherwise, examine the package contents carefully; someone may have tampered with them.

Is it possible there is something interfering that could have compromised the integrity of the packages you are retrieving?

[yakirm-cr]

@nblair
Thanks for the prompt response.

Here are the steps to reproduce the issue on Ubuntu 22.04 with docker:

1. Start Sonatype Nexus 3 via docker by running via terminal:
   docker run --rm -d --name nexus3 -p 8081:8081 sonatype/nexus3:3.66.0
2. Once Nexus is up (you can verify by running docker logs -f nexus3), login to it (you can obtain the admin password by running docker exec -ti nexus3 cat /nexus-data/admin.password) and follow the initial wizard.
3. Create the two proxy pypi repositories mentioned in the issue's description.
4. Create the group pypi repository (called pypi-all) mentioned in the issue's description (grouping the proxy repositories created in the previous step in the same order they are listed above - 1^st pypi-proxy and 2^nd piwheels-proxy).
5. Create a file called requirements.txt with the following context:

   attrs~=23.2.0
   bidict~=0.23.0
   cached-property~=1.5.2
   charset-normalizer~=3.3.0
   click~=8.1.3
   dnspython~=2.6.1
   fasteners~=0.18
   flasgger~=0.9.7.1
   flake8~=7.0.0
   Flask~=3.0.0
   Flask-Cors~=4.0.0
   Flask-SocketIO~=5.3.6
   gunicorn~=21.2.0
   importlib-metadata~=7.1.0
   importlib-resources~=6.4.0
   itsdangerous~=2.1.2
   Jinja2~=3.1.2
   jsonschema~=4.21.0
   MarkupSafe~=2.1.2
   mistune~=3.0.1
   netifaces~=0.11.0
   pip~=24.0
   pkgutil_resolve_name~=1.3.10
   pyrsistent~=0.20.0
   python-dateutil~=2.9.0
   python-engineio~=4.9.0
   python-socketio~=5.11.0
   PyYAML~=6.0
   referencing~=0.34.0
   requests~=2.31.0
   rpds-py~=0.18.0
   setuptools~=69.2.0
   six~=1.16.0
   urllib3~=2.2.0
   Werkzeug~=3.0.0
   zipp~=3.18.1

6. Run the following commands via terminal:

   python -m venv .env --prompt=test
   . .env/bin/activate
   pip install --no-cache --index-url=http://localhost:8081/repository/pypi-all/simple --trusted-host=localhost -r ./requirements.txt

Expected result:
All libraries listed in the requirments.txt are installed successfully in the Python virtual environment.

Actual results:
Installation of one of the libraries (random) fails with a similar error to the one provided in the issue's description.

[yakirm-cr]

@nblair
Were you able to reproduce the issue reported here with the steps I provided?

[yakirm-cr]

@nblair
Unfortunately, the issue is reproducible with version 3.68.0 as well. 🙁
It can be reroduced with the same steps I provided above by replacing the command in the 1^st step as follows:

docker run --rm -d --name nexus3 -p 8081:8081 sonatype/nexus3:3.68.0

[yakirm-cr]

Handled via #398