Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pip installed wheels are not reproducible currently #11424

Closed
1 task done
kanavin opened this issue Aug 30, 2022 · 2 comments
Closed
1 task done

Pip installed wheels are not reproducible currently #11424

kanavin opened this issue Aug 30, 2022 · 2 comments
Labels
type: support User Support

Comments

@kanavin
Copy link

kanavin commented Aug 30, 2022

Description

Pip installed wheels are not reproducible currently. The direct_url
files encode an installation path and the installed wheels compile
the python files at their location, not their final install location
which is incorrect.

To fix this, Yocto project simply disables the direct_urls and passes the "root" to
the python compile function to strip that path out of the compiled
files.

A version of the attached patch, perhaps stripping root from the direct_urls
may be something that could be considered by upstream.

reproducible.patch.txt

Expected behavior

No response

pip version

22.2.2

Python version

3.10.6

OS

Yocto Linux

How to Reproduce

This can be seen by checking that installed wheels contain the original installation path in them.

Output

No response

Code of Conduct

@kanavin kanavin added S: needs triage Issues/PRs that need to be triaged type: bug A confirmed bug or unintended behavior labels Aug 30, 2022
@uranusjr
Copy link
Member

I assume you are using pip install <path to wheel> directly? This command is unreproducible by design. You should use --find-links or --index-url instead for reproducibility, which would not create the direct_url.json file.

@uranusjr uranusjr added type: support User Support S: awaiting response Waiting for a response/more information and removed type: bug A confirmed bug or unintended behavior S: needs triage Issues/PRs that need to be triaged labels Aug 30, 2022
@kanavin
Copy link
Author

kanavin commented Aug 30, 2022

I assume you are using pip install <path to wheel> directly? This command is unreproducible by design. You should use --find-links or --index-url instead for reproducibility, which would not create the direct_url.json file.

Thanks for the quick response. I have checked what happens; some time ago we (the yocto project) switched over from 'pip install' to 'python3 -m installer', and so the issue of reproducibility no longer comes up. This ticket can be closed.

rpurdie pushed a commit to yoctoproject/poky that referenced this issue Sep 2, 2022
Since "python_pep517: use installer instead of pip" pip is no longer
used in builds, and therefore doesn't need to behave reproducibly.

I have submitted the patch upstream; upstream have suggested that the
way it had been used previously is non-reproducible by design, and
if pip is again brought into builds in the future, we should use
specific options for reproducibility:

pypa/pip#11424

(From OE-Core rev: 654f9d5)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
halstead pushed a commit to openembedded/openembedded-core that referenced this issue Sep 2, 2022
Since "python_pep517: use installer instead of pip" pip is no longer
used in builds, and therefore doesn't need to behave reproducibly.

I have submitted the patch upstream; upstream have suggested that the
way it had been used previously is non-reproducible by design, and
if pip is again brought into builds in the future, we should use
specific options for reproducibility:

pypa/pip#11424

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 30, 2022
@pradyunsg pradyunsg removed the S: awaiting response Waiting for a response/more information label Mar 17, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
type: support User Support
Projects
None yet
Development

No branches or pull requests

3 participants