Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert "Upgrade Pex to 2.1.95. (#16110)" #16125

Merged
merged 1 commit into from Jul 11, 2022
Merged

Conversation

benjyw
Copy link
Sponsor Contributor

@benjyw benjyw commented Jul 11, 2022

This reverts #16110, as that changed caused wheel building to fail.

To observe this failure, run ./build-support/bin/release.sh build-wheels

Note that the revert was not clean due to #15951 also writing user_reqs.lock.
So I regenerated that file to ensure validity, and manually verified that the
failure above is fixed.

[ci skip-rust]

[ci skip-build-wheels]

This reverts commit e7988a3.

[ci skip-rust]

[ci skip-build-wheels]
Copy link
Member

@jsirois jsirois left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I definitely got lazy with all the red CI emails numbing me there. So this is PEX_EXTRA_SYS_PATH propagation I added in Pex 2.1.95. The ./build-support/bin/release.sh build-wheels uses ./pants run ... under the covers and that tacks on path items using PEX_EXTRA_SYS_PATH. In particular it tacks on just the things the build-support/bin/_release_helper.py script needs, which includes a subset of src/python/pants/util. Although pants is a namespace package, pants.util is not; so boom. So this concept of propagating PEX_EXTRA_SYS_PATH falls down for non-ns packages in the right circumstances. I'll be able to think about how / where this should be fixed more clearly in the am, but a revert is definitely the right thing to do here now.

@jsirois
Copy link
Member

jsirois commented Jul 11, 2022

Aha, the fundamental issue is the mechanism of PEX_EXTRA_SYS_PATH propagation (PYTHONPATH) is not faithful. PEX_EXTRA_SYS_PATH appends to the sys.path but PYTHONPATH prepends.

@benjyw benjyw merged commit 032985a into pantsbuild:main Jul 11, 2022
@benjyw benjyw deleted the revert_16110 branch July 11, 2022 15:27
benjyw added a commit to benjyw/pants that referenced this pull request Jul 11, 2022
Note that the revert didn't apply cleanly, so
I regenerated all the lockfiles.

[ci skip-rust]

[ci skip-build-wheels]
@jsirois
Copy link
Member

jsirois commented Jul 11, 2022

I think I have a way to fix this robustly (for venvs only, but that's what Pants uses). I'll get out a 2.1.96 release with the fix tracked here: pex-tool/pex#1836. The summary is Pants no longer needs the PEX_EXTRA_SYS_PATH hack and could be injecting a .pth in the internal venvs it creates to do the same work, but robustly. Pex, though, won't break PEX_EXTRA_SYS_PATH in Pex 2.x; so I may as well just fix it there for the --venv case.

benjyw added a commit to benjyw/pants that referenced this pull request Jul 11, 2022
Note that the revert didn't apply cleanly, so
I regenerated all the lockfiles.

[ci skip-rust]

[ci skip-build-wheels]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants