Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade Pex to 2.1.65. #14256

Merged
merged 1 commit into from Jan 26, 2022
Merged

Upgrade Pex to 2.1.65. #14256

merged 1 commit into from Jan 26, 2022

Conversation

jsirois
Copy link
Member

@jsirois jsirois commented Jan 26, 2022

This picks up a support for mac universal2 wheels. This support is
pressing since cryptography publishes these things and crytography is a
library folks tend to have in their transitive dependency set.

The intervening changelogs are here:

Fixes #14244

[ci skip-rust]
[ci skip-build-wheels]

@@ -11,13 +11,15 @@
# "generated_with_requirements": [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe revert this file's changes? Ack that this is the script doing it, but the script seems to be behaving differently than we want. In the meantime to automating the fix, manually workaround.

I plan to investigate tomorrow (didn't get a chance today)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm loathe to mess with auto-generated files. I'll wait for the fix. Thanks for working on it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case it's not clear from the other PR, I think this is what you'll want to run when making PEX upgrades: build-support/bin/generate_all_lockfiles.sh --internal --tool lambdex. Lambdex is because it uses PEX.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, that's pretty titchy but good to know. Ideally a user could ./pants <update lock goal> -p pex and have what lock files need updating figured out. Otherwise it seems its much better to all or nothing rather than pick and choose and choose wrong.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will say, I'll definitely forget this and be relying upon reviewers to catch the forget; as such I think I'll lean towards --all, then see the world changed, then remember --internal, then forget --tool then ... ? I'm not sure this system is all that workable in practice to be honest.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given how frequently we update Pex...would you like a --pex flag that is shorthand for --internal --tool lambdex? I agree this is titchy. I'm +1 for the option.

Ideally a user could ./pants -p pex and have what lock files need updating figured out

For a user, that is what they do. When you update a tool lockfile, you can either do ./pants generate-lockfiles (all) or ./pants generate-lockfiles --resolve=black. When you update a user requirement, you can either do ./pants generate-lockfiles or ./pants generate-lockfiles --resolve=my_user_resolve.

It's a weird situation for pantsbuild/pants because we don't just have our internally used lockfiles; we also need to generate default tool lockfiles. Those default tool lockfiles might not be == the tool lockfile we use internally. Hence the script.

Copy link
Sponsor Member

@stuhood stuhood Jan 26, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, to be clear: the medium term goal (...maybe short? depends how much of a pain this is) is to operate more like other lockfile producing tools and update lockfiles as a secondary side-effect of running the tool: #12014.

That change invalidates a lot of UX questions, so it's worth keeping it in mind if we think the usability of this (even as a temporary strategy) isn't up to snuff.

Rebase, roll back lock changes and re-gen with:
`./build-support/bin/generate_all_lockfiles.sh --internal --tool lambdex`

# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
decorator==5.1.0; python_version >= "3.7" \
--hash=sha256:7b12e7c3c6ab203a29e157335e9122cb03de9ab7264b137594103fd4a683b374 \
--hash=sha256:e59913af105b9860aa2c8d3272d9de5a56a4e608db9a2f167a8480b323d529a7
coverage==6.3; python_version >= "3.7" \
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no clue why ./build-support/bin/generate_all_lockfiles.sh --internal --tool lambdex updates this file too, but I want to autogenerate locks and not hand edit / pick choose; so I'd like to stick with it unless there are further fixes needed to build-support/bin/generate_all_lockfiles.sh here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because this is the lockfile for our internal usage of Pytest. It is not the lockfile for the default tool lockfile. This is fine to change.

@jsirois jsirois added this to the 2.9.x milestone Jan 26, 2022
@jsirois jsirois merged commit 3f5f4d2 into pantsbuild:main Jan 26, 2022
@jsirois jsirois deleted the issues/14244 branch January 26, 2022 23:33
Eric-Arellano added a commit that referenced this pull request Jan 26, 2022
…14274)

See discussion at #14256 (comment). We frequently need to regenerate PEX lockfiles, but it was too twitchy for the right command to run (`generate_all_lockfiles.sh --internal --tool lambdex`). This makes it more obvious and avoids us updating unrelated lockfiles.

[ci skip-rust]
@wisechengyi wisechengyi mentioned this pull request Jan 29, 2022
jsirois added a commit to jsirois/pants that referenced this pull request Feb 25, 2022
This picks up a support for mac universal2 wheels. This support is
pressing since cryptography publishes these things and crytography is a
library folks tend to have in their transitive dependency set.

The intervening changelogs are here:
+ https://github.com/pantsbuild/pex/releases/tag/v2.1.65
+ https://github.com/pantsbuild/pex/releases/tag/v2.1.64
+ https://github.com/pantsbuild/pex/releases/tag/v2.1.63
+ https://github.com/pantsbuild/pex/releases/tag/v2.1.62

Fixes pantsbuild#14244

(cherry picked from commit 3f5f4d2)

[ci skip-rust]

[ci skip-build-wheels]
@jsirois
Copy link
Member Author

jsirois commented Feb 25, 2022

jsirois added a commit that referenced this pull request Feb 25, 2022
This picks up a support for mac universal2 wheels. This support is
pressing since cryptography publishes these things and crytography is a
library folks tend to have in their transitive dependency set.

The intervening changelogs are here:
+ https://github.com/pantsbuild/pex/releases/tag/v2.1.65
+ https://github.com/pantsbuild/pex/releases/tag/v2.1.64
+ https://github.com/pantsbuild/pex/releases/tag/v2.1.63
+ https://github.com/pantsbuild/pex/releases/tag/v2.1.62

Fixes #14244

(cherry picked from commit 3f5f4d2)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Pants not finding wheel for package cryptography
3 participants