Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate Python user lockfiles & improve tool lockfile error message #14353

Merged
merged 7 commits into from Feb 7, 2022

Conversation

Eric-Arellano
Copy link
Contributor

@Eric-Arellano Eric-Arellano commented Feb 3, 2022

Checks that interpreter constraints + requirements of the current run are compatible with the lockfile.

InvalidLockfileError: You are using the lockfile at 3rdparty/python/lockfiles/user_reqs.txt to install the resolve python-default (from [python].resolves). However, it is not compatible with the current targets:

  • The targets use requirements that are not in the lockfile: ['flake8-plugin==2.3']
    This most often happens when adding a new requirement to your project, or bumping requirement versions. You can fix this by regenerating the lockfile with generate-lockfiles.

  • The targets use interpreter constraints (CPython==3.9.*) that are not compatible with those used to generate the lockfile (CPython<3.10,>=3.7). You can fix this by either adjusting your targets to use different interpreter constraints (https://www.pantsbuild.org/v2.10/docs/python-interpreter-compatibility) or by generating the lockfile with different interpreter constraints by setting the option [python].resolves_to_interpreter_constraints, then running generate-lockfiles.

To regenerate your lockfile, run ./pants generate-lockfiles --resolve=python-default.

This also improves the error message for tool lockfiles to say how the resolves diverge:

  • You have set different requirements than those used to generate the lockfile. You can fix this by updating [black].version and/or [black].extra_requirements, or by using a new custom lockfile.
    In the input requirements, but not in the lockfile: ['typing-extensions>=3.50.0.0; python_version < "3.10"']
    In the lockfile, but not in the input requirements: ['typing-extensions>=3.10.0.0; python_version < "3.10"']

# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
Validation is now enabled for user lockfiles! Only the error messages are bad.

# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
This will help with error messages

# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
Copy link
Contributor Author

@Eric-Arellano Eric-Arellano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each commit is distinct, but this is probably easiest enough to review in a single pass.

Thanks @chrisjrn for laying the foundation for this!

@Eric-Arellano
Copy link
Contributor Author

Question, do we want to list what the relevant input targets were? I know how to implement it, only a question if we want it. I figure more diagnostics are better?

# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
@frozen_after_init
@dataclass(unsafe_hash=True)
class _RepositoryPexRequest:
addresses: Addresses
requirements: PexRequirements
Copy link
Sponsor Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good for docstring or field naming to clarify that both addresses and requirements are subsets (afaict) of the entire resolve, only used for staleness checks.

My first read of this (without thinking too much about the review title) was that we were passing in all of the requirements that made up of the resolve (which felt upside down, and I was about to comment to that effect).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed the names in general are confusing. I think Lockfile(req_strings=) is also confusing. I'm going to punt on this for a holistic rename: #14388

It's a little tricky that interpreter constraints do change how the _RepositoryPex is used, and addresses is used to determine those.

@stuhood
Copy link
Sponsor Member

stuhood commented Feb 4, 2022

Question, do we want to list what the relevant input targets were? I know how to implement it, only a question if we want it. I figure more diagnostics are better?

I don't think that that's necessary. The next step after detecting a stale lockfile (I think?) will almost always be regenerating the lockfile, rather than editing a target.

@Eric-Arellano Eric-Arellano merged commit 6972573 into pantsbuild:main Feb 7, 2022
@Eric-Arellano Eric-Arellano deleted the user-reqs-validation branch February 7, 2022 18:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants