validate extras against dependencies and in schema #542

Panaetius · 2022-12-22T09:26:34Z

Resolves: python-poetry/poetry/issues/7226

Added tests for changed code.
Updated documentation for changed code.

With this change, extras are validated to only contain valid characters and extras that reference dependencies that can't be found in the main dependency group raise a warning in poetry check (not in poetry lock/poetry install, though)

for more information, see https://pre-commit.ci

src/poetry/core/factory.py

Panaetius · 2023-01-03T08:24:52Z

I've addressed the PR comments.

@radoering do you have any thoughts on whether this being a warning in poetry check as per this PR is fine or should it be added to check_result["errors"] instead?

radoering · 2023-01-07T12:17:01Z

Good point. Comparing it to the other warnings and errors, I think it should be an error.

It's just an edge case but I just noticed extras should be checked if there are no dependencies. Can you unindent the new block by one level and replace config["dependencies"] by config.get("dependencies", {})?

sonarcloud · 2023-01-09T04:52:40Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

lovetheguitar · 2023-01-30T11:03:43Z

@radoering FYI, this was a breaking change for some.

E.g. isort 5.10.1 in our .pre-commit-config.yaml failed to install everywhere because of the 1.5 release.

https://github.com/PyCQA/isort/blob/9e4b5ee05264e6e566b18bea88b3073abd4b1014/pyproject.toml#L51

I guess version pinning of extras is not allowed/correct there (anymore)?

python-poetry/poetry-core#542 Committed via https://github.com/asottile/all-repos

works around some poetry breakage python-poetry/poetry-core#542

python-poetry/poetry-core#542 Committed via https://github.com/asottile/all-repos

radoering · 2023-01-30T15:42:28Z

I guess version pinning of extras is not allowed/correct there (anymore)?

It's still possible to define constraints for extras the same way as before: By defining an optional dependency as explained in the documentation and now done by isort in PyCQA/isort#2078. You can verify that the latest isort release has the same constraint on pip-shims ~~as before~~¹:

$ curl -s https://pypi.org/pypi/isort/5.12.0/json | jq .info.requires_dist | grep pip-shims
  "pip-shims (>=0.5.2) ; extra == \"pipfile-deprecated-finder\"",

To the best of my knowledge, the way isort specified the extra before is not documented and was never officially supported by poetry. We were not aware that this was possible ~~and even resulted in valid metadata~~¹.

¹ Before the invalid extra was just ignored as can be seen in distributions of isort 5.11.4. (Distributions of isort 5.11.5 were fixed some hours ago.)

python-poetry/poetry-core#542 Committed via https://github.com/asottile/all-repos

poetry-core 1.5.0 introduced a breaking change (in a minor version) by altering parsing of `extras` metadata (python-poetry/poetry-core#542). This broke package builds from source for isort (PyCQA/isort#2077), which is how pre-commit installs isort. This commit will update pre-commit to the fixed version of isort.

This PR contains the following updates: | Package | Update | Change | |---|---|---| | [poetry](https://python-poetry.org/) ([source](https://github.com/python-poetry/poetry), [changelog](https://python-poetry.org/history/)) | minor | `1.3.2` -> `1.4.0` | --- ### Release Notes <details> <summary>python-poetry/poetry</summary> ### [`v1.4.0`](https://github.com/python-poetry/poetry/blob/HEAD/CHANGELOG.md#140---2023-02-27) [Compare Source](python-poetry/poetry@1.3.2...1.4.0) ##### Added - **Add a modern installer (`installer.modern-installation`) for faster installation of packages and independence from pip** ([#6205](python-poetry/poetry#6205)). - Add support for `Private ::` trove classifiers ([#7271](python-poetry/poetry#7271)). - Add the version of poetry in the `@generated` comment at the beginning of the lock file ([#7339](python-poetry/poetry#7339)). - Add support for `virtualenvs.prefer-active-python` when running `poetry new` and `poetry init` ([#7100](python-poetry/poetry#7100)). ##### Changed - **Deprecate the old installer, i.e. setting `experimental.new-installer` to `false`** ([#7358](python-poetry/poetry#7358)). - Remove unused `platform` field from cached package info and bump the cache version ([#7304](python-poetry/poetry#7304)). - Extra dependencies of the root project are now sorted in the lock file ([#7375](python-poetry/poetry#7375)). - Remove upper boundary for `importlib-metadata` dependency ([#7434](python-poetry/poetry#7434)). - Validate path dependencies during use instead of during construction ([#6844](python-poetry/poetry#6844)). - Remove the deprecated `repository` modules ([#7468](python-poetry/poetry#7468)). ##### Fixed - Fix an issue where an unconditional dependency of an extra was not installed in specific environments ([#7175](python-poetry/poetry#7175)). - Fix an issue where a pre-release of a dependency was chosen even if a stable release fulfilled the constraint ([#7225](python-poetry/poetry#7225), [#7236](python-poetry/poetry#7236)). - Fix an issue where HTTP redirects were not handled correctly during publishing ([#7160](python-poetry/poetry#7160)). - Fix an issue where `poetry check` did not handle the `-C, --directory` option correctly ([#7241](python-poetry/poetry#7241)). - Fix an issue where the subdirectory information of a git dependency was not written to the lock file ([#7367](python-poetry/poetry#7367)). - Fix an issue where the wrong Python version was selected when creating an virtual environment ([#7221](python-poetry/poetry#7221)). - Fix an issue where packages that should be kept were uninstalled when calling `poetry install --sync` ([#7389](python-poetry/poetry#7389)). - Fix an issue where an incorrect value was set for `sys.argv[0]` when running installed scripts ([#6737](python-poetry/poetry#6737)). - Fix an issue where hashes in `direct_url.json` files were not written according to the specification ([#7475](python-poetry/poetry#7475)). - Fix an issue where poetry commands failed due to special characters in the path of the project or virtual environment ([#7471](python-poetry/poetry#7471)). - Fix an issue where poetry crashed with a `JSONDecodeError` when running a Python script that produced certain warnings ([#6665](python-poetry/poetry#6665)). ##### Docs - Add advice on how to maintain a poetry plugin ([#6977](python-poetry/poetry#6977)). - Update tox examples to comply with the latest tox release ([#7341](python-poetry/poetry#7341)). - Mention that the `poetry export` can export `constraints.txt` files ([#7383](python-poetry/poetry#7383)). - Add clarifications for moving configuration files ([#6864](python-poetry/poetry#6864)). - Mention the different types of exact version specifications ([#7503](python-poetry/poetry#7503)). ##### poetry-core ([`1.5.1`](https://github.com/python-poetry/poetry-core/releases/tag/1.5.1)) - Improve marker handling ([#528](python-poetry/poetry-core#528), [#534](python-poetry/poetry-core#534), [#530](python-poetry/poetry-core#530), [#546](python-poetry/poetry-core#546), [#547](python-poetry/poetry-core#547)). - Validate whether dependencies referenced in `extras` are defined in the main dependency group ([#542](python-poetry/poetry-core#542)). - Poetry no longer generates a `setup.py` file in sdists by default ([#318](python-poetry/poetry-core#318)). - Fix an issue where trailing newlines were allowed in `tool.poetry.description` ([#505](python-poetry/poetry-core#505)). - Fix an issue where the name of the data folder in wheels was not normalized ([#532](python-poetry/poetry-core#532)). - Fix an issue where the order of entries in the RECORD file was not deterministic ([#545](python-poetry/poetry-core#545)). - Fix an issue where zero padding was not correctly handled in version comparisons ([#540](python-poetry/poetry-core#540)). - Fix an issue where sdist builds did not support multiple READMEs ([#486](python-poetry/poetry-core#486)). ##### poetry-plugin-export ([`^1.3.0`](https://github.com/python-poetry/poetry-plugin-export/releases/tag/1.3.0)) - Fix an issue where the export failed if there was a circular dependency on the root package ([#118](python-poetry/poetry-plugin-export#118)). </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Reviewed-on: https://git.walbeck.it/walbeck-it/docker-python-poetry/pulls/655 Co-authored-by: renovate-bot <bot@walbeck.it> Co-committed-by: renovate-bot <bot@walbeck.it>

python-poetry/poetry-core#542 Committed via https://github.com/asottile/all-repos

validate extras against dependencies and in schema

6b3342a

Panaetius mentioned this pull request Dec 22, 2022

extras are not validated against dependencies python-poetry/poetry#7226

Closed

3 tasks

[pre-commit.ci] auto fixes from pre-commit.com hooks

98c4eb2

for more information, see https://pre-commit.ci

radoering reviewed Dec 31, 2022

View reviewed changes

src/poetry/core/factory.py Outdated Show resolved Hide resolved

Ralf Grubenmann added 2 commits January 3, 2023 09:15

address PR comments

b583f25

Merge remote-tracking branch 'poetry/main' into fix-validating-extras

b5118ac

always check extras and make it an error

017df47

Panaetius force-pushed the fix-validating-extras branch from 086c6c0 to 017df47 Compare January 8, 2023 11:02

Merge branch 'main' into fix-validating-extras

aa7321c

Panaetius requested a review from radoering January 8, 2023 11:17

Merge branch 'main' into fix-validating-extras

68bd594

radoering approved these changes Jan 9, 2023

View reviewed changes

radoering merged commit 978fb55 into python-poetry:main Jan 9, 2023

radoering mentioned this pull request Jan 24, 2023

Release 1.5.0 #551

Merged

asottile-sentry added a commit to getsentry/arroyo that referenced this pull request Jan 30, 2023

ref: upgrade isort to work around poetry breakage

e0416a1

python-poetry/poetry-core#542 Committed via https://github.com/asottile/all-repos

asottile-sentry added a commit to getsentry/responses that referenced this pull request Jan 30, 2023

ref: upgrade isort to work around poetry breakage

3f029ae

python-poetry/poetry-core#542 Committed via https://github.com/asottile/all-repos

asottile-sentry added a commit to getsentry/snuba that referenced this pull request Jan 30, 2023

ref: upgrade isort to work around poetry breakage

40c5ecd

python-poetry/poetry-core#542 Committed via https://github.com/asottile/all-repos

asottile-sentry added a commit to getsentry/snuba-sdk that referenced this pull request Jan 30, 2023

ref: upgrade isort to work around poetry breakage

08a20bd

python-poetry/poetry-core#542 Committed via https://github.com/asottile/all-repos

asottile-sentry added a commit to getsentry/pypi that referenced this pull request Jan 30, 2023

add latest isort

9ea3505

works around some poetry breakage python-poetry/poetry-core#542

markstory pushed a commit to getsentry/responses that referenced this pull request Jan 30, 2023

ref: upgrade isort to work around poetry breakage (#620)

faf6dd9

python-poetry/poetry-core#542 Committed via https://github.com/asottile/all-repos

asottile-sentry added a commit to getsentry/snuba that referenced this pull request Jan 30, 2023

ref: upgrade isort to work around poetry breakage (#3668)

2a5d17b

python-poetry/poetry-core#542 Committed via https://github.com/asottile/all-repos

asottile-sentry added a commit to getsentry/snuba-sdk that referenced this pull request Jan 30, 2023

ref: upgrade isort to work around poetry breakage (#111)

288109f

python-poetry/poetry-core#542 Committed via https://github.com/asottile/all-repos

asottile-sentry added a commit to getsentry/arroyo that referenced this pull request Jan 30, 2023

ref: upgrade isort to work around poetry breakage (#179)

ba53ef1

python-poetry/poetry-core#542 Committed via https://github.com/asottile/all-repos

radoering mentioned this pull request Feb 18, 2023

release: bump version to 1.4.0 python-poetry/poetry#7531

Merged

btbrunotome added a commit to btbrunotome/responses that referenced this pull request Apr 22, 2024

ref: upgrade isort to work around poetry breakage (#620)

cd35de8

python-poetry/poetry-core#542 Committed via https://github.com/asottile/all-repos

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

validate extras against dependencies and in schema #542

validate extras against dependencies and in schema #542

Panaetius commented Dec 22, 2022

Panaetius commented Jan 3, 2023

radoering commented Jan 7, 2023

sonarcloud bot commented Jan 9, 2023

lovetheguitar commented Jan 30, 2023

radoering commented Jan 30, 2023 •

edited

Loading

validate extras against dependencies and in schema #542

validate extras against dependencies and in schema #542

Conversation

Panaetius commented Dec 22, 2022

Panaetius commented Jan 3, 2023

radoering commented Jan 7, 2023

sonarcloud bot commented Jan 9, 2023

lovetheguitar commented Jan 30, 2023

radoering commented Jan 30, 2023 • edited Loading

radoering commented Jan 30, 2023 •

edited

Loading