Display `pyproject.toml`'s metatada parsing errors in verbose mode #1979

szobov · 2023-09-03T08:25:06Z

Before the changes implemented in this PR, the execution of pip-compile was silent about an error in pyproject.toml file.
If the user made a mistake in the configuration file, the output of the command was:

Backend subprocess exited when trying to invoke prepare_metadata_for_build_wheel

Now, the output is the following (the error is in the "licence" field for demonstration):

<truncated traceback>
ValueError: invalid pyproject.toml config: `project.license`.
configuration error: `project.license` must be valid exactly by one definition (2 matches found):

    - keys:
        'file': {type: string}
      required: ['file']
    - keys:
        'text': {type: string}
      required: ['text']

Backend subprocess exited when trying to invoke get_requires_for_build_wheel

There are several issues already mentioned on GitHub:
#1711
#1794

The changes implemented in this PR are possible only because of the new build version release:
pypa/build#615 (comment)
The particular change is this: pypa/build#566.

Please pay attention that in case of an error, the traceback is huge, but unfortunately, the error is captured and printed inside build.ProjectBuilder, and it's currently impossible to capture it and show only a short error description. On the other hand, the traceback gives the user an extra debug information.

Contributor checklist

Included tests for the changes.
PR title is short, clear, and ready to be included in the user-facing changelog.

Maintainer checklist

Verified one of these labels is present: backwards incompatible, feature, enhancement, deprecation, bug, dependency, docs or skip-changelog as they determine changelog listing.
Assign the PR to an existing or new milestone for the target version (following Semantic Versioning).

atugushev

Thanks for the PR! I've left a few comments below.

piptools/scripts/compile.py

tests/test_cli_compile.py

atugushev

Thanks for considering my suggestions! Here are the final adjustments I believe should improve code readability.

piptools/scripts/compile.py

tests/test_cli_compile.py

atugushev

LGTM 👍

szobov · 2023-09-05T13:30:11Z

I'm wondering how I can restart ci checks since they're failed due to a timeout error (most likely a network hiccup)

atugushev · 2023-09-05T13:39:56Z

I've restarted failed jobs.

szobov · 2023-09-05T13:45:40Z

Thanks, @atugushev, for your help and patience!
I think PR can now be merged since the only failing job is docs build, which you'll probably fix soon by merging #1983
But now it's your turn to take an action! :)

Since version "1.0.0" is released on PyPI we can pass a type of subprocess runner to "build.ProjectBuilder". Before the type was hardcoded insude "ProjectBuilder" and was not available to user. Therefore, whenever the error on parsing pyproject.toml file happened, user seen only short and non-informative `Backend subprocess exited when trying to invoke get_requires_for_build_wheel`. Now we can select `pyproject_hooks.default_subprocess_runner` from `pyproject_hooks` library. This runner will throw the error directly from toml parsing library.

In order to show user the error from toml's parsing library instead of `Backend subprocess exited when trying to invoke get_requires_for_build_wheel`.

for more information, see https://pre-commit.ci

* No more dependencies on flit * No network connection required * Test verbose and non-verbose modes

Co-authored-by: Albert Tugushev <albert@tugushev.ru>

To check only one action: either run in verbose or non-verbose mode

szobov · 2023-09-05T15:00:57Z

rebased on the fresh main branch

Bump pip_tools to >= 7.4.0 so that we can make use of better `pyproject.toml` parsing error messages during compiling. Specifically: jazzband/pip-tools#1979

atugushev reviewed Sep 3, 2023

View reviewed changes

piptools/scripts/compile.py Outdated Show resolved Hide resolved

tests/test_cli_compile.py Outdated Show resolved Hide resolved

tests/test_cli_compile.py Outdated Show resolved Hide resolved

szobov requested a review from atugushev September 4, 2023 08:28

atugushev reviewed Sep 4, 2023

View reviewed changes

piptools/scripts/compile.py Outdated Show resolved Hide resolved

tests/test_cli_compile.py Outdated Show resolved Hide resolved

tests/test_cli_compile.py Outdated Show resolved Hide resolved

szobov requested a review from atugushev September 5, 2023 07:11

atugushev reviewed Sep 5, 2023

View reviewed changes

tests/test_cli_compile.py Outdated Show resolved Hide resolved

szobov force-pushed the gh-1711-use-default-subprocess-runner branch 2 times, most recently from 59faba9 to 0209101 Compare September 5, 2023 12:02

szobov requested a review from atugushev September 5, 2023 12:23

atugushev approved these changes Sep 5, 2023

View reviewed changes

theryanwalker added the bug Something is not working label Sep 5, 2023

theryanwalker approved these changes Sep 5, 2023

View reviewed changes

szobov and others added 11 commits September 5, 2023 16:57

Use "default_subprocess_runner" when execute build.ProjectBuilder

9ac6d5f

In order to show user the error from toml's parsing library instead of `Backend subprocess exited when trying to invoke get_requires_for_build_wheel`.

Update "build" and "pyproject_hooks" versions in pre-commit hook

818bc8e

Add test to check pyproject.toml error is reported

f163771

[pre-commit.ci] auto fixes from pre-commit.com hooks

bb355d8

for more information, see https://pre-commit.ci

Use default_subprocess_runner only in verbose mode

b2bd640

Refactor test

2d3fbf7

* No more dependencies on flit * No network connection required * Test verbose and non-verbose modes

Update piptools/scripts/compile.py

1398d9c

Co-authored-by: Albert Tugushev <albert@tugushev.ru>

Update tests/test_cli_compile.py

91a339f

Co-authored-by: Albert Tugushev <albert@tugushev.ru>

Update tests/test_cli_compile.py

e9105f3

Co-authored-by: Albert Tugushev <albert@tugushev.ru>

Update tests/test_cli_compile.py:test_error_in_pyproject_toml

4557e7f

To check only one action: either run in verbose or non-verbose mode

szobov force-pushed the gh-1711-use-default-subprocess-runner branch from 0209101 to 4557e7f Compare September 5, 2023 14:58

atugushev changed the title ~~Show pyproject.toml parsing error~~ Display pyproject.toml's metatada parsing errors in verbose mode Sep 5, 2023

Merge branch 'main' into jazzbandgh-1711-use-default-subprocess-runner

a37352e

atugushev enabled auto-merge (squash) September 5, 2023 15:23

atugushev added the pyproject.toml Related to pyproject.toml support label Sep 5, 2023

atugushev added this to the 7.3.1 milestone Sep 5, 2023

atugushev mentioned this pull request Sep 5, 2023

Add Debugging Guidance to README #1797

Closed

4 tasks

atugushev merged commit e272f65 into jazzband:main Sep 5, 2023
36 checks passed

atugushev modified the milestones: 7.3.1, 7.4.0 Feb 16, 2024

dougthor42 mentioned this pull request Apr 11, 2024

build(deps): Bump pip tools to >= 7.4.0 bazelbuild/rules_python#1841

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Display `pyproject.toml`'s metatada parsing errors in verbose mode #1979

Display `pyproject.toml`'s metatada parsing errors in verbose mode #1979

szobov commented Sep 3, 2023 •

edited by atugushev

atugushev left a comment

atugushev left a comment

atugushev left a comment

szobov commented Sep 5, 2023

atugushev commented Sep 5, 2023

szobov commented Sep 5, 2023

szobov commented Sep 5, 2023

Display pyproject.toml's metatada parsing errors in verbose mode #1979

Display pyproject.toml's metatada parsing errors in verbose mode #1979

Conversation

szobov commented Sep 3, 2023 • edited by atugushev

Contributor checklist

Maintainer checklist

atugushev left a comment

Choose a reason for hiding this comment

atugushev left a comment

Choose a reason for hiding this comment

atugushev left a comment

Choose a reason for hiding this comment

szobov commented Sep 5, 2023

atugushev commented Sep 5, 2023

szobov commented Sep 5, 2023

szobov commented Sep 5, 2023

Display `pyproject.toml`'s metatada parsing errors in verbose mode #1979

Display `pyproject.toml`'s metatada parsing errors in verbose mode #1979

szobov commented Sep 3, 2023 •

edited by atugushev