Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Building internal PEXes for built-in tools with [pex].emit_warnings shows warnings that seem useless #20577

Open
huonw opened this issue Feb 19, 2024 · 7 comments
Labels

Comments

@huonw
Copy link
Contributor

huonw commented Feb 19, 2024

Describe the bug

In 2.20.x, we're introduced the [pex].emit_warnings option (https://www.pantsbuild.org/2.20/reference/subsystems/pex#emit_warnings) to be able to surface more warnings from PEX, but building internal PEXes for builtin tools can show the warnings. Users can't do anything about them. This option is thus off by default.

For instance:

cd $(mktemp -d)

cat > pants.toml <<EOF
[GLOBAL]
pants_version = "2.22.0.dev1"

backend_packages = []

[pex]
emit_warnings = true
EOF

touch BUILD

pants update-build-files ::

Running that shows:

11:59:12.71 [INFO] waiting for pantsd to start...
11:59:13.23 [INFO] pantsd started
11:59:13.54 [INFO] Initializing scheduler...
11:59:13.92 [INFO] Scheduler initialized.
11:59:16.88 [WARN] /Users/huon/.cache/pants/named_caches/pex_root/installed_wheels/8d43c1ed16877f3d2ed39e5ef3e92aa8885d4427cd7d9ce62683c922a5c0cd07/pex-2.3.1-py2.py3-none-any.whl/pex/bin/pex.py:947: PEXWarning: Could not calculate a targeted shebang for:
cp310-cp310-macosx_14_0_arm64 interpreter at /Users/huon/.pyenv/versions/3.10.1/bin/python3.10
cp310-cp310-macosx_14_0_arm64 interpreter at /Users/huon/.pyenv/versions/3.10.4/bin/python3.10
cp310-cp310-macosx_14_0_arm64 interpreter at /opt/homebrew/Cellar/python@3.10/3.10.12_1/Frameworks/Python.framework/Versions/3.10/bin/python3.10
cp311-cp311-macosx_14_0_arm64 interpreter at /Users/huon/.pyenv/versions/3.11.0/bin/python3.11
cp311-cp311-macosx_14_0_arm64 interpreter at /opt/homebrew/Cellar/python@3.11/3.11.9/Frameworks/Python.framework/Versions/3.11/bin/python3.11
cp312-cp312-macosx_14_0_arm64 interpreter at /Users/huon/.pyenv/versions/3.12.0/bin/python3.12
cp312-cp312-macosx_14_0_arm64 interpreter at /opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/bin/python3.12
cp37-cp37m-macosx_14_0_arm64 interpreter at /Users/huon/.pyenv/versions/3.7.13/bin/python3.7
cp38-cp38-macosx_14_0_arm64 interpreter at /Users/huon/.pyenv/versions/3.8.7/bin/python3.8
cp38-cp38-macosx_14_0_arm64 interpreter at /opt/homebrew/Cellar/python@3.8/3.8.19/Frameworks/Python.framework/Versions/3.8/bin/python3.8
cp39-cp39-macosx_14_0_arm64 interpreter at /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/bin/python3.9
cp39-cp39-macosx_14_0_arm64 interpreter at /Users/huon/.pyenv/versions/3.9.1/bin/python3.9
cp39-cp39-macosx_14_0_arm64 interpreter at /Users/huon/.pyenv/versions/3.9.10/bin/python3.9
cp39-cp39-macosx_14_0_arm64 interpreter at /Users/huon/.pyenv/versions/3.9.17/bin/python3.9
cp39-cp39-macosx_14_0_arm64 interpreter at /Users/huon/.pyenv/versions/3.9.7/bin/python3.9

Using shebang: #!/usr/bin/env python3.7
If this is not appropriate, you can specify a custom shebang using the --python-shebang option.
  pex_warnings.warn(

11:59:16.95 [INFO] Starting: Building black.pex from resource://pants.backend.python.lint.black/black.lock
11:59:18.13 [INFO] Completed: Building black.pex from resource://pants.backend.python.lint.black/black.lock
11:59:18.54 [INFO] No required changes to BUILD files found. However, there may still be deprecations that `update-build-files` doesn't know how to fix. See https://www.pantsbuild.org/2.22/docs/releases/upgrade-tips for upgrade tips.

There's a big chunk of PEX warnings, that I don't think I as a normal user can do anything to silence as they seem to be about some PEXes that pants is building internally. (I could change the interpreters I have installed, but that doesn't seem appropriate.)

Pants version

2.20, 2.21, 2.22

This was previously shown by default, which was a regression between that and 2.20.0.dev7 likely related to #20480.

We turned down the default level in #20590.

OS
macOS

Additional info
N/A

@huonw huonw added the bug label Feb 19, 2024
@huonw huonw added this to the 2.20.x milestone Feb 19, 2024
@huonw
Copy link
Contributor Author

huonw commented Feb 20, 2024

Heya @cburroughs, thanks for exposing the warnings. Do you have thoughts on this consequence of it?

@cburroughs
Copy link
Contributor

In general this is surfacing the type of issues like #19514 that users run into which would be good to have dogfooded answer for.

In this case, I'm confused by how we got there though. Shouldn't Pants be using the bundled version of PBS from scie-pants?

huonw added a commit that referenced this issue Feb 22, 2024
This is a short-term workaround for #20577 and #20586, where Pants'
internal/default use of Pex triggers user-visible warnings, and those
warnings are now visible due to #20480... so, instead of showing them by
default, let's hide them for now. Users with desire for insight into the
warnings can still enable this.

Doing this means we're not having to rush in fixes for the root causes
of these warnings for 2.20.0 stable release, and thus reduce
feature-creep/risk. We can/should reenable warnings by default in a
future release.

Per @cburroughs's idea in
#20586 (comment),
this explicitly switches it on for the Pants repo itself, so we're
eating our own dogfood and catching real and/or spurious errors earlier.
WorkerPants pushed a commit that referenced this issue Feb 22, 2024
This is a short-term workaround for #20577 and #20586, where Pants'
internal/default use of Pex triggers user-visible warnings, and those
warnings are now visible due to #20480... so, instead of showing them by
default, let's hide them for now. Users with desire for insight into the
warnings can still enable this.

Doing this means we're not having to rush in fixes for the root causes
of these warnings for 2.20.0 stable release, and thus reduce
feature-creep/risk. We can/should reenable warnings by default in a
future release.

Per @cburroughs's idea in
#20586 (comment),
this explicitly switches it on for the Pants repo itself, so we're
eating our own dogfood and catching real and/or spurious errors earlier.
huonw added a commit that referenced this issue Feb 22, 2024
…#20593)

This is a short-term workaround for #20577 and #20586, where Pants'
internal/default use of Pex triggers user-visible warnings, and those
warnings are now visible due to #20480... so, instead of showing them by
default, let's hide them for now. Users with desire for insight into the
warnings can still enable this.

Doing this means we're not having to rush in fixes for the root causes
of these warnings for 2.20.0 stable release, and thus reduce
feature-creep/risk. We can/should reenable warnings by default in a
future release.

Per @cburroughs's idea in
#20586 (comment),
this explicitly switches it on for the Pants repo itself, so we're
eating our own dogfood and catching real and/or spurious errors earlier.

Co-authored-by: Huon Wilson <huon@exoflare.io>
@huonw
Copy link
Contributor Author

huonw commented Feb 22, 2024

Removing this from the 2.20.x milestone, since we landed #20593 as a stop-gap to reduce severity in 2.20.0

@huonw huonw removed this from the 2.20.x milestone Feb 22, 2024
@huonw huonw added this to the 2.21.x milestone Apr 21, 2024
@huonw huonw changed the title 2.20 shows warnings when building internal PEXes(?) that seem useless 2.21 shows warnings when building internal PEXes(?) that seem useless Apr 21, 2024
@cburroughs
Copy link
Contributor

Is there something actionable/blocking for 2.21 here? (I'd like to change the default, but I think that's a new feature at this point.)

@huonw
Copy link
Contributor Author

huonw commented May 14, 2024

Ah, yes, thanks, I think this is now fully covered by #20586. The stopgap from #20590 is still around in 2.21 (and 2.22 etc.): https://github.com/pantsbuild/pants/blob/2.21.x/src/python/pants/backend/python/util_rules/pex_environment.py#L85

@huonw huonw closed this as completed May 14, 2024
@huonw
Copy link
Contributor Author

huonw commented May 14, 2024

Oh, wait, hm, no, #20586 covers tests related to Pants' dependencies. The warnings are there for built-in tools. I'll reframe this one.

@huonw huonw reopened this May 14, 2024
@huonw huonw changed the title 2.21 shows warnings when building internal PEXes(?) that seem useless Building internal PEXes(?) for built-in tools with [pex].emit_warnings shows warnings that seem useless May 14, 2024
@huonw huonw removed this from the 2.21.x milestone May 14, 2024
@huonw huonw changed the title Building internal PEXes(?) for built-in tools with [pex].emit_warnings shows warnings that seem useless Building internal PEXes for built-in tools with [pex].emit_warnings shows warnings that seem useless May 14, 2024
@huonw
Copy link
Contributor Author

huonw commented May 14, 2024

And indeed, not blocking for 2.21 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants