New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add 'bootstrap tools' (bootstrap-cache-key) via PANTS_BOOTSTRAP_TOOLS=1 #128
Conversation
This functionality that can introspect the bootstrapping process, similar to `PEX_TOOLS=1 ./some.pex ...` being able to introspect a PEX file. The motivating tool, added here, is `PANTS_BOOTSTRAP_TOOLS=1 ./pants bootstrap-cache-key` that prints a somewhat opaque string designed to be used as a key for CI caching systems. This is designed to help with pantsbuild/actions#5.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is super-cool! I tried it out and it works great.
In the future I can imagine wanting to also emit a more parseable version, but for now this is 👨🍳
Thanks for merging. Based on your comment about adding new subcommands in future, I was prompted to think about versioning for scripts, and thus opened #129. |
Ah, and I discovered a small bug: #130 |
) This fixes a (small) bug in #128: the virtualenv may have a symlink to the 'actual' path of the python executable, and the python_path value used may've itself been a symlink or a shim. This uses sys.executable as an estimate of this path, which will hopefully be good enough for the circumstances where bootstrap-cache-key is useful.
This expands on #128 to add another bootstrap tool subcommand: `bootstrap-version`. This is designed to just be a simple number that's bumped whenever there's a feature scripts might need to query for. The version number is also automatically checked against the (numeric) value of the `PANTS_BOOTSTRAP_TOOLS` key. For instance, suppose the bootstrap tools already existed (at version 1), and then `bootstrap-cache-key` command was added, the version number would be bumped to 2, and then a consumer (such as pantsbuild/actions#6) could run something like: ```shell PANTS_BOOTSTRAP_CACHE_KEY=$(PANTS_BOOTSTRAP_TOOLS=2 ./pants bootstrap-cache-key) ```
The setup cache includes symlinks to things outside the cache, such as the (system) Python executable. Thus, the cache should only be reused on machines where those parts of the system exactly match up. This switches from trying to compute an appropriate cache key externally to letting the pants bootstrap script do it itself, via the new `PANTS_BOOTSTRAP_TOOLS=1 ./pants bootstrap-cache-key` (pantsbuild/setup#128). This is a breaking change, in that it requires updating the checked in pants scripts to support the new tool. Fixes #5
Github updated the default minor version of python 3.9.x The cache key did not include the minor version which broke things for us. Luckily, this was already fixed in pantsbuild/actions#6. The fix requires an update to the `./pants` entrypoint/bootstrap script which was updated in pantsbuild/setup#128. updated `./pants` with: curl -L -O https://static.pantsbuild.org/setup/pants && chmod +x ./pants
update GHA pants-init action to fix build Github updated the default minor version of python 3.9.x The cache key did not include the minor version which broke things for us. Luckily, this was already fixed in pantsbuild/actions#6. The fix requires an update to the `./pants` entrypoint/bootstrap script which was updated in pantsbuild/setup#128. updated `./pants` with: curl -L -O https://static.pantsbuild.org/setup/pants && chmod +x ./pants
This adds functionality that can introspect the bootstrapping process, similar to
PEX_TOOLS=1 ./some.pex ...
being able to introspect a PEX file.The motivating tool, added here, is
PANTS_BOOTSTRAP_TOOLS=1 ./pants bootstrap-cache-key
that prints a somewhat opaque string designed to be used as a key for CI caching systems. This is designed to help with pantsbuild/actions#5, and is prompted by the discussion in pantsbuild/actions#6 (comment).For example, on my system:
(This PR is a quick sketch, please lemme know if it doesn't make sense!)