Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce PAR support / add an option to ship a PEX with Python included. #2096

Closed
jsirois opened this issue Mar 14, 2023 · 3 comments · Fixed by #2466
Closed

Introduce PAR support / add an option to ship a PEX with Python included. #2096

jsirois opened this issue Mar 14, 2023 · 3 comments · Fixed by #2466
Assignees

Comments

@jsirois
Copy link
Member

jsirois commented Mar 14, 2023

With the advent of scies, Pex's can now be shipped as standalone native executables with the Python interpreter included without much effort. If Pex were to incorporate this feature, it might be spelled like:

--par {eager,lazy}

This would cause Pex to generate one or more binaries using scie-jump that either directly include the appropriate PBS Python archive or else lazily fetch it using ptex. The fiddly bit here will be selecting an appropriately matched PBS per Pex target Python / abbreviated --platform / --complete-platform.

@jsirois
Copy link
Member Author

jsirois commented Mar 14, 2023

This has been a dream since ~day 1 of Pex, it was just too much work to consider. More recent surfacing of this desire has been in #636 and #1007.

@jsirois
Copy link
Member Author

jsirois commented Apr 22, 2023

Noting that the last PBS release to support 3.7 is https://github.com/indygreg/python-build-standalone/releases/tag/20200822 and that support 3.7.9. Modern PBS releases support 3.8 through 3.11. So Pex could support --par for 3.8 through 3.11 trivially, but possibly also for 3.7 as well.

@jsirois jsirois self-assigned this Jul 14, 2024
@jsirois
Copy link
Member Author

jsirois commented Jul 15, 2024

Alright, since --par is a Googlism and the real backend is scie; I'm just rolling with --scie {lazy,eager} and a few --scie-* options for exact control of the PBS Python selected when needed. The initial support is just as a Pex CLI build option. If someone wants a pex3 tool to turn an existing PEX file into a scie, that is easy enough to add later. This change contains all the infrastructure to do that.

jsirois added a commit to jsirois/pex that referenced this issue Jul 15, 2024
You can now specify `--scie {eager,lazy}` when building a PEX file and
one or more additional native executable PEX scies will be produced
along side the PEX file. These PEX scies will contain a portable CPython
interpreter from [Python Standalone Builds][PBS] in the `--scie eager`
case and will instead fetch a portable CPython interpreter just in time
on first boot on a given machine if needed in the `--scie lazy` case.

Although Pex will pick the target platforms and target portable CPython
interpreter version automatically, if more control is desired over which
platforms are targeted and which Python version is used, then
`--scie-platform`, `--scie-pbs-release`, and `--scie-python-version` can
be specified.

Closes pex-tool#636
Closes pex-tool#1007
Closes pex-tool#2096

[PBS]: https://github.com/indygreg/python-build-standalone
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant