Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

consider bootstrapping spack as a standalone PEX to avoid interpreter clashes #20430

Open
3 tasks
cosmicexplorer opened this issue Dec 16, 2020 · 1 comment
Open
3 tasks
Labels
bootstrap Anything that has to do with Spack building its own dependencies. feature python

Comments

@cosmicexplorer
Copy link
Contributor

Summary: Expose the pex python tool as a spack package. This allows creating hermetic python environments as executable zip files, otherwise known as PEX files.

Rationale

  • Even after bootstrapping: bootstrap spack dependencies (executable and python module) #20207, the version of python used to bootstrap clingo must remain the first python executable on the PATH -- otherwise another python executable is used to execute spack, which will then fail to import clingo.
    • If the user wants to use a different python version, they then lose the ability to concretize with clingo.
  • If spack's source files and bootstrapped python dependencies from bootstrapping: bootstrap spack dependencies (executable and python module) #20207 are made into a PEX file, the correct interpreter will be selected each time, just for that PEX process, and we can remove the shell loop at the top of bin/spack!
  • It would also allow bootstrapping spack on a remote machine without git installed, by cping the PEX file

Description

  1. Add a PexPackage class which exposes the pex package.
    • It should probably download it from github releases, as that avoids any dependency on a specific python version at install time.
  2. Create a PEX file for spack itself after bootstrapping with bootstrapping: bootstrap spack dependencies (executable and python module) #20207.
    • This should not modify the bootstrap process from that PR whatsoever.
  3. Fix the spack PEX so spack executes from its checkout directory, not in ~/.pex.
    • The spack PEX will use the unpack directory within ~/.pex as its checkout directory by default, since that is where pex puts the actual files.
    • Can "hardcode" a __main__ script which makes spack execute in the correct checkout directory.
      • This should be doable without editing bin/spack at all, but that could also work.
@cosmicexplorer cosmicexplorer added feature python bootstrap Anything that has to do with Spack building its own dependencies. labels Dec 16, 2020
@cosmicexplorer
Copy link
Contributor Author

This may not end up being needed. But I would prefer we avoid interpreter entanglement rather than letting some users experience issues with isolation after bootstrapping clingo, and learn to accept that that's necessary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bootstrap Anything that has to do with Spack building its own dependencies. feature python
Projects
None yet
Development

No branches or pull requests

1 participant