Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Chaining shared Spack #15939

Open
wants to merge 28 commits into
base: develop
Choose a base branch
from

Conversation

scheibelp
Copy link
Member

@scheibelp scheibelp commented Apr 8, 2020

(This is very preliminary and I don't recommend even trying to use this at this point, but it covers some of the most essential changes that need to be made)

Changes which allow for a single system-installed Spack, with an installation root maintained by a system admin and an installation root maintained by a user (but for example admins can set config values that apply to any user of the Spack instance). Overall this is intended to allow admins to deploy an instance of Spack which looks like any other system-installed tool.

This includes:

  • Move module roots to install tree root (not strictly required but in general for users they need to be located outside the Spack prefix)
  • Add config scope associated with the install tree itself (this allows admins to assign specific permissions to the packages installed that are meant to be available to all users)
  • (5/20/20) Environments are stored in the install root. This allows users with a shared Spack instance to make their own environments. This does not allow admins of the shared Spack instance to provide environments (that could be handled in a later PR)
  • (5/20/20) For now it is assumed that users would not need to manage their own GPG keys: they would use the admin-created/added keys to sign new binary caches or to install them.

TODOs:

  • Testing
  • Move download cache to ~ (users can't store cached archives in the Spack prefix)

Non-essential TODOs:

  • It came up that indices (e.g. the provider cache) should be instanced on a per-install-tree basis, that is important for correctness in general but is not essential to this PR
  • If users of shared Spack could access the download cache used by the admins, this would reduce downloads and space used across all users of the shared Spack instance. For now, admins can create mirrors and add them to the config that users see.

See also: #11871

…store the install location in the install root)
… config for recording available install trees and shared install trees; add logic for initializing spack install location based on provided options and config
… that container logic still expects it to exists, so that needs an update
…or module indices in upstream install trees (in particular if module paths are not configured for upstreams)
@tgamblin tgamblin self-requested a review April 8, 2020 21:38
@scheibelp
Copy link
Member Author

Add config scope associated with the install tree itself (this allows admins to assign specific permissions to the packages installed that are meant to be available to all users)

In more detail:

  • There is a config scope associated with the install tree
  • If the user runs spack --install-tree=X and X appears in config:shared_install_trees, the user config scope (i.e. from ~/.spack) is ignored and the installation-tree config scope is loaded
  • If the user runs spack --install-tree=Y and Y is not a shared install tree (i.e. not in config:shared_install_trees), then the user config scope is loaded along with the installation tree config scope (in that order, with values from the install tree overriding values from ~)
    • Because of this ordering, the user can configure install trees in ~

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
spack-chains Related to chaining Spack
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants