Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'spack compiler add' can directly register spack installed compilers #7868

Conversation

alalazo
Copy link
Member

@alalazo alalazo commented Apr 23, 2018

The command line interface for spack compiler add has been extended with an additional flag that adds the prefix of specs to the search paths when registering a compiler.

The logic is such that a basic check is performed to ensure that the specified spec is that of a compiler. By default the compiler must be already installed but, if the --install-missing flag is passed, it will be bootstrapped.

Examples

$ spack compiler add --spec autoconf --spec gcc --spec cmake
==> Error: these packages are not compilers [autoconf, cmake]

$ spack compiler list
==> Available compilers
-- gcc rhel7-x86_64 ---------------------------------------------
gcc@4.8.5

$ spack compiler add --spec 'gcc %gcc@4.8.5' 
==> Error: the following specs need to be installed first [gcc %gcc@4.8.5]

$ spack compiler add --spec 'gcc %gcc@4.8.5' --install-missing
==> Installing missing spec for 'gcc %gcc@4.8.5' (might take a while)
==> Installing libsigsegv
==> Using cached archive: /home/culpo/github/spack-scitas/var/spack/cache/libsigsegv/libsigsegv-2.11.tar.gz
==> Staging archive: /home/culpo/github/spack-scitas/var/spack/stage/libsigsegv-2.11-3byxblkv2nvzumj3wk5cbzyud5hpjdlv/libsigsegv-2.11.tar.gz
==> Created stage in /home/culpo/github/spack-scitas/var/spack/stage/libsigsegv-2.11-3byxblkv2nvzumj3wk5cbzyud5hpjdlv
==> No patches needed for libsigsegv
==> Building libsigsegv [AutotoolsPackage]
==> Executing phase: 'autoreconf'
==> Executing phase: 'configure'
...
==> Successfully installed gcc
  Fetch: 10.95s.  Build: 13m 19.90s.  Total: 13m 30.85s.
[+] /home/culpo/github/spack-scitas/opt/spack/linux-rhel7-x86_64/gcc-4.8.5/gcc-7.3.0-qrjpi76aeo4bysagruwwfii6oneh56lj
==> Added 1 new compiler to /home/culpo/.spack/linux/compilers.yaml
    gcc@7.3.0
==> Compilers are defined in the following files:
    /home/culpo/.spack/linux/compilers.yaml

@alalazo
Copy link
Member Author

alalazo commented Apr 23, 2018

I marked the PR WIP as I would like to hear what people think about:

  • whether this PR might be of general interest
  • how the CLI should look like (should it be extended or modified?)

To what concerns us at EPFL, I'll use this small PR to simplify a bit the CI scripts for deployment (and move logic from a shell script that was driving Spack into Spack itself).

Copy link
Contributor

@healther healther left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor suggestions for the error messages

supported = spack.compilers.supported_compilers() + ['llvm']
not_compilers = [x for x in specs if x.abstract.name not in supported]
if not_compilers:
msg = 'these packages are not compilers [{0}]'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe make that "spack-supported compilers"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we decide to continue with this PR, it's probably worth to mark in some way the packages that provide a compiler (a compiler tag?) and rely on that for the check above. Currently, for instance, flang and intel-parallel-studio are not accounted as specs that provide a compiler.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The question is more what are we talking about long-term wise. Will compilers become "normal" build dependencies? Then I'd put here a "spack-supported" tag as this is really all it will amount to. If compilers will remain a thing, then we need to have a larger discussions on what constitutes a compiler that is recognised by spack (i.e. there are other languages besides C/C++ and Fortran that are compiled, do they get their own compiler support?)

I really don't have the knowledge to offer a well founded opinion, but it feels like wack a mole to keep up with all possible compilers. And (at least from my well of ignorance) there isn't much difference between py-setuptools and gcc, as in both modify the installed package and therefore (sometimes) need to be consistent within a runtime environment.

tty.msg(msg.format(item.request))
install(item.request, echo=True)
elif not_installed:
msg = 'the following specs need to be installed first [{0}]'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add something like "or provide --install-missing"

for item in not_installed:
msg = "Installing missing spec for '{0}' (might take a while)"
tty.msg(msg.format(item.request))
install(item.request, echo=True)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not too familiar with this part of spack, but what does echo do?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It tells a SpackCommand to echo the output to stdout (by default the output is not echoed, but only captured in a StringIO and returned to the caller as a string).

The command line interface for `spack compiler add` has been extended
with an additional flag (--spec SPEC) that adds the prefix of SPEC to
the search paths when registering a compiler.

The logic is such that a basic check is performed to ensure that the
specified spec is that of a compiler. By default the compiler must be
already installed but, if the `--install-missing` flag is passed, it
will be bootstrapped.
@alalazo alalazo force-pushed the features/register_spack_installed_compilers branch from 85e90c3 to bb9fdcd Compare May 21, 2018 07:13
@alalazo
Copy link
Member Author

alalazo commented Mar 8, 2019

Superseded by #10761

@alalazo alalazo closed this Mar 8, 2019
@alalazo alalazo deleted the features/register_spack_installed_compilers branch March 8, 2019 04:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants