Mix Clang and GNU Fortran on OS-X

[davydden]

Spack correctly finds compilers on El Capitan

$ ./spack compilers
==> Available compilers
-- gcc ----------------------------------------------------------
gcc@5.3.0  gcc@4.2.1

-- clang --------------------------------------------------------
clang@7.0.2-apple

and the compilers.yaml looks all right

compilers:
  darwin-x86_64:
    clang@7.0.2-apple:
      cc: /usr/bin/clang
      cxx: /usr/bin/clang++
      f77: null
      fc: null
    gcc@4.2.1:
      cc: /usr/bin/gcc
      cxx: /usr/bin/g++
      f77: null
      fc: null
    gcc@5.3.0:
      cc: null
      cxx: null
      f77: /usr/local/bin/gfortran
      fc: /usr/local/bin/gfortran

To use this on OS-X, one really need a way to specify the usage of

cc: /usr/bin/clang
cxx: /usr/bin/clang++

together with

f77: /usr/local/bin/gfortran
fc: /usr/local/bin/gfortran

when building, say, OpenMPI with fortran support. That also includes changing the installation prefix darwin-x86_64/gcc-5.3.0/ to something like darwin-x86_64/gcc-5.3.0-clang-7.0.2/.

As a current workaround I guess one can edit compilers.yaml manually to something like

compilers:
  darwin-x86_64:
    clang@7.0.2-apple:
      cc: /usr/bin/clang
      cxx: /usr/bin/clang++
      f77: /usr/local/bin/gfortran
      fc: /usr/local/bin/gfortran

and then make clang default.

[adamjstewart]

@davydden: You're not alone! I'm also using mixed compilers on Linux. I have a user who wants his entire software stack built with GNU C/C++ and NAG Fortran compilers. I haven't gotten around to testing this yet, but your solution of manually editing compilers.yaml should work fine. Mine looks like:

compilers:
  linux-x86_64:
    nag@6.0:
      cc: /blues/gpfs/home/software/spack/opt/spack/linux-x86_64/gcc-4.4.7/gcc-5.3.0-fygfl7rvyuiteto27dlhmilp5cstw2o2/bin/gcc
      cxx: /blues/gpfs/home/software/spack/opt/spack/linux-x86_64/gcc-4.4.7/gcc-5.3.0-fygfl7rvyuiteto27dlhmilp5cstw2o2/bin/g++
      f77: /blues/gpfs/home/software/nag/6.0/bin/nagfor
      fc: /blues/gpfs/home/software/nag/6.0/bin/nagfor

However, some packages behave differently based on compiler. For example, look at the package for ncurses. It applies a GCC patch when the compiler is set to %gcc. If you name your compiler clang@7.0.2-apple, this patch won't get applied and the build may not succeed.

If it doesn't, then we might need to start a conversation about better mixed compiler support. Allowing compilers like gcc-5.3.0-clang-7.0.2 may not be a simple task.

[eschnett]

Spack shouldn't treat compilers specially. Compilers are just a build (or run) dependency. Compilers, Python, Perl, etc., are all quite similar. A package that requires a C (or Fortran) compiler should specify so as a dependency.

Combining C and C++ compilers into one configuration item might make sense. Fortran compilers should probably be handled separately.

[davydden]

Perhaps something like

compilers:
  darwin-x86_64:
    fortran:
      gcc@5.3.0:
         f77: /usr/local/bin/gfortran
         fc: /usr/local/bin/gfortran
    c:
      clang@7.0.2-apple:
        cc: /usr/bin/clang
        cxx: /usr/bin/clang++
      gcc@5.3.0:
         cc: /usr/local/bin/gcc
         cxx: /usr/local/bin/g++

[citibeth]

Is this a documentation issue or a coding issue? Maybe the manual needs a Mac OSX section explaining how to set up mixed compilers. This practice seems to be common, but is not documented anywhere online (that I know of).

[adamjstewart]

I would say this is both a Spack issue and a documentation issue.

Documenation issue:
There currently isn't much documentation on setting up compilers. You may have noticed on GitHub and the Google Group that a lot of people are having problems getting the Intel compilers to work with Spack. There are dozens of suggested workaround, and I've been meaning to add documentation for these. I would like an entire section on compilers, with specific instructions for each compiler. It could include instructions on installing PGI with Spack (see #558), workarounds for setting up Intel (see https://groups.google.com/forum/#!topic/spack/NxyNTAZyMQg), and instructions for setting up mixed compilers.

Spack issue:
The issue I see with Spack is that there is no proper handling of mixed compilers. Let's say you want to build with GCC and NAG. Your compilers.yaml will include:

compilers:
  linux-x86_64:
    nag@6.0:
      cc: /blues/gpfs/home/software/spack/opt/spack/linux-x86_64/gcc-4.4.7/gcc-5.3.0-fygfl7rvyuiteto27dlhmilp5cstw2o2/bin/gcc
      cxx: /blues/gpfs/home/software/spack/opt/spack/linux-x86_64/gcc-4.4.7/gcc-5.3.0-fygfl7rvyuiteto27dlhmilp5cstw2o2/bin/g++
      f77: /blues/gpfs/home/software/nag/6.0/bin/nagfor
      fc: /blues/gpfs/home/software/nag/6.0/bin/nagfor

Now, should this be named nag@6.0 or gcc@5.3.0? What if you want to be able to use nag@6.0/gcc@5.3.0 and nag@6.0/gcc@4.4.7 compiler combinations? Or nag@6.0/gcc@5.3.0 and nag@5.0/gcc@5.3.0? You can't currently name it nag@6.0-gcc@5.3.0 as far as I know.

@davydden: I like your suggestion of keeping C and Fortran compilers separate. I don't think you need separate c: and fortran: tags though, since cc: and cxx: imply C and f77: and fc: imply Fortran. The only problem I see with that is how to specify it on the command line. You would need a way for people to build %gcc and use it for both C/Fortran, and a way for people to build %gcc %clang to get gcc for C and clang for Fortran. And what if someone wants to mix GCC and PGI for some reason? Which is used for C and which is used for Fortran?

I don't think users should be made to modify compilers.yaml just to be able to use Clang or NAG. I do like being able to specify %nag when I want to build it with NAG and GCC though. Maybe, if a user specifies a compiler that only has C or Fortran compilers (not both), then unless otherwise specified, the other compilers would come from the default_order. When I run spack find, I would like it to specify that the packages were built with both though.

What are other people's thoughts on the ideal way to support mixed compilers?

[davydden]

@adamjstewart you are right, that one does not need to have separate sections, something like

compilers:
  darwin-x86_64:
    clang@7.0.2-apple:
      cc: /usr/bin/clang
      cxx: /usr/bin/clang++
      f77: null
      fc: null
    gcc@5.3.0:
      cc: /usr/local/bin/gcc
      cxx: /usr/local/bin/g++
      f77: /usr/local/bin/gfortran
      fc: /usr/local/bin/gfortran

would be enough. And I like your idea about checking null. So

packages:
  all:
    compiler: [clang, gcc]

will lead to a mixture of compilers whereas

packages:
  all:
    compiler: [gcc,clang]

will lead to the usage of gcc throughout. That's user-side related.

In addition to that, one would still need to be able to query

if self.fortran_compiler.name == 'gcc':

IMHO that will be enough to cover mixing with an exception when two different compilers provide both fortran and C. For that one would need to hack compilers.yaml to set null in desired places. But this is most likely a minority of usage cases.

Documentation
In addition to what you said, i think more on setting system-provided MPI compilers (say also Intel MPI) would be good.

[citibeth]

Spack issue:
The issue I see with Spack is that there is no proper handling of mixed
compilers. Let's say you want to build with GCC and NAG. Your
compilers.yaml will include:

compilers:
linux-x86_64:
nag@6.0:
cc: /blues/gpfs/home/software/spack/opt/spack/linux-x86_64/gcc-4.4.7/gcc-5.3.0-fygfl7rvyuiteto27dlhmilp5cstw2o2/bin/gcc
cxx: /blues/gpfs/home/software/spack/opt/spack/linux-x86_64/gcc-4.4.7/gcc-5.3.0-fygfl7rvyuiteto27dlhmilp5cstw2o2/bin/g++
f77: /blues/gpfs/home/software/nag/6.0/bin/nagfor
fc: /blues/gpfs/home/software/nag/6.0/bin/nagfor

Now, should this be named nag@6.0 or gcc@5.3.0? What if you want to be
able to use nag@6.0/gcc@5.3.0 and nag@6.0/gcc@4.4.7 compiler
combinations? Or nag@6.0/gcc@5.3.0 and nag@5.0/gcc@5.3.0? You can't
currently name it nag@6.0-gcc@5.3.0 as far as I know.

Two thoughts:

1. Is there any need to keep installed packages separated by compile, as
   Spack currently does? As long as the compiler(s) are part of the hash, it
   should be possible to throw everything into one directory. This would
   solve some (but maybe not all) of our need to name a toolchain.
2. We could go the route of EasyBuild and get creative with our own
   toolchain names.

What are other people's thoughts on the ideal way to support mixed

compilers?

Now that we have build dependencies, get rid of special compiler support
altogether. And separate out the toolchains based on C, C++, Fortran, etc.
portions. A compiler is just a build dependency, and can be satisfied
using the virtual package mechanism. This solves a bunch of problems:

1. Simpler code, because we have just ONE type of build dependency, rather
   than two (regular build dependencies, plus compilers, which are specified
   with a '%'). We've replaced an increasingly complex specialized, limited
   feature with a simpler, more generalized, more powerful feature.
2. Get rid of compilers.yaml, packages.yaml can now do that for us.
3. No problems of what happens when people want to use strange new
   compilers or code generators. "How do I add a Pascal compiler to
   compilers.yaml?"
4. Packages are now explicit about what compilers they need. I can
   depends_on('cxx_compiler'), or depends_on('fortran90_compiler'), etc.
5. Compilers are now no different than other kinds of code generations (eg
   bison, flex), which are also build dependencies.

-- Elizabeth

[adamjstewart]

1. Is there any need to keep installed packages separated by compile, as Spack currently does? As long as the compiler(s) are part of the hash, it should be possible to throw everything into one directory. This would solve some (but maybe not all) of our need to name a toolchain.

My users specifically request NetCDF, HDF5, and HDF4 built with OpenMPI, MPICH, and MVAPICH using GCC, Intel, PGI, and NAG. This is the reason we started using Spack instead of installing everything by hand. I actually really don't like the hash, and don't want to absorb anything else into it. If I want to uninstall all packages that were built with PGI and start over again, currently I can run rm -rf /soft/spack/opt/spack/linux-x86_64/pgi-16.3-0. I would like to keep this ability, at least until #360 is merged.

My biggest complaint about the hash is that it's very difficult to find the installation path for a package, and the path is very long. Since I need 4 compilers, 3 MPI implementation, and 2 variants, I end up having to install 24 versions of HDF5. The only way to figure out which one is which is to run:

spack find -p hdf5 %pgi ^openmpi+psm
==> 2 installed packages.
-- linux-x86_64 / pgi@16.3-0 ------------------------------------
    hdf5@1.8.16~cxx~debug+fortran+mpi+shared+szip~threadsafe  /blues/gpfs/home/software/spack/opt/spack/linux-x86_64/pgi-16.3-0/hdf5-1.8.16-h2dssxiylgwvhtcqhuji22o4bwuukumu
    hdf5@1.8.16+cxx~debug+fortran~mpi+shared+szip~threadsafe  /blues/gpfs/home/software/spack/opt/spack/linux-x86_64/pgi-16.3-0/hdf5-1.8.16-gezhgvksjcbzfcrp5bd5jsqljwmsmvcy

(For the record, one of those packages doesn't depend on OpenMPI, but I haven't file a bug report yet). This is fine, but I don't see why we couldn't also install things in a path like:

/soft/spack/opt/spack/linux-x86_64/pgi-16.3-0/hdf5-1.8.16/openmpi-1.10.2-<enabled variants>

@tgamblin: from what I understand, the reason that we started using the super-long-hash is because there were problems with special characters in filenames. I don't see why we couldn't just use dashes for all of these instead. Regardless of whether we include more stuff in the installation path, I don't see why we need the hashes at all. Any information previously stored in the hash could be stored in a spack spec file in the directory. This would also solve the overly long shebang problem.

We could go the route of EasyBuild and get creative with our own toolchain names.

One of the reasons we decided to go with Spack instead of EasyBuild was that the EasyBuild toolchains seemed overly complicated and we didn't want to have to write configuration files for every installation. Spack is just simpler in that respect. With that said, I never tried out EasyBuild because it had too many dependencies that I didn't want to have to install.

Now that we have build dependencies, get rid of special compiler support altogether. And separate out the toolchains based on C, C++, Fortran, etc. portions. A compiler is just a build dependency, and can be satisfied using the virtual package mechanism.

I really like this! Some packages only depend on F77, not FC, and actually raise warnings when we try to supply both. We should specify compilers as build dependencies and use Spack's dependency resolution magic. I'm not sure how I feel about dropping %compiler and compilers.yaml though. I guess this would make them obsolete.

If we make them build dependencies, how do you propose to tell Spack to compile with Clang for C/C++ and GCC for Fortran? Or something crazy specific like using 4 different compilers? How about ^clang.c ^gcc.f for the first case and ^gcc.cc ^pgi.cxx ^intel.f77 ^nag.fc for a ridiculous example that is very finely tuned?

[davydden]

I'm not sure how I feel about dropping %compiler and compilers.yaml though. I guess this would make them obsolete.

I think dropping % would make it difficult (if not impossible) to specify different compilers to be used for different dependencies. I would keep % and also compilers.yaml file.

[citibeth]

If we make them build dependencies, how do you propose to tell Spack to
compile with Clang for C/C++ and GCC for Fortran? Or something crazy
specific like using 4 different compilers? How about ^clang.c ^gcc.f for
the first case and ^gcc.cc ^pgi.cxx ^intel.f77 ^nag.fc for a ridiculous
example that is very finely tuned?

Same as we tell Spack to use OpenMPI vs. MPICH. Something like:

mylib/package.py:

    depend_on('compiler-cxx@11:')    # This needs C++11
    depends_on('compiler-fortran:2013')   # This needs Fortran 2013

gcc-cxx/package.py:

provides('compiler-cxx@:14', when='@5:')
provides('compiler-cxx@:11', when='@4.9:')
provides('compiler-cxx@:03', when='4.0:')

packages.yaml:

packages:
    gcc-cxx:
        version: [4.9.3 5.1.0 3.3.2]

all:
    providers:
        compiler-cxx: [gcc-cxx clang-cxx]
        compiler-fortran: [nag-fortran]

Now I can just say:

spack install myproj

Or I specify which compiler I want on the command line, just as I specify
which MPI I want (sorry, I forgot how to dothis after packages.yaml came
into being).

My biggest complaint about the hash is that it's very difficult to find the

installation path for a package, and the path is very long. Since I need 4
compilers, 3 MPI implementation, and 2 variants, I end up having to install
24 versions of HDF5. The only way to figure out which one is which is to
run:

Our options as I see it are:

1. Use hashes
2. Encode the full spec into the installation directory name. This can get
   VERY long. One project I work with has over 50 dependencies encoded into
   its spec. Without formatting, the spec is completely unreadable.
3. Skip relevant information needed to identify a build (eg, something
   ad-hoc, like EasyBuild toolchains).

Of the three options, I think that hashes are the least evil. By far. We
should focus on designing features that help us navigate the hashes.

-- Elizabeth

[adamjstewart]

I think dropping % would make it difficult (if not impossible) to specify different compilers to be used for different dependencies. I would keep % and also compilers.yaml file.

Shit, you're right. I need this to install openmpi %pgi +tm +psm ^libpciaccess%gcc. But I don't see why we need compilers.yaml.

[alalazo]

We could go the route of EasyBuild and get creative with our own toolchain names.

I find this a really BAD idea, which has possible ripercussion on the formal language used by spack (imho its best innovation if compared with other package managers). Besides :

spack install foo%clang@3.8.0

is comprehensible by anyone, while :

spack install foo%golf@11.3.107

will be a little bit more cryptic...

[citibeth]

On Fri, Mar 25, 2016 at 1:32 PM, Adam J. Stewart notifications@github.com
wrote:

I think dropping % would make it difficult (if not impossible) to specify
different compilers to be used for different dependencies. I would keep %
and also compilers.yaml file.

Shit, you're right. I need this to install openmpi %pgi +tm +psm
^libpciaccess%gcc. But I don't see why we need compilers.yaml.

I see... Spack as it is currently designed requires you to use compatible
versions of all your dependencies. What you would WANT to say is:

spack install openmpi +tm +psm ^compiler-pgi-c  ^(libpciaccess
^compiler-gcc-c)

Build dependencies are quite new. One we figure out how to specify a set
of incompatible build dependencies (which should be perfectly legal), then
this problem caused by replacing compilers with build dependencies should
go away.

[adamjstewart]

Our options as I see it are:

1. Use hashes
2. Encode the full spec into the installation directory name. This can get VERY long. One project I work with has over 50 dependencies encoded into its spec. Without formatting, the spec is completely unreadable.
3. Skip relevant information needed to identify a build (eg, something ad-hoc, like EasyBuild toolchains).

@citibeth: Why encode any of the spec in the installation directory name? Everything Spack needs to find how a package was built can be found in $ROOT/.spack/spec.yaml. All we need is a unique installation path. This can be something like:

/soft/spack/opt/spack/linux-x86_64/pgi-16.3-0/hdf5-1.8.16/openmpi-1.10.2-1
/soft/spack/opt/spack/linux-x86_64/pgi-16.3-0/hdf5-1.8.16/openmpi-1.10.2-2
...

or something as simple as:

/soft/spack/opt/spack/openmpi-1
/soft/spack/opt/spack/openmpi-2
...

It's not any less human readable, but it is much shorter. I'm just hoping for a way to tell my 24 hdf5 installations apart based on the path. And I know the super-long-hash is causing shebang problems for some people.

[tgamblin]

Now that we have build dependencies, get rid of special compiler support altogether. And separate out the toolchains based on C, C++, Fortran, etc. portions. A compiler is just a build dependency, and can be satisfied using the virtual package mechanism.

I really like this! Some packages only depend on F77, not FC, and actually raise warnings when we try to supply both. We should specify compilers as build dependencies and use Spack's dependency resolution magic. I'm not sure how I feel about dropping %compiler and compilers.yaml though. I guess this would make them obsolete.

I think dropping % would make it difficult (if not impossible) to specify different compilers to be used for different dependencies. I would keep % and also compilers.yaml file.

@davydden's point for % is actually true for all build dependencies. I have been thinking of making it a special desginator for build deps. Some background:

Link Dependencies
If you depend on two packages that both depend on petsc, Spack guarantees you that you'll have one petsc in the final build DAG. This is important for building things with sane, deterministic runtime behavior. If you don't do that (i.e. if you have RPATHs or deps that need different versions of some package) then you risk either having things explode at runtime when ABI issues arise, or you have a race to determine which version of petsc gets loaded first. You really don't want either.

Build dependencies
Build dependencies could be different on different nodes in your DAG, and they don't have to be consistent. You could use different versions of Python to run scripts as part of the build for different nodes in a DAG, and you'd be fine. We typically assume that there are no linking requirements or ABI issues with build dependencies.

Compilers
Compilers are more complicated:

1. People want to swap them in and out of builds to test performance characteristics and different flags. Spack allows this with somewhat specialized compiler wrapper support.
2. The compiler you use affects the ABI of generated code. You can't mix some compilers with others, and which ones are compatible with which gets complicated quickly. They can affect how one package calls another, and this can break things.
3. Related to 2, compilers imply something about how your application is linked. They come with implicit runtime libraries. These may or may not be compatible with one another. We started thinking about this in spack.abi, which is used in concretization. It's by no means complete.

So really, compilers aren't build dependencies. They're build/link dependencies with special compatibility semantics. I would like Spack's dependency system to understand that, and I think people should remember the above distinctions before we go "fully general".

That said, I think you could keep % as a designator for dependencies that don't get normalized across the DAG. i.e., deps that are just for one node and do not affect the others. That could include compilers, but it is more complicated than just making everything a build dependency or a link dependency.

[tgamblin]

Shit, you're right. I need this to install openmpi %pgi +tm +psm ^libpciaccess%gcc. But I don't see why we need compilers.yaml.

I think compilers.yaml should eventually go away and merge into packages.yaml, or some auto-generated file (from a general mechanism for detecting external compilers and build deps) that is referenced from a packages.yaml

[alalazo]

@adamjstewart Regarding the "shorter" directory name, remember that if you don't hash you need to track:

1. all virtual dependencies (not only MPI)
2. all variants

In the end I am not sure that removing the hashes in the installation dir will be either more readable or shorter.

[adamjstewart]

@alalazo: We are keeping track of those, in the .spack/spec.yaml for every package that is built.

[citibeth]

@citibeth https://github.com/citibeth: Why encode any of the spec in
the installation directory name? Everything Spack needs to find how a
package was built can be found in $ROOT/.spack/spec.yaml. All we need is a
unique installation path. This can be something like:

/soft/spack/opt/spack/linux-x86_64/pgi-16.3-0/hdf5-1.8.16/openmpi-1.10.2-1
/soft/spack/opt/spack/linux-x86_64/pgi-16.3-0/hdf5-1.8.16/openmpi-1.10.2-2
...

or something as simple as:

/soft/spack/opt/spack/openmpi-1
/soft/spack/opt/spack/openmpi-2
...

In these proposals, you HAVE encoded part of the spec in the installation
directory name. In fact, you've chosen two different ways to do it.
Neither of them is fully general, in the sense that it prevents collisions
between two different versions of a package. The problem is that there
really IS no general and correct way to fully specify the version of a
package, other than the full spec.

However, that does not help you with YOUR problem. You're not building
every version of OpenMPI out there; you're building "just" 24 versions.
And you have a good mental map of how you would like those 24 versions to
be named.

Maybe what we need is some kind of pluggable nicknames. When a packages is
installed, it would be installed by hash as is currently done. But Spack
would ALSO use one more nickname generators you supplied to come up with
the kinds of names above; and it would generate symlinks from your
nicknames to the hash directories. Nickname generators would produce
nicknames based on the full spec, by throwing out parts of the spec you
don't care to look at. They don't have to be fully general, just general
enough for a particular application of Spack.

It's not any less human readable, but it is much shorter. I'm just hoping
for a way to tell my 24 hdf5 installations apart based on the path.

Would pluggable nicknames solve your issues?

And I know the super-long-hash is causing shebang problems for some people.

What are the problems specifically? There's no reason hashes can't be
somewhat shorter...

-- Elizabeth

[davydden]

i actually do liked the current directory naming. It is a good balance between opt/package-<hash> and opt/platform/compiler/package/mpi-dep1-dep2-dep3-dep4-...-dep99. You can still navigate in folders and see what you have installed. I don't think this should be touched at all.

[adamjstewart]

@davydden: But what is better about:

opt/spack/linux-x86_64/pgi-16.3-0/openmpi-1.10.2-gibftzb3wyfkxc2lqxvnmtkiv2kr6hib

than:

opt/spack/linux-x86_64/pgi-16.3-0/openmpi-1.10.2-1

Both are unique (you just need to use an incrementor to add a number to the end of each build). Both supply all of the necessary information on how it was build (all of this can be gathered from the .spack/spec.yaml file.

[adamjstewart]

@tgamblin: do you have any opinions on hash vs. no hash? I would like to get this discussion out of the way so we can focus on the more important issue of mixed compiler support. This was just a side thought that appears to have blown up 😛

[citibeth]

On Fri, Mar 25, 2016 at 1:56 PM, Denis Davydov notifications@github.com
wrote:

i actually do liked the current directory naming. It is a good balance
between opt/package- and
opt/platform/compiler/package/mpi-dep1-dep2-dep3-dep4-...-dep99. You can
still navigate in folders and see what you have installed. I don't think
this should be touched at all.

Back to the original problem in the thread... what happens if you're using
a "mixed" compiler? How do you name the /compiler/ portion of the name
then? Workable proposals so far are:

1. Eliminate /compiler/ from the package path.
2. Come up with EasyBuild-style names for mixed toolchains ("gompi",
   "gooper", etc.)

-- Elizabeth

[tgamblin]

All we need is a unique installation path. This can be something like:

/soft/spack/opt/spack/openmpi-1
/soft/spack/opt/spack/openmpi-2

You really don't want a directory name that depends on the order things were built. I need a quick way to query what is already installed so that I do not rebuild it. With this scheme, I would need to do that lookup somewhere else.

Of the three options, I think that hashes are the least evil. By far. We
should focus on designing features that help us navigate the hashes.

I agree with this. I think we need a feature that we have implemented but not merged yet...

Views

These are described in the spack paper. @mplegendre implemented them, and we use them here. We can generate an arbitrarily-formatted hierarchy of symlinks that point back into the real Spack install tree. We call these views. (note that this is different from the "views" that @hegner recently described on the mailing list -- his views are more like the thing we've been wanting called "environments"... we'll have to sort the terminology out sometime).

With views you could write something like:

${^mpi.name}/${compiler.name}-${compiler.version}/${name}-${version}-${variants}

and Spack will generate a tree with that layout, full of symlinks back to the real Spack install. Now, when you generate a tree like that, many hashes may map to one path, because your path is essentially describing some smaller subspace of the full spack package space.

Since no spack feature would be complete without some mechanism to deal with combinatorial issues like this, we added some preference rules so that you can choose which of the many hashes to prefer for each unique name in your view. That is, if I have something like:

${name}-${version.major}.${version.minor}

Many patch versions may map to that. You can say which ones you want to prefer, and in what order. Spack will link in the first one that exists in opt and matches.

You can use this to generate a consistent view of your package tree, and regenerate it over time as you upgrade things.

Is this something you'd want @adamjstewart? I think we could resurrect the feature and iterate on it. We had postponed merging it because we didn't know how it would work with "environments", which are basically where you link a bunch of packages into the same root. We thought we could unify those two ideas somehow and allow hierarchies where some things might actually exist in the same root and others might be separate. Seems like what we've got now might solve your issues.

[davydden]

How do you name the /compiler/ portion of the name
then?

in case it's a single compiler family, it would be /compiler/package-<hash>,
whereas if you mix two (and two is a max), then /compiler-compiler/packag-<hash>. Just need to agree which one is which. Say first - C/CXX, second F77/FC.

[tgamblin]

@adamjstewart: why don't we start a separate topic on views and move the view stuff there.

[citibeth]

Suppose I have a complete spec. With the current system, I can generate a
directory name from that spec, look for that directory, and come up with an
immediate answer of whether or not I've built that spec.

With an incrementor-based naming scheme, I would have to look in every
directory with the same base name (but different incrementors) to make that
determination.

-- Elizabeth

On Fri, Mar 25, 2016 at 2:01 PM, Adam J. Stewart notifications@github.com
wrote:

@davydden https://github.com/davydden: But what is better about:

opt/spack/linux-x86_64/pgi-16.3-0/openmpi-1.10.2-gibftzb3wyfkxc2lqxvnmtkiv2kr6hib

than:

opt/spack/linux-x86_64/pgi-16.3-0/openmpi-1.10.2-1

Both are unique (you just need to use an incrementor to add a number to
the end of each build). Both supply all of the necessary information on how
it was build (all of this can be gathered from the .spack/spec.yaml file.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#568 (comment)

[adamjstewart]

@tgamblin: Views sound pretty sweet. I would be happy with that compromise someday. In the meantime, judging by the number of open issues, I think we have bigger problems to deal with. How about we save that for another day. Don't want to add too many features before fixing the ones we already have.

[tgamblin]

@adamjstewart Ok let's table the directory layout discussion and chalk it up to something we can solve with views. The features is (mostly) done and working, so I think it wouldn't be a huge pain. There are more complicated things on the table though.

[tgamblin]

@davydden @citibeth:

How do you name the /compiler/ portion of the name then?

in case it's a single compiler family, it would be /compiler/package-,
whereas if you mix two (and two is a max), then /compiler-compiler/packag-. Just need to agree which one is which. Say first - C/CXX, second F77/FC.

in general, I like this. This isn't a complete description of the space, though. You could have different compilers for CC, CXX, F77, and FC. What then? Someone will ask. Should the C/C++ compilers be required to be consistent? Should the Fortran ones? I think your solution covers the space people care about, so maybe these rules are sufficient, but I worry.

That said, I do not think we are covering the full space of compiler mixing yet. In particular, the Intel compilers, in a noble effort to achieve gcc binary compatibility, allow you to specify which gcc they are compatible with, and they do a really weird search of your environment to find out what version of gcc you have installed in order to decide what to do by default. Read this, but wear a helmet so your head doesn't explode.

I think intel builds probably need to be parameterized by which gcc they're built to be compatible with. This essentially means you've got two compiler dependencies -- one on intel and another on the gcc you're targeting for compatibility. So, you may need some other name for that. I think hyphens are insufficient for representing both of these types of compiler mixing.

[davydden]

Someone will ask. Should the C/C++ compilers be required to be consistent? Should the Fortran ones? I think your solution covers the space people care about, so maybe these rules are sufficient, but I worry.

i guess one has to constrain things at some point. I would check for consistency between C/CXX and F77/FC. Same applies to the @@adamjstewart 's idea about checking for null to decide on mixing. That is, both C/CXX or F77/FC should be null. If that's not the case, nothing should be mixed, a kind of sanity check.

[tgamblin]

@davydden: this sounds reasonable. I do not currently have time to get this implemented, though, and I think it should wait until after #378, #561, and probably #360 get merged in. Those are going to add some changes that will conflict with any implementation of this.

What you can actually do right now is just edit compilers.yaml and add version suffixes, e.g.:

compilers:
    darwin-x86_64:
        nag@5.0-clang-3.8:
            # etc
    darwin-x86_64:
        nag@5.0-gcc-5.0:
            # etc

This isn't wonderful, but it does get the information into the spec, and it provides a way to specify things on the command line.

It does not handle the Intel/gcc issue I brought up above.

[adamjstewart]

@tgamblin: This works for me for now. The only problem I see is that some packages build differently when='%compiler'. Right now I need to add F77FLAGS=-mismatch for pretty much everything built with NAG. I also need to use -Wl,-Wl,, instead of -Wl, for NAG. If we start allowing people to use arbitrary names in their compilers.yaml, then this will break.

[davydden]

It does not handle the Intel/gcc issue I brought up above.

@tgamblin i think this would also break some packages, for example in PETSc there is

if self.compiler.name == "clang":
    #do-something-else

That, i suppose is bound to be broken. I am with @adamjstewart here that one should not allow people to name compilers arbitrary. I would keep the naming fixed and add mixing logic as well as

if self.fortran_compiler.name == "gcc" # a bit strange name, but it's alredy used to denote GNU

In case there is no mixing of compilers, the two should return the same family.

[tgamblin]

If the workaround above is good enough for now, can we close this? I think a broader discussion of compilers as "proper" dependencies is in order, but I am not sure when we'll get to it and there is an increasing number of issues to deal with.

[bartlettroscoe]

What is the workaround? I need to figure out how to mix clang C++ and C with gfortran to build things like 'hdf5' and 'cgns'.

[citibeth]

I believe it's pretty standard now, and in the Spack manual.

…

On Fri, Aug 2, 2019 at 4:27 PM Roscoe A. Bartlett ***@***.***> wrote: What is the workaround? I need to figure out how to mix clang C++ and C with gfortran to build things like 'hdf5' and 'cgns'. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#568?email_source=notifications&email_token=AAOVY56TZZUJ4HWLLAOA55DQCSKBHA5CNFSM4B6O4JUKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3OYJMA#issuecomment-517833904>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAOVY56X2VUOW3MBXNEYJ6DQCSKBHANCNFSM4B6O4JUA> .

[bartlettroscoe]

Okay, as per:

• https://spack.readthedocs.io/en/latest/tutorial_configuration.html?highlight=clang%20fortran#compiler-configuration

we have to manually modify the generated compilers.yaml file to add the path for 'f77' to point to the gfortran that Spack isntalled? How can I automated that?