trilinos recipe: spec['blas'].prefix.lib is wrong for mkl

[KineticTheory]

I can no longer build trilinos ^mkl because spec['blas'].prefix.lib does not point to the correct directory for MKL. At least for my installation, the MKL libraries are located at $MKLROOT/lib/intel64 and not at $MKLROOT/lib. I'm not sure how to fix this because I don't know if there is significant variation in directory names for MKL installations.

It is simple to modify the trilinos recipe to do something special for MKL. Should I submit a PR with these changes? Does anyone else have experience building trilinos against MKL? Are there other configurations that I should be aware of?

[adamjstewart]

@alalazo @davydden This is an external package not built by Spack, so the directory layout doesn't match. We're going to have to figure out some way to handle this...

[tgamblin]

Specs are marked external so we have the information. May need to record it better.

[davydden]

I can no longer build trilinos ^mkl because spec['blas'].prefix.lib does not point to the correct directory for MKL.

you can't use external MKL, you should build it with Spack, then symlinks will be created. That way Trilinos is buildable with MKL.

[davydden]

p.s. tested on on Ubuntu@16.04+gcc@5.4.0 in this PR #1852

[adamjstewart]

I think we need to make external MKL work. A lot of users will already have it installed. Can't we search for libraries recursively instead of asserting that they have to be in prefix.lib?

[davydden]

I think we need to make external MKL work.

essentially that would amount to changing everywhere spec['blas'].prefix.lib to spec['blas']. directories (the latter is a list, AFAIK). Same for lapack and scalapack.
This would rely on LibraryList class, introduced in #1682 .

[davydden]

@KineticTheory

It is simple to modify the trilinos recipe to do something special for MKL. Should I submit a PR with these changes?

do you want to try this for Trilinos and submit a PR? If that works for your external MKL, the other packages can be gradually switched to this scheme.

[adamjstewart]

@KineticTheory For the record, this is a modification that should be made in the mkl package, not in the trilinos package.

[davydden]

@KineticTheory For the record, this is a modification that should be made in the mkl package, not in the trilinos package.

Not exactly. It should be done in both places.
In trilinos:

spec['blas'].prefix.lib -> ";".join(blas.directories)

and in MKL actually you don't need to do anything as

mkl_libs = find_libraries(
            mkl_integer + ['libmkl_core'] + mkl_threading,
            root=join_path(self.prefix.lib, 'intel64'),
            shared=shared
        )

already searches in native directory structure. One can then remove

# Unfortunately MKL libs are natively distrubted in prefix/lib/intel64.
        # To make MKL play nice with Spack, symlink all files to prefix/lib:
        mkl_lib_dir = os.path.join(prefix, "lib", "intel64")
        for f in os.listdir(mkl_lib_dir):
            os.symlink(os.path.join(mkl_lib_dir, f),
                       os.path.join(self.prefix, "lib", f))

from MKL.

[davydden]

p.s. symlinking was introduced prior to #1682 .

[tgamblin]

With the new LAPACK/BLAS functionality do we need symlinking? I think depending on the native directory structure for externals is better.

[davydden]

With the new LAPACK/BLAS functionality do we need symlinking? I think depending on the native directory structure for externals is better.

we don't need to symlink. It is indeed good to avoid it everywhere.