spack install paraview fails with cmake #3296

certik · 2017-03-01T23:50:00Z

$ spack install paraview
==> Installing paraview
==> Installing bzip2
==> Fetching http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz
######################################################################## 100.0%
==> Staging archive: /home/certik/repos/spack/var/spack/stage/bzip2-1.0.6-4eysx5
2qz5t36t3d4fr2rvdfmptumq4n/bzip2-1.0.6.tar.gz
==> Created stage in /home/certik/repos/spack/var/spack/stage/bzip2-1.0.6-4eysx5
2qz5t36t3d4fr2rvdfmptumq4n
==> Ran patch() for bzip2
==> Building bzip2 [Package]
==> Executing phase : 'install'
==> Successfully installed bzip2
  Fetch: 1.97s.  Build: 3.18s.  Total: 5.16s.
[+] /home/certik/repos/spack/opt/spack/linux-ubuntu16-x86_64/gcc-6.1.1/bzip2-1.0
.6-4eysx52qz5t36t3d4fr2rvdfmptumq4n
==> Installing zlib
==> Fetching http://zlib.net/fossils/zlib-1.2.11.tar.gz
######################################################################## 100.0%
==> Staging archive: /home/certik/repos/spack/var/spack/stage/zlib-1.2.11-cssk2k
ioj3w6z3and7is45gz7d2l6mui/zlib-1.2.11.tar.gz
==> Created stage in /home/certik/repos/spack/var/spack/stage/zlib-1.2.11-cssk2k
ioj3w6z3and7is45gz7d2l6mui
==> Ran patch() for zlib
==> Building zlib [AutotoolsPackage]
==> Executing phase : 'autoreconf'
==> Executing phase : 'configure'
==> Executing phase : 'build'
==> Executing phase : 'install'
==> Successfully installed zlib
  Fetch: 0.75s.  Build: 4.24s.  Total: 4.99s.
[+] /home/certik/repos/spack/opt/spack/linux-ubuntu16-x86_64/gcc-6.1.1/zlib-1.2.
11-cssk2kioj3w6z3and7is45gz7d2l6mui
==> Installing cmake
==> Installing ncurses
==> Fetching http://ftp.gnu.org/pub/gnu/ncurses/ncurses-6.0.tar.gz
######################################################################## 100.0%
==> Staging archive: /home/certik/repos/spack/var/spack/stage/ncurses-6.0-oatrzx
h3bllizmnqlnzlcpgztwyhlidd/ncurses-6.0.tar.gz
==> Created stage in /home/certik/repos/spack/var/spack/stage/ncurses-6.0-oatrzx
h3bllizmnqlnzlcpgztwyhlidd
==> Applied patch sed_pgi.patch
==> Applied patch patch_gcc_5.txt
==> Ran patch() for ncurses
==> Building ncurses [AutotoolsPackage]
==> Executing phase : 'autoreconf'
==> Executing phase : 'configure'
==> Executing phase : 'build'
==> Executing phase : 'install'
==> Successfully installed ncurses
  Fetch: 1.80s.  Build: 25.13s.  Total: 26.94s.
[+] /home/certik/repos/spack/opt/spack/linux-ubuntu16-x86_64/gcc-6.1.1/ncurses-6
.0-oatrzxh3bllizmnqlnzlcpgztwyhlidd
==> Installing openssl
==> zlib is already installed in /home/certik/repos/spack/opt/spack/linux-ubuntu
16-x86_64/gcc-6.1.1/zlib-1.2.11-cssk2kioj3w6z3and7is45gz7d2l6mui
==> Installing perl
==> Fetching http://www.cpan.org/src/5.0/perl-5.24.1.tar.gz
######################################################################## 100.0%
==> Fetching http://search.cpan.org/CPAN/authors/id/M/MI/MIYAGAWA/App-cpanminus-
1.7042.tar.gz
######################################################################## 100.0%
==> Staging archive: /home/certik/repos/spack/var/spack/stage/perl-5.24.1-66uin3
sg4xeeqm7gsou2ojkjupkzruxl/perl-5.24.1.tar.gz
==> Created stage in /home/certik/repos/spack/var/spack/stage/perl-5.24.1-66uin3
sg4xeeqm7gsou2ojkjupkzruxl
==> Staging archive: /home/certik/repos/spack/var/spack/stage/resource-cpanm-66u
in3sg4xeeqm7gsou2ojkjupkzruxl/App-cpanminus-1.7042.tar.gz
==> Created stage in /home/certik/repos/spack/var/spack/stage/resource-cpanm-66u
in3sg4xeeqm7gsou2ojkjupkzruxl
==> Moving resource stage
        source : /home/certik/repos/spack/var/spack/stage/resource-cpanm-66uin3s
g4xeeqm7gsou2ojkjupkzruxl/App-cpanminus-1.7042/
        destination : /home/certik/repos/spack/var/spack/stage/perl-5.24.1-66uin
3sg4xeeqm7gsou2ojkjupkzruxl/perl-5.24.1/cpanm/cpanm
==> No patches needed for perl
==> Building perl [Package]
==> Executing phase : 'install'
==> Successfully installed perl
  Fetch: 2.12s.  Build: 1m 33.04s.  Total: 1m 35.16s.
[+] /home/certik/repos/spack/opt/spack/linux-ubuntu16-x86_64/gcc-6.1.1/perl-5.24
.1-66uin3sg4xeeqm7gsou2ojkjupkzruxl
==> Fetching http://www.openssl.org/source/openssl-1.0.2k.tar.gz
######################################################################## 100.0%
==> Staging archive: /home/certik/repos/spack/var/spack/stage/openssl-1.0.2k-4fa
lmidfhcv4kz3rtxqpdlaqnim6r5ml/openssl-1.0.2k.tar.gz
==> Created stage in /home/certik/repos/spack/var/spack/stage/openssl-1.0.2k-4fa
lmidfhcv4kz3rtxqpdlaqnim6r5ml
==> No patches needed for openssl
==> Building openssl [Package]
==> Executing phase : 'install'
==> Successfully installed openssl
  Fetch: 2.35s.  Build: 2m 59.00s.  Total: 3m 1.34s.
[+] /home/certik/repos/spack/opt/spack/linux-ubuntu16-x86_64/gcc-6.1.1/openssl-1
.0.2k-4falmidfhcv4kz3rtxqpdlaqnim6r5ml
==> Fetching https://cmake.org/files/v3.7/cmake-3.7.2.tar.gz
######################################################################## 100.0%
==> Staging archive: /home/certik/repos/spack/var/spack/stage/cmake-3.7.2-nm4n7t
oyldmcrniaqdxcs6h7jt4blz5d/cmake-3.7.2.tar.gz
==> Created stage in /home/certik/repos/spack/var/spack/stage/cmake-3.7.2-nm4n7t
oyldmcrniaqdxcs6h7jt4blz5d
==> Building cmake [Package]
==> Executing phase : 'install'
==> Error: ProcessError: Command exited with status 7:
    './bootstrap' '--prefix=/home/certik/repos/spack/opt/spack/linux-ubuntu16-x8
6_64/gcc-6.1.1/cmake-3.7.2-nm4n7toyldmcrniaqdxcs6h7jt4blz5d' '--parallel=32' '--
no-system-jsoncpp' '--no-system-libs' '--no-qt-gui' '--' '-DCMAKE_USE_OPENSSL=ON
'
/home/certik/repos/spack/var/spack/repos/builtin/packages/cmake/package.py:126, 
in install:
     91       def install(self, spec, prefix):
     92           # Consistency check
     93           self.validate(spec)
     94   
     95           options = [
     96               '--prefix={0}'.format(prefix),
     97               '--parallel={0}'.format(make_jobs)]
     98           if spec.satisfies("@3.2:"):
     99               options.append(
     100                  # jsoncpp requires CMake to build
     101                  # use CMake-provided library to avoid circular depende
ncy
     102                  '--no-system-jsoncpp'
     103              )
     104  
     105          if '+ownlibs' in spec:
     106              # Build and link to the CMake-provided third-party librari
es
     107              options.append('--no-system-libs')
     108          else:
     109              # Build and link to the Spack-installed third-party librar
ies
     110              options.append('--system-libs')
     111  
     112          if '+qt' in spec:
     113              options.append('--qt-gui')
     114          else:
     115              options.append('--no-qt-gui')
     116  
     117          if '+doc' in spec:
     118              options.append('--sphinx-html')
     119              options.append('--sphinx-man')
     120  
     121          if '+openssl' in spec:
     122              options.append('--')
     123              options.append('-DCMAKE_USE_OPENSSL=ON')
     124  
     125          bootstrap = Executable('./bootstrap')
  >> 126          bootstrap(*options)
     127  
     128          make()
     129          if self.run_tests:
     130              make('test')  # some tests fail, takes forever
     131          make('install')

See build log for details:
  /tmp/certik/spack-stage/spack-stage-Dgr6ow/cmake-3.7.2/spack-build.out

and the spack-build.out file contains:

==> './bootstrap' '--prefix=/home/certik/repos/spack/opt/spack/linux-ubuntu16-x86_64/gcc-6.1.1/cmake-3.7.2-nm4n7toyldmcrniaqdxcs6h7jt4blz5d' '--parallel=32' '--no-system-jsoncpp' '--no-system-libs' '--no-qt-gui' '--' '-DCMAKE_USE_OPENSSL=ON'
---------------------------------------------
CMake 3.7.2, Copyright 2000-2016 Kitware, Inc. and Contributors
C compiler on this system is: /home/certik/repos/spack/lib/spack/env/gcc/gcc 
---------------------------------------------
Error when bootstrapping CMake:
Cannot find appropriate C++ compiler on this system.
Please specify one using environment variable CXX.
See cmake_bootstrap.log for compilers attempted.
---------------------------------------------
Log of errors: /tmp/certik/spack-stage/spack-stage-Dgr6ow/cmake-3.7.2/Bootstrap.cmk/cmake_bootstrap.log
---------------------------------------------

But I have C and C++ compilers installed, e.g.:

$ spack compilers
==> Available compilers
-- gcc ----------------------------------------------------------
gcc@6.1.1  gcc@5.4.0

-- intel --------------------------------------------------------
intel@16.0.2
$ gcc --version
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ g++ --version
g++ (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

The cmake_bootstrap.log file is available at: https://gist.github.com/certik/9084b0cb3282420cab745d0dcaf8183c

The text was updated successfully, but these errors were encountered:

adamjstewart · 2017-03-02T01:36:40Z

Did you recently install C++ compilers? Like since you first installed Spack? You may have to add them to your ~/.spack/linux/compilers.yaml file, or delete it and recreate it with spack compiler find. It doesn't get updated automatically.

certik · 2017-03-02T04:48:16Z

I didn't, but just to make sure I removed the ~/.spack directory and rerun with the same result.

However I discovered by careful examination of the cmake log (https://gist.github.com/certik/9084b0cb3282420cab745d0dcaf8183c) that spack was using the gcc@6.1.1 compiler and that compiler only had Fortran and C installed, but not C++. I think that's a bug in spack, it should be clever enough to figure out that the C++ compiler is missing and fail earlier with a nice error message.

Anyway, the above problem is fixed by doing:

spack install paraview %gcc@5.4.0

Then it builds cmake just fine and continues all the way to building paraview.

So this problem is fixed for me. I am going to keep this issue open if you want to extract the bug about the compiler. Feel free to close it anytime.

davydden · 2017-03-02T05:30:06Z

I think one just need to add a check to Paraview to make sure c++ compiler is present. @certik want to create a PR with the fix?

Ps. Not all packages need all compilers, so it's allowed to miss some compilers

certik · 2017-03-02T06:51:21Z

In this case the C++ compiler is needed for the cmake package, not the paraview package.

How do you check that C++ is present in spack?

davydden · 2017-03-02T06:52:48Z

ah, then this should be in cmake, for example look at openmpi

    @run_before('autoreconf')
    def die_without_fortran(self):
        if (self.compiler.f77 is None) or (self.compiler.fc is None):
            raise InstallError(
                'OpenMPI requires both C and Fortran compilers!'
            )

certik · 2017-03-02T06:54:04Z

There are tons of C++ packages in spack, should all of them have this check? I think perhaps spack itself should handle this, rather than the packages.py.

davydden · 2017-03-02T06:58:42Z

There are tons of C++ packages in spack, should all of them have this check?

Until we have a way to say that a given package needs certain compilers, i.e. #896, then technically all of the C++ package should be checking it. I would not add it everywhere, though, as this is most likely a temporary solution. But it probably still makes sense to check in the most crucial packages, like cmake.

@adamjstewart @alalazo @tgamblin ping.

davydden · 2017-03-02T07:04:28Z

I think perhaps spack itself should handle this, rather than the packages.py

IMO yes and no. Spack should make it easier for packages to declare which compilers they need to build. But each given package should still be able to build only with those compilers needed and others being null.

tgamblin · 2017-03-02T07:15:38Z

@certik @davydden: so we've been putting this off for reasons @davydden mentioned, because we would like to make compilers proper dependencies that provide languages and/or features at particular versions. However, thanks to @certik I was thinking about this and I think we could handle the checks we need (if not the compilers themselves) already.

Suggestion:

Add a package called compiler-check (this could be a zero-dependency MetaPackage when BundlePackage #3133 is merged, which should be soon):

class CompilerCheck(Package):
    ...
    # These will satisfy things like, e.g.:
    #    depends_on('cxx@11:')
    #    depends_on('fortran@90:')
    provides('c')
    provides('cxx')
    provides('fortran')

    # Things can depends_on('openmp@4.0:')
    provides('openmp')

    def setup_dependent_package(self, module, dependent_spec):
        # add checks here, since we can just inspect the dependent's requirements.

    def install(self, spec, prefix):
        # currently need this because we don't have a metapackage,
        # but this won't be needed soon.
        touch('%s/dummy' % prefix)

We can consolidate the check logic in this package, AND we can express the language requirements the way we eventually want to, anyway. Example:

class Paraview(Package):
    ...
    depends_on('cxx@03:', type='build')

Done, in a future proof way, without actually needing to modify the core.

Thoughts? @alalazo @davydden @adamjstewart @certik

davydden · 2017-03-02T07:28:06Z

@tgamblin this looks reasonable to me. Essentially every DAG which have anywhere depends_on(c|cxx|fortran|openmp) will have this dummy/meta CompilerCheck package in it solely for checking purposes, right?

I like depends_on('cxx@11:'). We should probably mark a specific compiler as being c++11 compliant iff it implements the standard in full. For GCC that would probably be 4.8.1.

p.s. i was thinking about moving these checks into Package class and make them trigger from the derived classes like CMake somehow.

tgamblin · 2017-03-02T07:53:42Z

@tgamblin this looks reasonable to me. Essentially every DAG which have anywhere depends_on(c|cxx|fortran|openmp) will have this dummy/meta CompilerCheck package in it solely for checking purposes, right?

Yes. It could be a build dependency, so that it's really only there for checking at build time. We could consider making this type of package special so that it wouldn't actually create an install directory.

I like depends_on('cxx@11:'). We should probably mark a specific compiler as being c++11 compliant iff it implements the standard in full. For [GCC that would probably be 4.8.1].(https://gcc.gnu.org/projects/cxx-status.html#cxx11)

There are certainly complexities with the versioning and the mapping to compilers. I would like to provide these semantics because I think they're simple and easy to read, and I agree that we can be conservative about when we say a compiler actually provides cxx@11.

If we want finer granularity, we can also have this package implement checks for features. e.g. depends_on('cxx_tr01') or depends_on('cxx_lambda'). Those could be separate virtual dependencies and users could get finer-grained if they really wanted to.

Finally, we don't support it right now but once we have the new concretizer, we could add or dependencies and just depend on fine-grained compiler versions -- e.g. a user could depends_on('gcc@4.9.3:' | 'intel@11:' | etc.) or something like that -- I'd have to think about the syntax because the | there clearly won't work out of the box.

p.s. i was thinking about moving these checks into Package class and make it trigger from the derived classes like CMake somehow.

I think using the dependency system is better, as it's not another special case and this way would allow us to easily drop the dependency-ized compilers in. Does that sound better?

alalazo · 2017-03-02T08:55:46Z

I think using the dependency system is better, as it's not another special case and this way would allow us to easily drop the dependency-ized compilers in. Does that sound better?

If we need to have support for compiler dependencies with some urgency, I think this is the way to go. Otherwise I wouldn't mind waiting for a complete (and more consistent?) rework of compilers.

If we want finer granularity, we can also have this package implement checks for features. e.g. depends_on('cxx_tr01') or depends_on('cxx_lambda'). Those could be separate virtual dependencies and users could get finer-grained if they really wanted to.

This can be a rabbit hole... I wouldn't go into that if we are not forced to. For instance lambda may be ambiguous so, if we are supporting finer queries we need to support things like cxx11_lambda, generic_lambda, lambda_capture_expression, etc.

Finally, we don't support it right now but once we have the new concretizer, we could add or dependencies and just depend on fine-grained compiler versions -- e.g. a user could depends_on('gcc@4.9.3:' | 'intel@11:' | etc.) or something like that -- I'd have to think about the syntax because the | there clearly won't work out of the box.

👍

chuckatkins · 2019-12-04T17:57:35Z

So this problem is fixed for me. ... Feel free to close it anytime.

Agreed.

tgamblin mentioned this issue Mar 29, 2017

Re-work shared vs. static builds (Universal Variants) #2492

Closed

alalazo added the build-error label Nov 22, 2017

chuckatkins closed this as completed Dec 4, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

spack install paraview fails with cmake #3296

spack install paraview fails with cmake #3296

certik commented Mar 1, 2017 •

edited

adamjstewart commented Mar 2, 2017

certik commented Mar 2, 2017 •

edited

davydden commented Mar 2, 2017

certik commented Mar 2, 2017

davydden commented Mar 2, 2017

certik commented Mar 2, 2017

davydden commented Mar 2, 2017

davydden commented Mar 2, 2017

tgamblin commented Mar 2, 2017 •

edited

davydden commented Mar 2, 2017 •

edited

tgamblin commented Mar 2, 2017 •

edited

alalazo commented Mar 2, 2017

chuckatkins commented Dec 4, 2019 •

edited

spack install paraview fails with cmake #3296

spack install paraview fails with cmake #3296

Comments

certik commented Mar 1, 2017 • edited

adamjstewart commented Mar 2, 2017

certik commented Mar 2, 2017 • edited

davydden commented Mar 2, 2017

certik commented Mar 2, 2017

davydden commented Mar 2, 2017

certik commented Mar 2, 2017

davydden commented Mar 2, 2017

davydden commented Mar 2, 2017

tgamblin commented Mar 2, 2017 • edited

davydden commented Mar 2, 2017 • edited

tgamblin commented Mar 2, 2017 • edited

alalazo commented Mar 2, 2017

chuckatkins commented Dec 4, 2019 • edited

certik commented Mar 1, 2017 •

edited

certik commented Mar 2, 2017 •

edited

tgamblin commented Mar 2, 2017 •

edited

davydden commented Mar 2, 2017 •

edited

tgamblin commented Mar 2, 2017 •

edited

chuckatkins commented Dec 4, 2019 •

edited