Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installation doesn't find configured packages #146

Open
2 of 13 tasks
fbonhoff opened this issue Jun 21, 2024 · 8 comments
Open
2 of 13 tasks

Installation doesn't find configured packages #146

fbonhoff opened this issue Jun 21, 2024 · 8 comments

Comments

@fbonhoff
Copy link

Issue type

  • New feature
  • Bug
  • Discussion
  • Other

Relates to

  • OPFLOW
  • SOPFLOW
  • SCOPFLOW
  • TCOPFLOW
  • CMake build system
  • Spack configuration
  • Manual
  • Web docs
  • Other

Summary

I have followed the Manual installation as carefully as possieble, installed all dependencies ( PETSC 3.16.0, CMake 3.22.1, OpenMPI 4.1.2 and blas) and the build call runs, but the make -j call fails.

I installed petsc in a non-standard directory and and it passed the make test all call. The Environment variables $PETSC_DIR and $PETSC_ARCH are set specific to that installation. The pther dependencies are installed in standard locations.

I also tried installation using spacks but it failed also. I have no experience with spacks just gave it a quick try. I tried different petsc versions and installations but never managed to fully run make install.

Exact commands to reproduce, if applicable
The cmake call:

cmake .. -DCMAKE_INSTALL_PREFIX=$INSTALLDIR -DPETSC_DIR=$PETSC_DIR -DPETSC_ARCH="petsc-3-16" -DCMAKE_BUILD_TYPE=Debug

followed by:
make install
or
make -j

which fails.

I am running everything on a virtual linux machine (Ubuntu).

Relevant logs and/or screenshots, if applicable

The successful cmake call specifically finding petsc:

cmake .. -DCMAKE_INSTALL_PREFIX=$INSTALLDIR -DPETSC_DIR=$PETSC_DIR -DPETSC_ARCH="petsc-3-16" -DCMAKE_BUILD_TYPE=Debug
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Submodule update
-- Found MPI_C: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so (found version "3.1") 
-- Found MPI_CXX: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi_cxx.so (found version "3.1") 
-- Found MPI: TRUE (found version "3.1") found components: C CXX 
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.2") 
-- Checking for module 'PETSc'
--   Found PETSc, version 3.16.0
-- Ipopt library not found! Please provide correct filepath.
-- Ipopt include directory  not found! Please provide correct path.
-- Looking for sgemm_
-- Looking for sgemm_ - not found
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Looking for sgemm_
-- Looking for sgemm_ - found
-- Found BLAS: /usr/lib/x86_64-linux-gnu/libblas.so  
-- Looking for cheev_
-- Looking for cheev_ - not found
-- Looking for cheev_
-- Looking for cheev_ - not found
-- Looking for cheev_
-- Looking for cheev_ - found
-- Found LAPACK: /usr/lib/x86_64-linux-gnu/liblapack.so;/usr/lib/x86_64-linux-gnu/libblas.so  
-- Found LAPACK libraries: /usr/lib/x86_64-linux-gnu/liblapack.so;/usr/lib/x86_64-linux-gnu/libblas.so
-- Installing: /home/bof/libraries/include/ExaGO/build/options/pflowoptions
-- Installing: /home/bof/libraries/include/ExaGO/build/options/opflowoptions
-- Installing: /home/bof/libraries/include/ExaGO/build/options/hiop.options
-- Installing: /home/bof/libraries/include/ExaGO/build/options/scopflowoptions
-- Installing: /home/bof/libraries/include/ExaGO/build/options/sopflowoptions
-- Installing: /home/bof/libraries/include/ExaGO/build/options/tcopflowoptions
-- Installing: /home/bof/libraries/include/ExaGO/build/options/ipopt.opt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/case118.m
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/case_ACTIVSg200.m
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/case118.cont
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/case_ACTIVSg200.cont
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/10_scenarios_ACTIVSg200.csv
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/10_scenarios_ACTIVSg2000.csv
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/case9
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/case9/case9mod.m
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/case9/case9mod_loadloss.m
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/case9/scenarios_9bus.csv
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/case9/case9mod_gen3_wind.m
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/case9/load_P.csv
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/case9/case9_pw.con
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/case9/case9mod_gen3_wind2.m
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/case9/load_Q.csv
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/case9/case9.con
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/case9/case9.cont
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/case9/10_scenarios_9bus.csv
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/TAMU200_scenarios
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/TAMU200_scenarios/scenarios_200bus.csv
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/TAMU200_scenarios/load_P.csv
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/TAMU200_scenarios/actuals.csv
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/TAMU200_scenarios/load_Q.csv
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/unit
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/unit/opflow
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/unit/opflow/gradient
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/unit/opflow/gradient/OFG_unittest1.m
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/unit/opflow/gradient/OFG_unittestx3.m
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/unit/opflow/gradient/OFG_unittestx600.m
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/unit/opflow/objective
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/unit/opflow/objective/OF_unittest1.m
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/unit/opflow/objective/OF_unittestx2.m
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/unit/opflow/objective/OF_unittestx3.m
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/unit/opflow/objective/OF_unittestx600.m
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case9mod.m
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case9mod.m/obj_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case9mod.m/grad_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case9mod.m/Gu_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case9mod.m/Xl_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case9mod.m/Jeq_valid.bin
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case9mod.m/G_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case9mod.m/Jineq_valid.bin.info
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case9mod.m/obj_value_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case9mod.m/Gl_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case9mod.m/Lambda_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case9mod.m/X_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case9mod.m/obj_factor_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case9mod.m/Jineq_valid.bin
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case9mod.m/Hess_valid.bin.info
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case9mod.m/Xu_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case9mod.m/Jeq_valid.bin.info
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case9mod.m/Hess_valid.bin
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case_ACTIVSg200.m
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case_ACTIVSg200.m/obj_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case_ACTIVSg200.m/grad_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case_ACTIVSg200.m/Gu_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case_ACTIVSg200.m/Xl_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case_ACTIVSg200.m/Jeq_valid.bin
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case_ACTIVSg200.m/G_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case_ACTIVSg200.m/Jineq_valid.bin.info
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case_ACTIVSg200.m/obj_value_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case_ACTIVSg200.m/Gl_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case_ACTIVSg200.m/Lambda_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case_ACTIVSg200.m/X_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case_ACTIVSg200.m/obj_factor_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case_ACTIVSg200.m/Jineq_valid.bin
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case_ACTIVSg200.m/Hess_valid.bin.info
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case_ACTIVSg200.m/Xu_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case_ACTIVSg200.m/Jeq_valid.bin.info
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case_ACTIVSg200.m/Hess_valid.bin
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case118.m
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case118.m/obj_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case118.m/grad_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case118.m/Gu_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case118.m/Xl_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case118.m/Jeq_valid.bin
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case118.m/G_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case118.m/obj_value_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case118.m/Gl_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case118.m/Lambda_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case118.m/X_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case118.m/obj_factor_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case118.m/Hess_valid.bin.info
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case118.m/Xu_valid.txt
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case118.m/Jeq_valid.bin.info
-- Installing: /home/bof/libraries/include/ExaGO/build/datafiles/test_validation/case118.m/Hess_valid.bin
-- Found Python: /home/bof/libraries/include/Exagopy/bin/python3.10 (found version "3.10.12") found components: Interpreter Development Development.Module Development.Embed 
-- Found Python interpreter: /home/bof/libraries/include/Exagopy/bin/python3.10
-- Using Python site library prefix: lib/python3.10/site-packages
-- ExaGO Python bindings will be installed in: /home/bof/libraries/include/ExaGO/install/lib/python3.10/site-packages
-- pybind11 v2.12.0 dev1
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- Found mpi4py: /home/bof/libraries/include/Exagopy/lib/python3.10/site-packages/mpi4py  
-- Found pytest: /home/bof/libraries/include/Exagopy/lib/python3.10/site-packages/pytest  
-- Configuring tests
-- Configuring PFLOW Functionality Tests.
-- Configuring TCOPFLOW functionality tests
-- Configuring SCOPFLOW Functionality Tests.
-- Configuring SOPFLOW functionality tests
-- Configuring done
-- Generating done
-- Build files have been written to: /home/bof/libraries/include/ExaGO/build

And the error message for make -j:

[  2%] Building CXX object src/utils/CMakeFiles/UTILS_obj_static.dir/scopflowutils.cpp.o
[  2%] Building CXX object src/utils/CMakeFiles/UTILS_obj_static.dir/comm.cpp.o
[  4%] Building CXX object src/utils/CMakeFiles/UTILS_obj_static.dir/utils.cpp.o
[  4%] Building CXX object src/utils/CMakeFiles/UTILS_obj_static.dir/version.cpp.o
In file included from /home/bof/libraries/include/ExaGO/src/utils/comm.cpp:1:
/home/bof/libraries/include/ExaGO/include/common.h:9:10: fatal error: petsc.h: No such file or directory
    9 | #include <petsc.h>
      |          ^~~~~~~~~
In file included from /home/bof/libraries/include/ExaGO/src/utils/utils.cpp:1:
/home/bof/libraries/include/ExaGO/include/common.h:9:10: fatal error: petsc.h: No such file or directory
    9 | #include <petsc.h>
      |          ^~~~~~~~~
compilation terminated.
compilation terminated.
In file included from /home/bof/libraries/include/ExaGO/src/utils/version.cpp:1:
/home/bof/libraries/include/ExaGO/include/common.h:9:10: fatal error: petsc.h: No such file or directory
    9 | #include <petsc.h>
      |          ^~~~~~~~~
compilation terminated.
make[2]: *** [src/utils/CMakeFiles/UTILS_obj_static.dir/build.make:118: src/utils/CMakeFiles/UTILS_obj_static.dir/comm.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: *** [src/utils/CMakeFiles/UTILS_obj_static.dir/build.make:76: src/utils/CMakeFiles/UTILS_obj_static.dir/utils.cpp.o] Error 1
make[2]: *** [src/utils/CMakeFiles/UTILS_obj_static.dir/build.make:90: src/utils/CMakeFiles/UTILS_obj_static.dir/version.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:509: src/utils/CMakeFiles/UTILS_obj_static.dir/all] Error 2
make: *** [Makefile:146: all] Error 2

make install throws a similar message regarding petsc.h

p.s. I am fairly unexperienced, if I can approve on describing the issue please let me know.

@abhyshr
Copy link
Collaborator

abhyshr commented Jun 21, 2024

What do you have for PETSC_INCLUDE_DIR in build/CMakeCache.txt? Can you send this file? @cameronrutherford @jaelynlitz

@fbonhoff
Copy link
Author

CMakeCache.txt

Of course!

I attached the file and also copied out the petsc relevant parts:

PETSC_CFLAGS:INTERNAL=-I/home/bof/libraries/include/petsc-3.16.0/install/include PETSC_CFLAGS_I:INTERNAL= PETSC_CFLAGS_OTHER:INTERNAL= PETSC_FOUND:INTERNAL=1 PETSC_INCLUDEDIR:INTERNAL=/home/bof/libraries/include/petsc-3.16.0/install/include PETSC_INCLUDE_DIRS:INTERNAL=/home/bof/libraries/include/petsc-3.16.0/install/include PETSC_LDFLAGS:INTERNAL=-L/home/bof/libraries/include/petsc-3.16.0/install/lib;-lpetsc PETSC_LDFLAGS_OTHER:INTERNAL= PETSC_LIBDIR:INTERNAL=/home/bof/libraries/include/petsc-3.16.0/install/lib PETSC_LIBRARIES:INTERNAL=petsc PETSC_LIBRARY_DIRS:INTERNAL=/home/bof/libraries/include/petsc-3.16.0/install/lib PETSC_LIBS:INTERNAL= PETSC_LIBS_L:INTERNAL= PETSC_LIBS_OTHER:INTERNAL= PETSC_LIBS_PATHS:INTERNAL= PETSC_MODULE_NAME:INTERNAL=PETSc PETSC_PETSc_INCLUDEDIR:INTERNAL= PETSC_PETSc_LIBDIR:INTERNAL= PETSC_PETSc_PREFIX:INTERNAL= PETSC_PETSc_VERSION:INTERNAL= PETSC_PREFIX:INTERNAL=/home/bof/libraries/include/petsc-3.16.0/install PETSC_STATIC_CFLAGS:INTERNAL=-I/home/bof/libraries/include/petsc-3.16.0/install/include PETSC_STATIC_CFLAGS_I:INTERNAL= PETSC_STATIC_CFLAGS_OTHER:INTERNAL= PETSC_STATIC_INCLUDE_DIRS:INTERNAL=/home/bof/libraries/include/petsc-3.16.0/install/include PETSC_STATIC_LDFLAGS:INTERNAL=-L/home/bof/libraries/include/petsc-3.16.0/install/lib;-L/usr/lib/gcc/x86_64-linux-gnu/11;-lpetsc;-lflapack;-lfblas;-lm;-lX11;-lstdc++;-ldl;-lmpifort;-lmpi;-lgfortran;-lm;-lgfortran;-lm;-lgcc_s;-lquadmath;-lstdc++;-ldl PETSC_STATIC_LDFLAGS_OTHER:INTERNAL= PETSC_STATIC_LIBDIR:INTERNAL= PETSC_STATIC_LIBRARIES:INTERNAL=petsc;flapack;fblas;m;X11;stdc++;dl;mpifort;mpi;gfortran;m;gfortran;m;gcc_s;quadmath;stdc++;dl PETSC_STATIC_LIBRARY_DIRS:INTERNAL=/home/bof/libraries/include/petsc-3.16.0/install/lib;/usr/lib/gcc/x86_64-linux-gnu/11 PETSC_STATIC_LIBS:INTERNAL= PETSC_STATIC_LIBS_L:INTERNAL= PETSC_STATIC_LIBS_OTHER:INTERNAL= PETSC_STATIC_LIBS_PATHS:INTERNAL= PETSC_VERSION:INTERNAL=3.16.0

@cameronrutherford
Copy link
Contributor

I am a little confused about what's going wrong here. If possible, I would suggest in updating PETSc to a newer version, and running with VERBOSE=1 make to get more detailed logs.

Since you aren't building with Ipopt, and you are running on a linux box, have you considered using our ghcr package to get started? You can try pulling this version of the ExaGO package for example.

Based on our .devcontainer/Dockerfile, something like this in a Dockerfile should get you started:

# Build stage with Spack pre-installed and ready to be used
FROM spack/ubuntu-jammy as builder

# Install OS packages needed to build the software
RUN apt-get -yqq update && apt-get -yqq upgrade \
 && apt-get -yqq install autoconf \
 && rm -rf /var/lib/apt/lists/*

# What we want to install and how we want to install it
# is specified in a manifest file (spack.yaml)
RUN mkdir /opt/spack-environment \
&&  (echo spack: \
&&   echo '  specs:' \
&&   echo '  - exago@1.6.0~ipopt+python+mpi' \
&&   echo '    ^libffi@3.4.2' \
&&   echo '    ^petsc~fortran~hdf5~hypre+metis' \
&&   echo '  concretizer:' \
&&   echo '    unify: true' \
&&   echo '    reuse: true' \
&&   echo '  mirrors:' \
&&   echo '    spack: https://binaries.spack.io/develop' \
&&   echo '  packages:' \
&&   echo '    all:' \
&&   echo '      providers:' \
&&   echo '        mpi:' \
&&   echo '        - openmpi' \
&&   echo '        zlib-api:' \
&&   echo '        - zlib' \
&&   echo '    zlib-ng:' \
&&   echo '      buildable: false' \
&&   echo '    mpich:' \
&&   echo '      buildable: false' \
&&   echo '  config:' \
&&   echo '    install_tree: /opt/software' \
&&   echo '  view: /opt/views/view') > /opt/spack-environment/spack.yaml

# Find external packages
RUN cd /opt/spack-environment && spack env activate . && spack external find --all --exclude python

# Do this separate of install to cache keys...
RUN cd /opt/spack-environment && spack env activate . && spack mirror add develop https://binaries.spack.io/develop && spack buildcache keys --install --trust && spack concretize -f && ( spack mirror create -a || true )

# Install buggy linux-uuid package
RUN cd /opt/spack-environment && spack env activate . && spack install --no-checksum --fail-fast util-linux-uuid

# Install PETSc
RUN cd /opt/spack-environment && spack env activate . && spack install --fail-fast petsc

# Install the software, remove unnecessary deps
RUN cd /opt/spack-environment && spack env activate . && spack install --fail-fast && spack gc -y

# Strip all the binaries
RUN find -L /opt/views/view/* -type f -exec readlink -f '{}' \; | \
    xargs file -i | \
    grep 'charset=binary' | \
    grep 'x-executable\|x-archive\|x-sharedlib' | \
    awk -F: '{print $1}' | xargs strip

# Modifications to the environment that are necessary to run
RUN cd /opt/spack-environment && \
    spack env activate --sh -d . > activate.sh

RUN { \
      echo '#!/bin/sh' \
      && echo '.' /opt/spack-environment/activate.sh \
      && echo 'exec "$@"'; \
    } > /entrypoint.sh \
&& chmod a+x /entrypoint.sh \
&& ln -s /opt/views/view /opt/view

RUN apt-get -yqq update && apt-get -yqq upgrade \
 && apt-get -yqq install gfortran \
 && rm -rf /var/lib/apt/lists/*
ENTRYPOINT [ "/entrypoint.sh" ]
CMD [ "/bin/bash" ]

# Make sure root user gets spack packages
RUN echo "source /entrypoint.sh" >> /root/.bashrc

@fbonhoff
Copy link
Author

Hi Cameron, haven't considered but I will try.
I am specifically using that version of PETsc as it was mentioned in the dependencies and I couldn't figure out where the errors came from using the latest version.

I have not really used docker before, but probably something that I should do anyway so I will see how far I get with that, thanks!

@fbonhoff
Copy link
Author

Running the above as a Dockefile fails while installing PETsc and/or Blas:
I saved the script in Dockerfile, and with docker installed I ran:
docker build -t spack-environment .

 => ERROR [ 7/13] RUN cd /opt/spack-environment && spack env activate . && spack install --fail-fast petsc                                                                                                                     203.7s 
------                                                                                                                                                                                                                                
 > [ 7/13] RUN cd /opt/spack-environment && spack env activate . && spack install --fail-fast petsc:                                                                                                                                  
2.762 [+] /usr (external diffutils-3.8-enmvrffztilwxiikkrb6ecvy5hsjb56b)
2.762 [+] /usr (external glibc-2.35-kpqgx3w6iph5fz3t3bwf66r3a2zlblv4)
2.762 [+] /usr (external glibc-2.27-yw5erx4enja73o74rurllfukltkfjtzt)
2.762 [+] /usr (external curl-7.81.0-2t2wsy7vky7ld6wf2lvvn7osdjtwz732)
2.762 [+] /usr (external perl-5.34.0-dgirojtbssabs6o6udwbfodykrithqnq)
2.762 [+] /usr (external autoconf-2.71-qjwbrrz66fjbtmyyvz7uoal6lk5kjaen)
2.762 [+] /usr (external automake-1.16.5-6k34gurkwqdf2xhql665tpfmvai2irfe)
2.762 [+] /usr (external findutils-4.8.0-spqruwhfeqf4lnkpoz37k26vybh6eaob)
2.762 [+] /usr (external m4-1.4.18-6g7hqnfvtkqwgf5fkdnanehxc6orv3ea)
2.762 [+] /usr (external xz-5.2.5-kemc5sdpotneyvzffbjent5dlxjvyttd)
2.762 [+] /usr (external openssl-3.0.2-btqjat2hi4z2idnnjcdx6l65wp7i5kes)
2.762 [+] /usr (external bzip2-1.0.8-nftzrmlqhx7fpux4ljkko7jckoa7oedc)
2.762 [+] /usr (external tar-1.34-c326xrdcga5uxzcaxnwzbrk5r3td46sq)
2.762 [+] /opt/software/linux-ubuntu22.04-x86_64/gcc-11.4.0/gcc-runtime-11.4.0-v2ijgukwrehpr6hdzre65vab6npz2nxl
2.762 [+] /opt/software/linux-ubuntu18.04-x86_64/gcc-7.5.0/gcc-runtime-7.5.0-wfzklsvdan4cn25cyn5qqxm2wwywcet2
2.762 [+] /opt/software/linux-ubuntu22.04-x86_64/gcc-11.4.0/util-linux-uuid-2.38.1-v4h7c57cxrnrl6nnns7ua67vc6e7yyyh
2.762 [+] /opt/software/linux-ubuntu18.04-x86_64/gcc-7.5.0/gmake-4.4.1-6wourj6ffjsxgb3xjb2pl4dij446y5w3
2.762 ==> Installing openblas-0.3.26-ddf7dkrpyb6vxm3dlg3ztxerismsb427 [18/48]
2.762 ==> No binary for openblas-0.3.26-ddf7dkrpyb6vxm3dlg3ztxerismsb427 found: installing from source
3.891 ==> Using cached archive: /opt/spack/var/spack/cache/_source-cache/archive/4e/4e6e4f5cb14c209262e33e6816d70221a2fe49eb69eaf0a06f065598ac602c68.tar.gz
3.891 ==> No patches needed for openblas
3.895 ==> openblas: Executing phase: 'edit'
3.906 ==> openblas: Executing phase: 'build'
201.4 ==> Error: ProcessError: Command exited with status 2:
201.4     'make' '-j16' '-s' 'CC=/opt/spack/lib/spack/env/gcc/gcc' 'FC=/opt/spack/lib/spack/env/gcc/gfortran' 'MAKE_NB_JOBS=0' 'ARCH=x86_64' 'DYNAMIC_ARCH=1' 'DYNAMIC_OLDER=1' 'TARGET=GENERIC' 'DYNAMIC_ARCH=1' 'USE_LOCKING=1' 'USE_OPENMP=0' 'USE_THREAD=0' 'RANLIB=ranlib' 'all'
201.5 
201.5 2 errors found in build log:
201.5      2853    Backtrace for this error:
201.5      2854    #0  0x7f7eafef7960 in ???
201.5      2855    #1  0x7f7eafef6ac5 in ???
201.5      2856    #2  0x7f7eafcea51f in ???
201.5      2857    #3  0x560588809208 in sgemm_oncopy_OPTERON_SSE3
201.5      2858    Illegal instruction (core dumped)
201.5   >> 2859    make[1]: *** [Makefile:167: level3] Error 132
201.5      2860    make[1]: *** Waiting for unfinished jobs....
201.5   >> 2861    make: *** [Makefile:160: tests] Error 2
201.5 
201.5 See build log for details:
201.5   /tmp/root/spack-stage/spack-stage-openblas-0.3.26-ddf7dkrpyb6vxm3dlg3ztxerismsb427/spack-build-out.txt
201.5 
201.5 ==> Warning: Skipping build of petsc-3.21.1-zduwixjogwl547jluo7qhnh22246zxsy since openblas-0.3.26-ddf7dkrpyb6vxm3dlg3ztxerismsb427 failed
201.6 ==> Updating view at /opt/views/view
201.6 ==> Error: Terminating after first install failure: ProcessError: Command exited with status 2:
201.6     'make' '-j16' '-s' 'CC=/opt/spack/lib/spack/env/gcc/gcc' 'FC=/opt/spack/lib/spack/env/gcc/gfortran' 'MAKE_NB_JOBS=0' 'ARCH=x86_64' 'DYNAMIC_ARCH=1' 'DYNAMIC_OLDER=1' 'TARGET=GENERIC' 'DYNAMIC_ARCH=1' 'USE_LOCKING=1' 'USE_OPENMP=0' 'USE_THREAD=0' 'RANLIB=ranlib' 'all'
------
Dockerfile:47
--------------------
  45 |     
  46 |     # Install PETSc
  47 | >>> RUN cd /opt/spack-environment && spack env activate . && spack install --fail-fast petsc
  48 |     
  49 |     # Install the software, remove unnecessary deps
--------------------
ERROR: failed to solve: process "docker-shell cd /opt/spack-environment && spack env activate . && spack install --fail-fast petsc" did not complete successfully: exit code: 1

@fbonhoff
Copy link
Author

The linked docker image works though.. Is there also an available docker image including GPU support ? And if so, how can I find the available images?

Thank you already though to get me started!

@cameronrutherford
Copy link
Contributor

The linked docker image works though.. Is there also an available docker image including GPU support ? And if so, how can I find the available images?

Thank you already though to get me started!

We don't currently have a docker image that we build / publish with GPU support unfortunately. We really need CoinHSL in order to get most of ExaGO's complex applications functioning, so this for a while has blocked us supporting builds for SCOPFLOW / SOPFLOW and GPU builds out of the box.

I suppose we could go a ghcr image for CUDA / ROCm as well, but again they wouldn't have SOPFLOW / SCOPFLOW supported...

@cameronrutherford
Copy link
Contributor

Running the above as a Dockefile fails while installing PETsc and/or Blas: I saved the script in Dockerfile, and with docker installed I ran: docker build -t spack-environment .

 => ERROR [ 7/13] RUN cd /opt/spack-environment && spack env activate . && spack install --fail-fast petsc                                                                                                                     203.7s 
------                                                                                                                                                                                                                                
 > [ 7/13] RUN cd /opt/spack-environment && spack env activate . && spack install --fail-fast petsc:                                                                                                                                  
2.762 [+] /usr (external diffutils-3.8-enmvrffztilwxiikkrb6ecvy5hsjb56b)
2.762 [+] /usr (external glibc-2.35-kpqgx3w6iph5fz3t3bwf66r3a2zlblv4)
2.762 [+] /usr (external glibc-2.27-yw5erx4enja73o74rurllfukltkfjtzt)
2.762 [+] /usr (external curl-7.81.0-2t2wsy7vky7ld6wf2lvvn7osdjtwz732)
2.762 [+] /usr (external perl-5.34.0-dgirojtbssabs6o6udwbfodykrithqnq)
2.762 [+] /usr (external autoconf-2.71-qjwbrrz66fjbtmyyvz7uoal6lk5kjaen)
2.762 [+] /usr (external automake-1.16.5-6k34gurkwqdf2xhql665tpfmvai2irfe)
2.762 [+] /usr (external findutils-4.8.0-spqruwhfeqf4lnkpoz37k26vybh6eaob)
2.762 [+] /usr (external m4-1.4.18-6g7hqnfvtkqwgf5fkdnanehxc6orv3ea)
2.762 [+] /usr (external xz-5.2.5-kemc5sdpotneyvzffbjent5dlxjvyttd)
2.762 [+] /usr (external openssl-3.0.2-btqjat2hi4z2idnnjcdx6l65wp7i5kes)
2.762 [+] /usr (external bzip2-1.0.8-nftzrmlqhx7fpux4ljkko7jckoa7oedc)
2.762 [+] /usr (external tar-1.34-c326xrdcga5uxzcaxnwzbrk5r3td46sq)
2.762 [+] /opt/software/linux-ubuntu22.04-x86_64/gcc-11.4.0/gcc-runtime-11.4.0-v2ijgukwrehpr6hdzre65vab6npz2nxl
2.762 [+] /opt/software/linux-ubuntu18.04-x86_64/gcc-7.5.0/gcc-runtime-7.5.0-wfzklsvdan4cn25cyn5qqxm2wwywcet2
2.762 [+] /opt/software/linux-ubuntu22.04-x86_64/gcc-11.4.0/util-linux-uuid-2.38.1-v4h7c57cxrnrl6nnns7ua67vc6e7yyyh
2.762 [+] /opt/software/linux-ubuntu18.04-x86_64/gcc-7.5.0/gmake-4.4.1-6wourj6ffjsxgb3xjb2pl4dij446y5w3
2.762 ==> Installing openblas-0.3.26-ddf7dkrpyb6vxm3dlg3ztxerismsb427 [18/48]
2.762 ==> No binary for openblas-0.3.26-ddf7dkrpyb6vxm3dlg3ztxerismsb427 found: installing from source
3.891 ==> Using cached archive: /opt/spack/var/spack/cache/_source-cache/archive/4e/4e6e4f5cb14c209262e33e6816d70221a2fe49eb69eaf0a06f065598ac602c68.tar.gz
3.891 ==> No patches needed for openblas
3.895 ==> openblas: Executing phase: 'edit'
3.906 ==> openblas: Executing phase: 'build'
201.4 ==> Error: ProcessError: Command exited with status 2:
201.4     'make' '-j16' '-s' 'CC=/opt/spack/lib/spack/env/gcc/gcc' 'FC=/opt/spack/lib/spack/env/gcc/gfortran' 'MAKE_NB_JOBS=0' 'ARCH=x86_64' 'DYNAMIC_ARCH=1' 'DYNAMIC_OLDER=1' 'TARGET=GENERIC' 'DYNAMIC_ARCH=1' 'USE_LOCKING=1' 'USE_OPENMP=0' 'USE_THREAD=0' 'RANLIB=ranlib' 'all'
201.5 
201.5 2 errors found in build log:
201.5      2853    Backtrace for this error:
201.5      2854    #0  0x7f7eafef7960 in ???
201.5      2855    #1  0x7f7eafef6ac5 in ???
201.5      2856    #2  0x7f7eafcea51f in ???
201.5      2857    #3  0x560588809208 in sgemm_oncopy_OPTERON_SSE3
201.5      2858    Illegal instruction (core dumped)
201.5   >> 2859    make[1]: *** [Makefile:167: level3] Error 132
201.5      2860    make[1]: *** Waiting for unfinished jobs....
201.5   >> 2861    make: *** [Makefile:160: tests] Error 2
201.5 
201.5 See build log for details:
201.5   /tmp/root/spack-stage/spack-stage-openblas-0.3.26-ddf7dkrpyb6vxm3dlg3ztxerismsb427/spack-build-out.txt
201.5 
201.5 ==> Warning: Skipping build of petsc-3.21.1-zduwixjogwl547jluo7qhnh22246zxsy since openblas-0.3.26-ddf7dkrpyb6vxm3dlg3ztxerismsb427 failed
201.6 ==> Updating view at /opt/views/view
201.6 ==> Error: Terminating after first install failure: ProcessError: Command exited with status 2:
201.6     'make' '-j16' '-s' 'CC=/opt/spack/lib/spack/env/gcc/gcc' 'FC=/opt/spack/lib/spack/env/gcc/gfortran' 'MAKE_NB_JOBS=0' 'ARCH=x86_64' 'DYNAMIC_ARCH=1' 'DYNAMIC_OLDER=1' 'TARGET=GENERIC' 'DYNAMIC_ARCH=1' 'USE_LOCKING=1' 'USE_OPENMP=0' 'USE_THREAD=0' 'RANLIB=ranlib' 'all'
------
Dockerfile:47
--------------------
  45 |     
  46 |     # Install PETSc
  47 | >>> RUN cd /opt/spack-environment && spack env activate . && spack install --fail-fast petsc
  48 |     
  49 |     # Install the software, remove unnecessary deps
--------------------
ERROR: failed to solve: process "docker-shell cd /opt/spack-environment && spack env activate . && spack install --fail-fast petsc" did not complete successfully: exit code: 1

That's quite the ugly error... If you add a copy of the line RUN cd /opt/spack-environment && spack env activate . && spack install --fail-fast petsc with opeblas instead of petsc, and then specify the openblas version in your spack: spec in the dockerfile with ^openblas@version, you should be able to iterate through a few openblas versions (listed here) to see if it's just the latest version that's buggy.

If you can't get anywhere with that, it might be worth reporting a bug with the openblas folk...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants