Infrastructure: Trilinos shipping with Ubuntu 18.04 / 20.04 and Debian bullseye (stable) no longer usable #14637

peterrum · 2023-01-04T07:13:56Z

Since tonight some of our nightly-tests using the official Docker image are failing. We have following lines in our CMakeLists.txt file: https://github.com/kronbichler/adaflo/blob/cf7636e90cd530fb5b19985331101f78773d71fd/CMakeLists.txt#L26-L54

During configuration it complains that Trilinos is missing:

-- Configuring incomplete, errors occurred!
adaflo requires a deal.II installation built with support for Trilinos and p4est but one or both of these appears to be missing!

CMake Error at CMakeLists.txt:42 (MESSAGE):
  

  -- deal.II was built without support for Trilinos!

see https://github.com/kronbichler/adaflo/actions/runs/3835904943/jobs/6529539908.

@tamiko Do you have a clue? Locally, I cannot replicate the issue. Is it related to #14634?

FYI @mschreter

The text was updated successfully, but these errors were encountered:

tamiko · 2023-01-04T07:20:04Z

@peterrum You will need to rebuild Trilinos without Kokkos support, or update Trilinos to version 13.2, see #14626

Unfortunately, this has created significantly more fallout than expected, see https://cdash.dealii.org/build/7/configure

peterrum · 2023-01-04T07:55:21Z

The official deal.II Docker image is configured here: https://hub.docker.com/layers/dealii/dealii/latest/images/sha256-f513d6505d293e2b331487ebb383d57aa4d2762c8fa2818d88a32dc5d8a0f291?context=explore

The modules are loaded here: https://hub.docker.com/layers/dealii/dealii/latest/images/sha256-f513d6505d293e2b331487ebb383d57aa4d2762c8fa2818d88a32dc5d8a0f291?context=explore

It looks like that Trilinos is loaded as a dependency of libdeal.ii-dev (https://packages.ubuntu.com/search?keywords=libdeal.ii-dev). On bionic (18.4LTS), trilinos-all-dev (12.12.1-5) is loaded; on focal (20.04LTS) trilinos-all-dev (12.14.1-3), on jammy (22.04LTS) trilinos-all-dev (13.2.0-1ubuntu1) and on kinetic (22.10) trilinos-all-dev (13.2.0-3). Probably the Docker images are using one of the older versions of Ubuntu, which is a bit odd since

dealii/.github/workflows/docker.yml

Line 15 in e06d568

runs-on: ubuntu-latest

specifies ubuntu-latest, which is according to https://github.blog/changelog/2022-11-09-github-actions-ubuntu-latest-workflows-will-use-ubuntu-22-04/ 22.04. @luca-heltai Do you have an insight?

tamiko · 2023-01-04T08:07:53Z

@peterrum @masterleinad

Correct, with this change it is no longer possible to use Trilinos bundled with anything older than Ubuntu LTS 22.04, and Debian testing (bookworm). I am not really happy about this - I can try to get a Trilinos backport into Ubuntu LTS 20.04 but that doesn't solve the problem for Ubuntu LTS 18.04, and Debian stable (bullseye) 😞

masterleinad · 2023-01-04T10:40:46Z

We could lower the requirement to Trilinos 12.14.1 for the moment if 13.2 causes too many problems.

jppelteret · 2023-01-04T11:24:39Z

I also experience a problem when building a user project based off a custom deal.II docker instance (basically master with the commit frozen) that itself is FROM dealii/dependencies:focal. If continues with the bundled Kokkos (given that an acceptable Trilinos is no longer detected):

// detailed.log
#        DEAL_II_BUNDLED_INCLUDE_DIRS: /usr/src/dealii/bundled/kokkos-3.7.00/algorithms/src;/usr/src/dealii/bundled/kokkos-3.7.00/containers/src;/usr/src/dealii/bundled/kokkos-3.7.00/core/src;/usr/src/dealii/bundled/kokkos-3.7.00/simd/src;/usr/src/dealii/bundled/kokkos-3.7.00/tpls/desul/include
...
#      ( DEAL_II_WITH_TRILINOS = OFF )

then the build of deal.II is fine. The issue arises when finally building the user project:

/usr/local/include/deal.II/base/numbers.h:28:10: fatal error: Kokkos_Macros.hpp: No such file or directory
   28 | #include <Kokkos_Macros.hpp>
      |          ^~~~~~~~~~~~~~~~~~~

Are the DEAL_II_BUNDLED_INCLUDE_DIRS being propagated downstream?

blaisb · 2023-01-04T13:44:37Z

We obtain the same issue right now with Lethe. CMake configuration breaks on the most recent stable Github Actions.

This is problematic, since a lot of us are relying on the docker images for our continuous integration testing...

See the following link for the action instance:
https://github.com/lethe-cfd/lethe/actions/runs/3832127088/jobs/6535043565

masterleinad · 2023-01-04T14:02:42Z

Technically, we only promise to enable MPI support in the Docker image, see

dealii/contrib/docker/Dockerfile

Lines 9 to 15 in 55812cf

    
           && cmake -GNinja \ 
        
           -DCMAKE_PREFIX_PATH="/usr/lib/x86_64-linux-gnu/hdf5/openmpi;/usr/include/hdf5/openmpi" \ 
        
           -DDEAL_II_COMPILE_EXAMPLES=OFF \ 
        
           -DDEAL_II_COMPONENT_PYTHON_BINDINGS=ON \ 
        
           -DDEAL_II_WITH_MPI=ON \ 
        
           -DCMAKE_CXX_FLAGS="-std=c++17" \ 
        
           .. \

. We should be explicit about which packages should be enabled (and probably disable autodetection).

blaisb · 2023-01-04T14:05:07Z

Technically, we only promise to enable MPI support in the Docker image, see

dealii/contrib/docker/Dockerfile

Lines 9 to 15 in 55812cf

&& cmake -GNinja \

-DCMAKE_PREFIX_PATH="/usr/lib/x86_64-linux-gnu/hdf5/openmpi;/usr/include/hdf5/openmpi" \

-DDEAL_II_COMPILE_EXAMPLES=OFF \

-DDEAL_II_COMPONENT_PYTHON_BINDINGS=ON \

-DDEAL_II_WITH_MPI=ON \

-DCMAKE_CXX_FLAGS="-std=c++17" \

.. \

. We should be explicit about which packages should be enabled (and probably disable autodetection).

Please keep the docker image will as many packages as possible... Many of us rely on them...

tamiko · 2023-01-04T14:13:14Z

@jppelteret This is fixed in #14634

tamiko · 2023-01-04T14:18:38Z

All, Let us

make the docker instances more explicit
lower the requirement to ~~Trilinos 12.12.1 Ubuntu LTS 18.04~~ Trilinos 12.14.1 Ubuntu 20.04 LTS
work around the compilation errors with older Trilinos.

jppelteret · 2023-01-04T14:19:26Z

Thanks for the reply, @tamiko. I believe that I'm using the merge commit from that exact branch. So maybe its still not quite right? I can open another issue for this, if you'd prefer to discuss it further somewhere else.

/usr/src/dealii/build_logs$ cat revision.log 
###
#
#  Git information:
#        Branch:    master
#        Revision:  e06d56813949c2c93d5d40d48e485be72041ce94
#        Timestamp: 2023-01-03 16:04:38-05:00

peterrum · 2023-01-04T14:24:08Z

make the docker instances more explicit

I agree with that 👍 But we should keep in mind that the Docker images used to include all deal.II features (@luca-heltai Was always very quick with enabling features once we added a new optional library). I think we should continue like this but without auto detection as suggested by @masterleinad.

masterleinad · 2023-01-04T14:25:00Z

I think Trilinos 12.12.1 is too old for the Kokkos features we are using (but 12.14.1 would be fine).

blaisb · 2023-01-04T14:57:51Z

make the docker instances more explicit

I agree with that 👍 But we should keep in mind that the Docker images used to include all deal.II features (@luca-heltai Was always very quick with enabling features once we added a new optional library). I think we should continue like this but without auto detection as suggested by @masterleinad.

I second what @peterrum says. I think @luca-heltai did an amazing job in making sure we have a feature-complete deal.II docker image and this docker image is now used extensively by so many people and so many organizations. Let's please make sure we can keep it as feature complete as possible :).

tamiko · 2023-01-04T15:07:46Z

@masterleinad Roger. Let us ditch support for Ubuntu 18.04 LTS and go straight for Ubuntu 20.04 LTS which has Trilinos 12.14.1

tamiko · 2023-01-04T15:15:58Z

@jppelteret Let me investigate. Would you mind checking whether the bundled kokkos headers actually get installed?

jppelteret · 2023-01-04T17:56:29Z

Let me investigate. Would you mind checking whether the bundled kokkos headers actually get installed?

Thanks @tamiko. The installation should be in /usr/local as prescribed by

cmake                                                                                   \
    -GNinja                                                                                    \
    -DCMAKE_PREFIX_PATH="/usr/lib/x86_64-linux-gnu/hdf5/openmpi;/usr/include/hdf5/openmpi"     \
    -DDEAL_II_COMPILE_EXAMPLES=OFF                                                             \
    -DDEAL_II_COMPONENT_PYTHON_BINDINGS=ON                                                     \
    -DDEAL_II_WITH_MPI=ON                                                                      \
    -DCMAKE_CXX_FLAGS="-std=c++14 -march=native"                                               \
    -DCMAKE_INSTALL_PREFIX=/usr/local                                                          \
    ..

but I don't see any bundled components there

dealii@b040414f5186:~/build$ ls /usr/local/lib/
cmake  libdeal_II.g.so  libdeal_II.g.so.9.5.0-pre  libdeal_II.so  libdeal_II.so.9.5.0-pre  libsymengine.so  libsymengine.so.0.8  libsymengine.so.0.8.1  python3.8
dealii@b040414f5186:~/build$ ls /usr/local/include/
deal.II  symengine
dealii@b040414f5186:~/build$ ls /usr/local/include/deal.II/
algorithms  base            bundled  differentiation  dofs  gmsh  hp           lac          meshworker  non_matching  opencascade   particles  sundials
arborx      boost_adaptors  cgal     distributed      fe    grid  integrators  matrix_free  multigrid   numerics      optimization  physics    trilinos
dealii@b040414f5186:~/build$ ls /usr/local/include/deal.II/bundled/
dealii@b040414f5186:~/build$

tamiko · 2023-01-04T18:38:16Z

@jppelteret I see the problem: We do not install the header files correctly: As long as you're configuring withing the build directory DEAL_II_BUNDLED_INCLUDE_DIRS is set and everything is fine. As soon as we install, we forget to actually copy the kokkos header files to bundled. I'll open a pull request soon.

Fixes #14637

peterrum added High Priority ⚠️ Build system labels Jan 4, 2023

tamiko added Discussion and removed High Priority ⚠️ labels Jan 4, 2023

tamiko added High Priority ⚠️ Infrastructure and removed Build system labels Jan 4, 2023

tamiko changed the title ~~External project can not detect that deal.II was built with Trilinos~~ Infrastructure: Trilinos shipping with Ubuntu 18.04 / 20.04 and Debian bullseye (stable) no longer usable Jan 4, 2023

peterrum mentioned this issue Jan 4, 2023

Fix checkout version exadg/exadg#302

Merged

peterrum mentioned this issue Jan 4, 2023

Print missing external libraries kronbichler/adaflo#77

Open

masterleinad mentioned this issue Jan 4, 2023

Drop the requirement for Trilinos+Kokkos from 13.2 to 12.14.1 #14639

Merged

tamiko mentioned this issue Jan 4, 2023

CMake: fix kokkos header installation #14640

Merged

masterleinad closed this as completed in #14639 Jan 4, 2023

masterleinad added a commit that referenced this issue Jan 4, 2023

Merge pull request #14639 from masterleinad/require_trilinos_12_14_1

f24e3f1

Fixes #14637

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Infrastructure: Trilinos shipping with Ubuntu 18.04 / 20.04 and Debian bullseye (stable) no longer usable #14637

Infrastructure: Trilinos shipping with Ubuntu 18.04 / 20.04 and Debian bullseye (stable) no longer usable #14637

peterrum commented Jan 4, 2023

tamiko commented Jan 4, 2023 •

edited

peterrum commented Jan 4, 2023

tamiko commented Jan 4, 2023

masterleinad commented Jan 4, 2023

jppelteret commented Jan 4, 2023

blaisb commented Jan 4, 2023

masterleinad commented Jan 4, 2023

blaisb commented Jan 4, 2023

tamiko commented Jan 4, 2023

tamiko commented Jan 4, 2023 •

edited

jppelteret commented Jan 4, 2023

peterrum commented Jan 4, 2023

masterleinad commented Jan 4, 2023

blaisb commented Jan 4, 2023

tamiko commented Jan 4, 2023

tamiko commented Jan 4, 2023

jppelteret commented Jan 4, 2023

tamiko commented Jan 4, 2023

Infrastructure: Trilinos shipping with Ubuntu 18.04 / 20.04 and Debian bullseye (stable) no longer usable #14637

Infrastructure: Trilinos shipping with Ubuntu 18.04 / 20.04 and Debian bullseye (stable) no longer usable #14637

Comments

peterrum commented Jan 4, 2023

tamiko commented Jan 4, 2023 • edited

peterrum commented Jan 4, 2023

tamiko commented Jan 4, 2023

masterleinad commented Jan 4, 2023

jppelteret commented Jan 4, 2023

blaisb commented Jan 4, 2023

masterleinad commented Jan 4, 2023

blaisb commented Jan 4, 2023

tamiko commented Jan 4, 2023

tamiko commented Jan 4, 2023 • edited

jppelteret commented Jan 4, 2023

peterrum commented Jan 4, 2023

masterleinad commented Jan 4, 2023

blaisb commented Jan 4, 2023

tamiko commented Jan 4, 2023

tamiko commented Jan 4, 2023

jppelteret commented Jan 4, 2023

tamiko commented Jan 4, 2023

tamiko commented Jan 4, 2023 •

edited

tamiko commented Jan 4, 2023 •

edited