Build into container that already has spack

[vsoch]

Hi! I'm looking at this Dockerfile https://github.com/LLNL/serac/blob/develop/scripts/docker/dockerfile_gcc-11 and I'd like to install serac into a container already with a spack install. I've never used "uberenv" but when I tried it, it looks like it was cloning spack fresh (which is not desired). Do you have any instructions or a sense for how to install the same spack deps to an already existing environment? Thanks!

[jamiebramwell]

Hi @vsoch! I unfortunately don't know much about uberenv other than how to use it. @white238 (or maybe @chapman39 ) can give better insight. I do know @chapman39 is currently updating uberenv to use Spack environments directly.

[vsoch]

This is what I'm trying now - although I suspect I'll run into issues with needing particular versions, etc. I'll post an update when this fails! 😆

[jamiebramwell]

This is the PR where @chapman39 is using the environment-ized uberenv: #882 . Hopefully this helps!

[vsoch]

Hiya! So I'm stepping back and trying the suggested build, here is my Dockerfile:

ARG tag=gcc-11_latest
FROM seracllnl/tpls:${tag}

# docker build -t serac .
# docker exec -it serac bash

RUN git clone --depth 1 https://github.com/LLNL/serac /home/serac/serac && \
    cd /home/serac/serac && \
    git submodule init && \
    git submodule update
WORKDIR /home/serac/serac
RUN python3 ./config-build.py  -hc host-configs/docker/gcc@11.1.0.cmake -bp ../build -ip ../install #&& \
    cd ../build && \
    make && \
    make test

(also note the instruction references python but it's python3 that we have in the container.

However, it looks like the MFEM spack directory that is present doesn't match what the python build script needs.

-- Using installed MFEM
-- Looking for MFEM using MFEM_DIR = /home/serac/serac_tpls/gcc-11.1.0/mfem-4.5.2.1-dcl2shc7i2wwfpuf7kq5as3k3jzc6uk2
CMake Error at cmake/SeracMacros.cmake:115 (message):
  Given MFEM_DIR does not exist:
  /home/serac/serac_tpls/gcc-11.1.0/mfem-4.5.2.1-dcl2shc7i2wwfpuf7kq5as3k3jzc6uk2
Call Stack (most recent call first):
  cmake/thirdparty/FindMFEM.cmake:20 (serac_assert_is_directory)
  cmake/thirdparty/SetupSeracThirdParty.cmake:184 (include)
  CMakeLists.txt:126 (include)


-- Configuring incomplete, errors occurred!
See also "/home/serac/build/CMakeFiles/CMakeOutput.log".
Error: CMake command failed with return code: 1
serac@a350f33fe930:~/serac$ ls ../serac_tpls/
bin  gcc-11.1.0  spack
serac@a350f33fe930:~/serac$ ls ../serac_tpls/gcc-11.1.0/
axom-0.7.0.4-lxxz2lzq5kmkwohmjtqpjooewid2bfbm/       ncurses-6.3-7ryidwb4sgpq2dkry3bu7oxyfl4qep4l/
blt-0.5.2-hh6st6lqzpj526jgiwa5zyrydrts7xew/          netcdf-c-4.7.4-ote37ajjhrkqu7ntxkqxryn2g5pc2apx/
camp-2022.03.2-ecxxjszodysrw4tiujtia5kma7whjszv/     parmetis-4.0.3-lxhbfjt6ldkee4qychk54v64kklppzlt/
conduit-0.8.4-pfqoghwltlucmgrplqq364e5o6yhq7jr/      raja-2022.03.0-raeo25xb7cumpvjlhqmrx6fncfuwef55/
hdf5-1.8.21-h5dpyqnqdcwv6rcpdz6ywjp53yeqst7a/        readline-8.1.2-ihaiti2r7b5asve4o7jvpgcjhwftwrs2/
hypre-2.26.0-jm3f5gzyuocir3vkfoh7mmxnl34lfkkx/       superlu-dist-6.1.1-jke7rcnlsep6bbtvimfqkl3sgmgrjpzr/
lua-5.4.4-ve6ws2zzw7qkrzupei6mnhtwdcxxz6jf/          umpire-2022.03.1-pieu6hp5juhqvvm6dp645xxlea4igxj3/
metis-5.1.0-rlnws7c5nyehxcnegqsto5rgtwhho7ru/        unzip-6.0-il72j5gpvxopfg7spcbn5dnt6yu4lexx/
mfem-4.5.3.1-aona2r6vgnjhv7xe5l2ymspxxwmkirwe/       zlib-1.2.13-zfwba3stno36xdnevigxdscpu5kozrmg/
serac@a350f33fe930:~/serac$ ls ../serac_tpls/gcc-11.1.0/m

It's just not using the right hash. Was there a specific branch I should have cloned? I'm following instructions here: https://serac.readthedocs.io/en/latest/sphinx/dev_guide/docker_env.html#docker-label. Thank you!

[white238]

Unfortunately it looks like our latest container tag is out of sync with our develop branch host-configs. We are currently shuffling a lot of TPL build processes.

This one is the date stamped tag seracllnl/tpls:gcc-11_02-16-23_00h-14m. Found here:

https://github.com/LLNL/serac/blob/develop/azure-pipelines.yml#L11

Sorry for the confusion!

[vsoch]

Totally ok! I'll give that one a try.

[vsoch]

okay success! I'll have an automated build with core serac and the provided examples deployed soon. I'd like to test it out in the Flux Operator so my next (and last question, don't worry!) is about workflows. Do you have any example projects that do something cool with serac with parameters we could potentially scale, or any kind of extra complexity or services?

[white238]

Awesome! @jamiebramwell is the best person to answer that question.

[jamiebramwell]

Passing the buck again! Our biggest users these days are @barreracruz and @kswartz92 . Do either of you have a pure serac problem that @vsoch can use for a scalability study or parameter sweep?

[barreracruz]

What about the re-entrant honeycomb lattice problem that I shared with you a few days ago, @jamiebramwell? The one you use to test the new nonlinear solvers. I can generate a mesh with more elements through thickness if refining it is not enough for a scalability study.

[jamiebramwell]

What about the re-entrant honeycomb problem that I shared with you a few days ago, @jamiebramwell? The one you use to test the new nonlinear solvers. I can generate a mesh with more elements through thickness if refining it is not enough for a scalability study.

Ooo! Good idea! I should be able to package that up for @vsoch . Thanks!

[vsoch]

Awesome! Thank you! 🙏

[white238]

I think this is either not relevant anymore or solved. If this is still needed to be worked on please reach out or open it back up.