[ci] Test run Spack and EasyBuild tutorial examples in a container

.github/workflows/main.yml

@@ -47,6 +47,17 @@ jobs:
       run: |
         docker run reframe:${{ matrix.modules-version }}
 
+  tutorialtest:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v2
+    - name: Build Image for Tutorial Tests
+      run: |
+        docker build -f ci-scripts/dockerfiles/tutorials.dockerfile -t reframe:tutorials .
+    - name: Run Tutorial Tests
+      run: |
+        docker run reframe:tutorials
+
   unusedimports:
     runs-on: ubuntu-latest
     steps:

ci-scripts/dockerfiles/tutorials.dockerfile

@@ -0,0 +1,40 @@
+#
+# Execute this from the top-level ReFrame source directory
+#
+
+
+FROM reframehpc/rfm-ci-base:lmod
+
+ENV _SPACK_VER=0.16
+ENV _EB_VER=4.4.1
+
+# Required utilities
+RUN apt-get -y update && \
+    apt-get -y install curl
+
+# ReFrame user
+RUN useradd -ms /bin/bash rfmuser
+
+USER rfmuser
+
+# Install Spack
+RUN git clone https://github.com/spack/spack ~/spack && \
+    cd ~/spack && \
+    git checkout releases/v${_SPACK_VER}
+
+RUN pip3 install easybuild==${_EB_VER}
+
+ENV PATH="/home/rfmuser/.local/bin:${PATH}"
+
+# Install ReFrame from the current directory
+COPY --chown=rfmuser . /home/rfmuser/reframe/
+
+WORKDIR /home/rfmuser/reframe
+
+RUN ./bootstrap.sh
+
+RUN echo '. /usr/local/lmod/lmod/init/profile && . /home/rfmuser/spack/share/spack/setup-env.sh' > /home/rfmuser/setup.sh
+
+ENV BASH_ENV /home/rfmuser/setup.sh
+
+CMD ["/bin/bash", "-c", "./bin/reframe -r -C tutorials/config/settings.py -R -c tutorials/build_systems --system tutorials-docker"]

docs/tutorial_build_automation.rst

@@ -109,53 +109,36 @@ Here is the test's code:
 
 
 When :attr:`~reframe.core.pipeline.RegressionTest.build_system` is set to ``'Spack'``, ReFrame will leverage Spack environments in order to build the test code.
-If the environment is not specified by the user, ReFrame will automatically create one inside the stage directory.
-ReFrame treats Spack environments specified by the user as *test resources* so it expects to find them under the test's :attr:`~reframe.core.pipeline.RegressionTest.sourcesdir`, which defaults to ``'src'``.
-Here is the directory structure for the test in this particular example that we show here:
-
-.. code:: console
-
-   tutorials/build_systems/spack/
-   ├── spack_test.py
-   └── src
-       └── myenv
-           └── spack.yaml
-
-
-We could have placed ``spack.yaml`` directly under the ``src/`` directory, in which case we would need to specify ``'.'`` as an environment.
-For reference, here are the contents of ``spack.yaml``:
-
-.. literalinclude:: ../tutorials/build_systems/spack/src/myenv/spack.yaml
-
+By default, ReFrame will create a new Spack environment in the test's stage directory and add the requested :attr:`~reframe.core.buildsystems.Spack.specs` to it.
+Users may also specify an existing Spack environment by setting the :attr:`~reframe.core.buildsystems.Spack.environment` attribute.
+In this case, ReFrame treats the environment as a *test resource* so it expects to find it under the test's :attr:`~reframe.core.pipeline.RegressionTest.sourcesdir`, which defaults to ``'src'``.
 
 As with every other test, ReFrame will copy the test's resources to its stage directory before building it.
-ReFrame will then activate the environment and install the associated specs as in this case.
-Optionally, we can add more specs to the environment by setting the :attr:`~reframe.core.buildsystems.Spack.specs` attribute of the build system.
-Here is what ReFrame generates as a build script in this example:
+ReFrame will then activate the generated environment (either the one provided by the user or the one generated by ReFrame), add the given specs using the ``spack add`` command  and, finally, install the packages in the environment.
+Here is what ReFrame generates as a build script for this example:
 
 .. code:: bash
 
    . "$(spack location --spack-root)/share/spack/setup-env.sh"
-   spack env activate -V -d myenv
+   spack env create -d rfm_spack_env
+   spack env activate -V -d rfm_spack_env
+   spack config add "config:install_tree:root:opt/spack"
+   spack add bzip2@1.0.6
    spack install
 
-Any additional specs specified inside the ReFrame test will be added using the ``spack add`` command.
 As you might have noticed ReFrame expects that Spack is already installed on the system.
 The packages specified in the environment and the tests will be installed in the test's stage directory, where the environment is copied before building.
 Here is the stage directory structure:
 
 .. code:: console
 
    stage/generic/default/builtin/BZip2SpackCheck/
-   ├── myenv
+   ├── rfm_spack_env
    │   ├── spack
-   │   │   ├── opt
-   │   │   │   └── spack
-   │   │   │       ├── bin
-   │   │   │       └── darwin-catalina-skylake
-   │   │   └── share
-   │   │       └── spack
-   │   │           └── modules
+   │   │   └── opt
+   │   │       └── spack
+   │   │           ├── bin
+   │   │           └── darwin-catalina-skylake
    │   ├── spack.lock
    │   └── spack.yaml
    ├── rfm_BZip2SpackCheck_build.err
@@ -172,7 +155,9 @@ Finally, here is the generated run script that ReFrame uses to run the test, onc
 
    #!/bin/bash
    . "$(spack location --spack-root)/share/spack/setup-env.sh"
-   spack env activate -V -d myenv
+   spack env create -d rfm_spack_env
+   spack env activate -V -d rfm_spack_env
+   spack load bzip2@1.0.6
    bzip2 --help
 
 From this point on, sanity and performance checking are exactly identical to any other ReFrame test.

tutorials/build_systems/spack/spack_test.py

@@ -18,7 +18,7 @@ class BZip2SpackCheck(rfm.RegressionTest):
 
     @run_before('compile')
     def setup_build_system(self):
-        self.build_system.environment = 'myenv'
+        self.build_system.specs = ['bzip2@1.0.6']
 
     @sanity_function
     def assert_version(self):

tutorials/config/settings.py

@@ -23,6 +23,20 @@
                 }
             ]
         },
+        {
+            'name': 'tutorials-docker',
+            'descr': 'Container for running the build system tutorials',
+            'hostnames': ['docker'],
+            'modules_system': 'lmod',
+            'partitions': [
+                {
+                    'name': 'default',
+                    'scheduler': 'local',
+                    'launcher': 'local',
+                    'environs': ['builtin'],
+                }
+            ]
+        },
         {
             'name': 'daint',
             'descr': 'Piz Daint Supercomputer',