Review

.gitignore

@@ -1 +1,8 @@
-.ipynb_checkpoints
+*.pyc
+*.o
+*.exe
+*.swp
+*.swo
+build*/
+install*/
+*.ipynb_checkpoints

README.md

@@ -1,12 +1,91 @@
-# RAJAPerf Tutorial
+# RAJA Performance Suite
 
-This repository contains RAjAPerf tutorials! You can
-look at the [tutorial-actions](https://github.com/rse-ops/tutorial-actions)
-to better understand the automation and assets here. Each tutorial is focused
-around learning or trying a piece of software, and we will have a tool
+The RAJA Performance Suite is designed to explore performance of loop-based
+computational kernels found in HPC applications. Specifically, it can be
+used to assess and monitor runtime performance of kernels implemented using
+[RAJA] C++ performance portability abstractions and compare those to variants
+implemented using common parallel programming models, such as OpenMP and CUDA,
+directly. Some important terminology used in the Suite includes:
+
+  * `Kernel` is a distinct loop-based computation that appears in the Suite in
+    multiple variants (or implementations), each of which performs the same
+    computation.
+  * `Variant` is an implementation or set of implementations of a kernel in the
+    Suite that share the same approach/abstraction and programming model,
+    such as baseline OpenMP, RAJA OpenMP, etc.
+  * `Tuning` is a particular implementation of a variant of a kernel in the
+    Suite, such as gpu block size 128, gpu block size 256, etc.
+  * `Group` is a collection of kernels in the Suite that are grouped together
+    because they originate from the same source, such as a specific benchmark
+    suite.
+
+Each kernel in the Suite appears in multiple RAJA and non-RAJA (i.e., baseline)
+variants using parallel programming models that RAJA supports. Some kernels have
+multiple tunings of a variant to explore some of the parametrization that the
+programming model supports. The kernels originate from various HPC benchmark
+suites and applications. For example, the "Stream" group contains kernels from
+the Babel Stream benchmark, the "Apps" group contains kernels extracted from
+real scientific computing applications, and so forth.
+
+The suite can be run as a single process or with multiple processes when
+configured with MPI support. Running with MPI in the same configuration used
+by an hpc app allows the suite to gather performance data that is more relevant
+for that hpc app than performance data gathered running single process. For
+example running sequentially with one MPI rank per core vs running sequentially
+with a single process yields different performance results on most multi-core
+CPUs.
+
+Find complete documentation on RAJAPerf here [RAJAPerf Documentation](https://rajaperf.readthedocs.io/en/develop/)
+
+# RAJA Performance Suite Tutorial
+=================================
+
+The RAJA Performance Suite Tutorial guides the user through the process of running
+RAJAPerf, including generating sweeps through various problem sizes, visualizing
+the timing hierarchy, and walking through some simple post-processing analysis,
+comparing the performance across two compilers, GCC and Clang, respectively.
+In addition the tutorial guides the user with adding a new kernel to the suite,
+and rerunning the analysis on the new kernel.
+
+The Jupyter Lab environment is ready-built containing Caliper enabled versions of RAJAPerf,
+with GCC and Clang variants. Also, we are able to allow interactive C++ interpreter
+of OpenMP constructs utilizing Xeus-Cling under the hood.
+
+# Technologies Referenced in the Tutorial
+=======================================
+* [RAJA](https://raja.readthedocs.io/)
+* [Caliper](https://software.llnl.gov/Caliper/)
+* [Hatchet](https://llnl-hatchet.readthedocs.io/en/latest/user_guide.html)
+* [OpenMP](https://www.openmp.org/)
+* [Xeus-Cling](https://xeus-cling.readthedocs.io/en/latest/)
+* [Cling](https://root.cern/cling/)
+
+
+This repository is supported by [tutorial-actions](https://github.com/rse-ops/tutorial-actions).
+Each tutorial is focused around learning or trying a piece of software, and we will have a tool
 that can easily deploy them.
 
-🚧️ **under development** 🚧️
+# Start Tutorial
+================
+
+For local docker users start the tutorial with
+docker run -p 8888:8888 image_id
+
+Navigate to the Web pages highlighted at the end of the Jupyter Lab console output.
+
+It will look something like
+
+    To access the server, open this file in a browser:
+        file:///home/jovyan/.local/share/jupyter/runtime/jpserver-1-open.html
+    Or copy and paste one of these URLs:
+        http://597d5b663944:8888/lab?token=998b2bdda9a76fef4ccc7d3c1d6776a7e9d77e4cbab37549
+        http://127.0.0.1:8888/lab?token=998b2bdda9a76fef4ccc7d3c1d6776a7e9d77e4cbab37549
+
+# Tutorial Navigation
+=====================
+
+Open the notebooks folder to find notebooks 00-xxxx through 07-xxxx .ipynb files, the first 00-intro-and-contents may be used as an index to jump to the other notebooks. The notebooks should be run in the order listed 00 - 07.
+
 
 License
 -------

tutorials/basic/Dockerfile

@@ -1,60 +1,76 @@
-ARG ubuntu_version=22.04
-FROM ubuntu:${ubuntu_version}
-
-# docker build -t ghcr.io/rse-ops/rajaperf-tutorials:ubuntu-basic .
-# Note that I never got this working with spack, in any way/shape/form.
-# The build also failed as automated in the CI, so I built locally and pushed.
-# Is this ideal or perfect? No, but at least we have the container build to
-# start with, and someone a lot more patient with spack can try again.
-
-RUN apt-get update
-RUN DEBIAN_FRONTEND="noninteractive" apt-get -y install \
-    python3-pip \
-    git \
+FROM jupyter/scipy-notebook
+
+USER $NB_USER:$NB_GID
+
+RUN mamba create -n cling python=3.10.9 && source activate cling && \
+  mamba install -c conda-forge -y  \
+    xeus-cling=0.15.0 \
+    libstdcxx-devel_linux-64=*=*19 \
+    mpich \
     cmake \
-    wget
-
-# Install caliper
-RUN git clone https://github.com/LLNL/Caliper.git /caliper && \
-    cd /caliper && \
-    mkdir -p build && cd build && \
-    cmake -DCMAKE_INSTALL_PREFIX=/usr/local .. && \
-    make && make install
-
-# Install adiak
-RUN wget https://github.com/LLNL/Adiak/releases/download/v0.2.1/adiak-0.2.1.tar.gz && \
-    tar -xzvf adiak-0.2.1.tar.gz && \
-    cd adiak-0.2.1 && \
-    mkdir build && cd build && \
-    cmake -DCMAKE_INSTALL_PREFIX=/usr/local .. && \
-    make && make install
-
-# Install camp
-RUN git clone --recursive https://github.com/LLNL/camp /camp && \
-    cd /camp && \
-    git submodule update && \
-    mkdir -p build && cd build && \
-    cmake -DCMAKE_INSTALL_PREFIX=/usr/local .. && \
-    make && make install
-
-# Install hatchet
-RUN git clone https://github.com/LLNL/hatchet /hatchet && \
-    cd /hatchet && \
-    pip install llnl-hatchet && \
-    pip install -r requirements.txt
-
-RUN git clone --recursive -b use_caliper https://github.com/jonesholger/RAJAPerf /code && ldconfig
-COPY notebooks/build_rajaperf.sh /code/scripts/ubuntu-builds/build_rajaperf.sh
-RUN cd /code && \
-    git submodule update --recursive && \
-    chmod +x ./scripts/ubuntu-builds/build_rajaperf.sh && \
-    ./scripts/ubuntu-builds/build_rajaperf.sh 11 && \
-    cd build_ubuntu-gcc-caliper-11 && \
-    # Slower without -j but won't crash or freeze a machine!
-    make && make install
-
-# TODO we will want to:
-# 1. Figure out a visualization strategy here
-# 2. Get it into a notebook / VSCode interface to run and see
-# 3. Possibly re-add openmpi install.
-WORKDIR /code/build_ubuntu-gcc-caliper-11
+    vim \
+    gfortran=10.4.0 \
+    gxx=10.4.0 \
+    binutils \
+    make \
+    ninja \
+    elfutils \
+    zlib \
+    libunwind \
+    patchelf \
+    lmod 
+# fixup OpenMP header and lib paths to satisfy header search and ld search and ldconfig search
+# todo run ldconfig after to pickup libs in /usr/local/lib
+USER root
+RUN mkdir -p /opt/conda/envs/cling/lib/clang/9.0.1/include && \
+    cp /opt/conda/lib/clang/15.0.7/include/omp.h /opt/conda/envs/cling/lib/clang/9.0.1/include/omp.h  && \
+    cp  /opt/conda/lib/libomp.so /usr/local/lib/libomp.so && \
+    cp  /opt/conda/lib/libomp.so /opt/conda/envs/cling/x86_64-conda-linux-gnu/sysroot/lib64/ && \
+    ldconfig -v 2> /dev/null
+
+USER $NB_USER:$NB_GID
+# Make RUN commands use the new environment:
+RUN echo "source activate cling" >> $HOME/.bashrc
+SHELL ["/bin/bash", "-c"]
+
+ENV PREFIX=/opt/conda/envs/cling/share/jupyter/kernels
+COPY --chown=$NB_USER:$NB_GID xcpp17-omp/ $PREFIX/xcpp17-omp/
+
+RUN jupyter kernelspec install $PREFIX/xcpp17 --sys-prefix && \
+  jupyter kernelspec install $PREFIX/xcpp17-omp --sys-prefix
+
+COPY --chown=$NB_USER:$NB_GID notebooks $HOME/notebooks
+COPY --chown=$NB_USER:$NB_GID scripts $HOME/scripts
+COPY --chown=$NB_USER:$NB_GID skeletal $HOME/skeletal
+
+RUN chmod +x scripts/set-up-spack.sh && ./scripts/set-up-spack.sh
+#/opt/conda/envs/cling/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ENV PATH=${HOME}/spack/bin:/opt/conda/envs/cling/bin:/opt/conda/condabin:${PATH}
+RUN echo "export PATH=${PATH}" >> $HOME/.bashrc && \
+    echo ". ${HOME}/spack/share/spack/setup-env.sh" >> $HOME/.bashrc && \
+    echo "spack env activate  --dir ${HOME}/spack_env" >> $HOME/.bashrc
+COPY --chown=$NB_USER:$NB_GID spack.yaml $HOME/spack_env/
+#install raja gcc and clang variants after conda init shell commands in an active bash session
+# then install raja-perf using spack dev-build
+RUN chmod +x scripts/add_spack_packages.sh && ./scripts/add_spack_packages.sh && \
+    chmod +x scripts/add_code_packages.sh && ./scripts/add_code_packages.sh
+
+
+# Just some general notes about this Dockerfile before we call the entrypoint
+# g++ (conda-forge gcc 10.4.0-19) 10.4.0
+# 
+#(cling) jovyan@d37702b56b3d:~$ /opt/conda/envs/cling/bin/clang --version
+#clang version 9.0.1 (https://github.com/conda-forge/clangdev-feedstock 2ea3b72da24769de0dfc6dac99251a5d7a46144d)
+#
+# to run container use docker run -p 8888:8888
+
+# ENV
+# NB_USER=jovyan
+# NB_UID=1000
+# NB_GID=100
+# PWD=/home/jovyan
+# CONDA_PREFIX=/opt/conda
+# HOME=/home/jovyan
+
+
+ENTRYPOINT ["/bin/bash","-c","$HOME/scripts/entry_point_xeus.sh"]

tutorials/basic/Dockerfile-xeus-cling

@@ -0,0 +1,76 @@
+FROM jupyter/scipy-notebook
+
+USER $NB_USER:$NB_GID
+
+RUN mamba create -n cling python=3.10.9 && source activate cling && \
+  mamba install -c conda-forge -y  \
+    xeus-cling=0.15.0 \
+    libstdcxx-devel_linux-64=*=*19 \
+    mpich \
+    cmake \
+    vim \
+    gfortran=10.4.0 \
+    gxx=10.4.0 \
+    binutils \
+    make \
+    ninja \
+    elfutils \
+    zlib \
+    libunwind \
+    patchelf \
+    lmod 
+# fixup OpenMP header and lib paths to satisfy header search and ld search and ldconfig search
+# todo run ldconfig after to pickup libs in /usr/local/lib
+USER root
+RUN mkdir -p /opt/conda/envs/cling/lib/clang/9.0.1/include && \
+    cp /opt/conda/lib/clang/15.0.7/include/omp.h /opt/conda/envs/cling/lib/clang/9.0.1/include/omp.h  && \
+    cp  /opt/conda/lib/libomp.so /usr/local/lib/libomp.so && \
+    cp  /opt/conda/lib/libomp.so /opt/conda/envs/cling/x86_64-conda-linux-gnu/sysroot/lib64/ && \
+    ldconfig -v 2> /dev/null
+
+USER $NB_USER:$NB_GID
+# Make RUN commands use the new environment:
+RUN echo "source activate cling" >> $HOME/.bashrc
+SHELL ["/bin/bash", "-c"]
+
+ENV PREFIX=/opt/conda/envs/cling/share/jupyter/kernels
+COPY --chown=$NB_USER:$NB_GID xcpp17-omp/ $PREFIX/xcpp17-omp/
+
+RUN jupyter kernelspec install $PREFIX/xcpp17 --sys-prefix && \
+  jupyter kernelspec install $PREFIX/xcpp17-omp --sys-prefix
+
+COPY --chown=$NB_USER:$NB_GID notebooks $HOME/notebooks
+COPY --chown=$NB_USER:$NB_GID scripts $HOME/scripts
+COPY --chown=$NB_USER:$NB_GID skeletal $HOME/skeletal
+
+RUN chmod +x scripts/set-up-spack.sh && ./scripts/set-up-spack.sh
+#/opt/conda/envs/cling/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ENV PATH=${HOME}/spack/bin:/opt/conda/envs/cling/bin:/opt/conda/condabin:${PATH}
+RUN echo "export PATH=${PATH}" >> $HOME/.bashrc && \
+    echo ". ${HOME}/spack/share/spack/setup-env.sh" >> $HOME/.bashrc && \
+    echo "spack env activate  --dir ${HOME}/spack_env" >> $HOME/.bashrc
+COPY --chown=$NB_USER:$NB_GID spack.yaml $HOME/spack_env/
+#install raja gcc and clang variants after conda init shell commands in an active bash session
+# then install raja-perf using spack dev-build
+RUN chmod +x scripts/add_spack_packages.sh && ./scripts/add_spack_packages.sh && \
+    chmod +x scripts/add_code_packages.sh && ./scripts/add_code_packages.sh
+
+
+# Just some general notes about this Dockerfile before we call the entrypoint
+# g++ (conda-forge gcc 10.4.0-19) 10.4.0
+# 
+#(cling) jovyan@d37702b56b3d:~$ /opt/conda/envs/cling/bin/clang --version
+#clang version 9.0.1 (https://github.com/conda-forge/clangdev-feedstock 2ea3b72da24769de0dfc6dac99251a5d7a46144d)
+#
+# to run container use docker run -p 8888:8888
+
+# ENV
+# NB_USER=jovyan
+# NB_UID=1000
+# NB_GID=100
+# PWD=/home/jovyan
+# CONDA_PREFIX=/opt/conda
+# HOME=/home/jovyan
+
+
+ENTRYPOINT ["/bin/bash","-c","$HOME/scripts/entry_point_xeus.sh"]

tutorials/basic/Dockerfile-xeus-cling-advance

@@ -0,0 +1,76 @@
+FROM jupyter/scipy-notebook
+
+USER $NB_USER:$NB_GID
+
+RUN mamba create -n cling python=3.10.9 && source activate cling && \
+  mamba install -c conda-forge -y  \
+    xeus-cling=0.15.0 \
+    libstdcxx-devel_linux-64=*=*19 \
+    mpich \
+    cmake \
+    vim \
+    gfortran=10.4.0 \
+    gxx=10.4.0 \
+    binutils \
+    make \
+    ninja \
+    elfutils \
+    zlib \
+    libunwind \
+    patchelf \
+    lmod 
+# fixup OpenMP header and lib paths to satisfy header search and ld search and ldconfig search
+# todo run ldconfig after to pickup libs in /usr/local/lib
+USER root
+RUN mkdir -p /opt/conda/envs/cling/lib/clang/9.0.1/include && \
+    cp /opt/conda/lib/clang/15.0.7/include/omp.h /opt/conda/envs/cling/lib/clang/9.0.1/include/omp.h  && \
+    cp  /opt/conda/lib/libomp.so /usr/local/lib/libomp.so && \
+    cp  /opt/conda/lib/libomp.so /opt/conda/envs/cling/x86_64-conda-linux-gnu/sysroot/lib64/ && \
+    ldconfig -v 2> /dev/null
+
+USER $NB_USER:$NB_GID
+# Make RUN commands use the new environment:
+RUN echo "source activate cling" >> $HOME/.bashrc
+SHELL ["/bin/bash", "-c"]
+
+ENV PREFIX=/opt/conda/envs/cling/share/jupyter/kernels
+COPY --chown=$NB_USER:$NB_GID xcpp17-omp/ $PREFIX/xcpp17-omp/
+
+RUN jupyter kernelspec install $PREFIX/xcpp17 --sys-prefix && \
+  jupyter kernelspec install $PREFIX/xcpp17-omp --sys-prefix
+
+COPY --chown=$NB_USER:$NB_GID notebooks $HOME/notebooks
+COPY --chown=$NB_USER:$NB_GID scripts $HOME/scripts
+COPY --chown=$NB_USER:$NB_GID skeletal $HOME/skeletal
+
+RUN chmod +x scripts/set-up-spack.sh && ./scripts/set-up-spack.sh
+#/opt/conda/envs/cling/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ENV PATH=${HOME}/spack/bin:/opt/conda/envs/cling/bin:/opt/conda/condabin:${PATH}
+RUN echo "export PATH=${PATH}" >> $HOME/.bashrc && \
+    echo ". ${HOME}/spack/share/spack/setup-env.sh" >> $HOME/.bashrc && \
+    echo "spack env activate  --dir ${HOME}/spack_env" >> $HOME/.bashrc
+COPY --chown=$NB_USER:$NB_GID spack.yaml $HOME/spack_env/
+#install raja gcc and clang variants after conda init shell commands in an active bash session
+# then install raja-perf using spack dev-build
+RUN chmod +x scripts/add_spack_packages.sh && ./scripts/add_spack_packages.sh && \
+    chmod +x scripts/add_code_packages_advance.sh && ./scripts/add_code_packages_advance.sh
+
+
+# Just some general notes about this Dockerfile before we call the entrypoint
+# g++ (conda-forge gcc 10.4.0-19) 10.4.0
+# 
+#(cling) jovyan@d37702b56b3d:~$ /opt/conda/envs/cling/bin/clang --version
+#clang version 9.0.1 (https://github.com/conda-forge/clangdev-feedstock 2ea3b72da24769de0dfc6dac99251a5d7a46144d)
+#
+# to run container use docker run -p 8888:8888
+
+# ENV
+# NB_USER=jovyan
+# NB_UID=1000
+# NB_GID=100
+# PWD=/home/jovyan
+# CONDA_PREFIX=/opt/conda
+# HOME=/home/jovyan
+
+
+ENTRYPOINT ["/bin/bash","-c","$HOME/scripts/entry_point_xeus.sh"]