This section of tutorial will be completed in Amazon Web Services (AWS) environment. Once you have access to system, please installing_buildtest and then proceed to the next section.
Once you are done, please run the following commands to ensure you are using the correct configuration file
export BUILDTEST_CONFIGFILE=$BUILDTEST_ROOT/buildtest/settings/aws.yml
buildtest config pathYou can verify the configuration is valid by runnning the following:
buildtest config validateIf you see no errors, then you are ready to proceed to the next section.
Let's start off with a simple hello world compilation in C using the GNU compiler. We have the following buildspec that will compile a code source code hello.c using the gcc compiler wrapper.
../../aws_tutorial/hello_world/hello.yml
The source code is the following
../../aws_tutorial/hello_world/hello.c
Let's try building this example and inspect the test results to see what happens.
buildtest build -b $BUILDTEST_ROOT/aws_tutorial/hello_world/hello.yml
cat aws_examples/hello_build.txt
cat aws_examples/hello_inspect.txt
Buildtest supports multiple test creation based on compiler selection using compilers property. This can be useful to test a single test with multiple compilers. In this next example, we will attempt to compile the Hello World test in previous example using 2 versions of GNU compiler. The name property is used to search for compilers in the configuration file. The compilers section covers in detail how to define compilers in buildtest configuration.
../../aws_tutorial/hello_world/multi_compiler_hello.yml
Let's try building this example, you will see there are now 2 tests created, one for each compiler.
buildtest build -b $BUILDTEST_ROOT/aws_tutorial/hello_world/multi_compiler_hello.yml
cat aws_examples/multi_compiler_hello_build.txt
Take note in the generated test output, the variables BUILDTEST_CC, BUILDTEST_CXX and BUILDTEST_FC are using the compiler wrappers for each compiler.
cat aws_examples/multi_compiler_hello_inspect.txt
We can see the compiler configuration using the command buildtest config compilers list --yaml which will print the output in YAML format. Shown below is the compiler declaration
cat aws_examples/compiler_list_yaml.txt
In this next example, we will compile a OpenMP code using GNU compiler and specify custom compiler flags and environment variable. We will work with a Hello World OpenMP code that uses OpenMP pragma to parallelize the code. The source code is the following
../../aws_tutorial/openmp_hello.c
In order to build OpenMP code, we need to use -fopenmp flag to enable OpenMP support. We will set OMP_NUM_THREADS environment variable to specify number of OpenMP theads which will differ for each compiler test. We introduce a keyword config that allows us to specify customize compiler flags and environment variables. The env property is used to set environment variable for each compiler test and cflags is used to set compiler flags inorder to compile the test. This value is stored in environment BUILDTEST_CFLAGS that will be used in run section for compiling the source code.
../../aws_tutorial/openmp_example_custom_compiler.yml
Let's try building this example and inspect the test results to see what happens.
buildtest build -b $BUILDTEST_ROOT/aws_tutorial/openmp_example_custom_compiler.yml
cat aws_examples/openmp_example_build.txt
We will see in the generated test the values OMP_NUM_THREADS and BUILDTEST_CFLAGS are set for each compiler test. The OMP_NUM_THREADS will impact the number of threads used to run code therefore we will see different output for each compiler test.
cat aws_examples/openmp_example_inspect.txt
This image comes with several MPI flavors such as OpenMPI, mvapich2. In the next example we will run a MPI Proc Name test on login node that will run with 8 processes. The source code is available in the $HOME/examples directory as part of the image.
../../aws_tutorial/mpiproc.yml
We can run this test using the following command
buildtest build -b $BUILDTEST_ROOT/aws_tutorial/mpiproc.yml
cat aws_examples/mpiproc_build.txt
We can see this test prints Hello World message from each process.
cat aws_examples/mpiproc_inspect.txt
The OSU Microbenchmark is a collection of MPI-based benchmarks developed at Ohio State University. This benchmark is used to measure performance of MPI libraries and is a popular benchmark suite for MPI. We will run the osu_bw test which measures bandwidth. The test requires we specify 2 processes.
In this next example, we will run 2 tests, the first is simply invoking the osu_bw test with 2 processes, and the second test will run the same test and use comparison_operators <comparison_operators> to compare the bandwidth result. The metrics property is used to capture performance metrics that can be used for comparison. We will use the assert_ge <assert_ge> status check that will do a greater than or equal comparison with reference value. The metrics will capture performance results for message length 16384 and perform a comparison with reference value of 10000. If the test result is greater than or equal to 10000, the test will pass otherwise it will fail.
../../aws_tutorial/osu_bandwidth_test.yml
Let's run the test and see the results.
buildtest build -b $BUILDTEST_ROOT/aws_tutorial/osu_bandwidth_test.yml
cat aws_examples/osu_bandwidth_test_build.txt
Take note in the output of the second test, we will see a list of performance metrics captured in table output. If metrics is not captured, the value will be undefined.
cat aws_examples/osu_bandwidth_test_inspect.txt
In this next exercise, we provide a tensorflow test in python that train a model and generate a predicition model. We have the following source code
../../aws_tutorial/tensorflow_model.py
The buildspec for this test is the following, where we run the test using python3 interpreter.
../../aws_tutorial/tensorflow.yml
Please run the test yourself and inspect the output. You will want to run the following commands:
buildtest build -b $BUILDTEST_ROOT/aws_tutorial/tensorflow.yml
buildtest inspect query -o run_tensorflow_modelBuildtest has support for running tests in containers <running_tests_containers> such as docker and singularity. In this next examples, we will show how to run a hello world test in docker and singularity container. Let's start off with docker, we introduce a new property container that allows us to specify the container settings. The platform and image are required properties that specifies the container platform (e.g., docker, singularity, podman) and the image name.
../tutorials/containers/hello_world.yml
Let's run the test and inspect the output, you will notice the test output will print a message from the container. This test will run in a docker container, if you look at the generated test content you will see a docker run invocation.
buildtest build -b $BUILDTEST_ROOT/tutorials/containers/hello_world.yml
cat aws_examples/docker_helloworld_build.txt
cat aws_examples/docker_helloworld_inspect.txt
In the next example, we will run the same test in a singularity container. To do this we will simply change the platform to singularity and specify the image name. Since singularity can pull images from different registries we will specify docker::// prefix to pull the image from docker hub.
../tutorials/containers/hello_world_singularity.yml
Buildtest will invoke singularity run and bind mount the stage directory into the container and execute test from the container. Take note that that singularity will volume mount test into /buildtest in the container and then run test from that directory.
buildtest build -b $BUILDTEST_ROOT/tutorials/containers/hello_world_singularity.yml
cat aws_examples/singularity_helloworld_build.txt
cat aws_examples/singularity_helloworld_inspect.txt
In this last example, we will run a test using a container executor <container_executor> by defining a custom executor based on a container image. In all of our previous examples, we were running tests using executor generic.local.bash which is a generic executor that runs tests on the local system using bash shell.
Let's take a look at the executor configuration by running the following command. The container keyword under executors section is used to define container executors. We can specify arbitrary name for the executor and specify the container image and platform.
cat aws_examples/container_executor_list.txt
We have the following buildspec that will run test using a custom executor generic.container.ubuntu.
../tutorials/containers/container_executor/ubuntu.yml
Let's run the test and inspect the output. You will notice the test is run in a ubuntu 20.04 container. In the output of df -h you will see the filesystem is from the container image with an entry /buildtest that is bind mounted from the host system.
buildtest build -b $BUILDTEST_ROOT/tutorials/containers/container_executor/ubuntu.yml
cat aws_examples/container_executor_build.txt
cat aws_examples/container_executor_inspect.txt