Performance and Stability Optimisation

Performance

CLASS tends to perform many computations like

pow(pba->H0, 2) / pow(a, 4)

that require fetching data from a struct and calling a function like pow() that are less optimisable by the compiler than a simple multiplication (as those function also provide additional checks and safeguards). To slightly speed up those computations, we defined new variables like

double a2 = a * a;
double a3 = a2 * a;
double a4 = a3 * a;
double H0_sq = pba->H0 * pba->H0;

and substituted those in the code. Besides this general substitutions, we have many function- or loop-specific substitutions to accelerate the computation, e.g. by writing for-loops in a way that should make it easy for the compiler to unroll the loop and vectorise the computation, using modern CPU instructions sets, or by expressing first gathering parameter values and caching powers of those values that are repeatetly used. We mention such implementations in the code and the corresponding Wiki sections. An extensive example of the former is the scalar field potential. An example of the latter is the scalar field coupling to dark matter.

Compile flags

Besides the CLASS default compile flag -O3 I also activated the following flags to speed up the binary:

# your optimization flag
OPTFLAG = -O3
OPTFLAG += -funroll-loops -ftree-vectorize -ftree-slp-vectorize -flto=auto -fPIC
OPTFLAG += -march=native -mtune=native
OPTFLAG += -fno-math-errno -fno-trapping-math # CLASS does not check for math errors, and these flags allow the compiler to vectorize more code, which can lead to significant speed-ups. See https://gcc.gnu.org/wiki/Vectorization#Floating_Point_Math_Errors for more details.

This should encourage the compiler to unroll loops and vectorise whatever is possible, to make use of the modern CPU instruction set. The latter is also encouraged by the -march=native -mtune=native flags, that optimise for the specific CPU the compute node is using (AMD EPYC CPUs).

Additionally, we use profile-guided optimisation. First, the code is compiled with the OPTFLAG and LDFLAG compile flags -fprofile-generate. Then, when the code runs under a typical workload, it creates files in a pre-defined folder that inform the compiler about the hot paths of an actual production run. This information is then used in a second compilation with the flags -fprofile-use to optimise the binary with the typical use case in mind.

Stability

When adding new quantities to the background table, space must be reserved for the columns with the additional data. This can easily go forgotten. To implement a little safeguard against this developer-error, we changed strcpy to strncpy in the output_print_data() function in output.c:

strncpy(thetitle, titles, _MAXTITLESTRINGLENGTH_ - 1);
  thetitle[_MAXTITLESTRINGLENGTH_ - 1] = '\0';
  pch = strtok(thetitle, _DELIMITER_);

We did the same for the class_store_columntitle() function from common.h:

#define class_store_columntitle(titlestring,                                               \
                                title,                                                     \
                                condition)                                                 \
  {                                                                                        \
    if (condition == _TRUE_)                                                               \
    {                                                                                      \
      strncat(titlestring, title, _MAXTITLESTRINGLENGTH_ - strlen(titlestring) - 1);       \
      strncat(titlestring, _DELIMITER_, _MAXTITLESTRINGLENGTH_ - strlen(titlestring) - 1); \
    }                                                                                      \
  }