MARLIN

MARLIN is a library with JIT (Just-in-time) compilation support to optimize performance of small and medium matrix multiplication as well as convolutions. Currently, Intel AVX‑512 is supported. Compared to existing JIT GEMM libraries, MARLIN aims at converting data accesses to instruction accesses by embedding single precision floats within instructions as immediates. More details about the architecture of MARLIN is available in our paper published in CGO SRC 2021 (International Symposium on Code Generation and Optimization).

Getting Started: The following C++ code is an example of how the library can be used. A, B, C are three float matrices and B matrix is constant. Thus, matrix B can be converted using our JIT API and subsequently the sgemm function can be invoked.

#include <cstdlib>
#include <marlin>
#include <memory>

using namespace MARLIN;

int main(/*int argc, char* argv[]*/) {
  index_t m = 3;
  index_t n = 5;
  index_t k = 2;

  float *A = static_cast<float *>(std::malloc(m * k * sizeof(float)));
  float *B = static_cast<float *>(std::malloc(n * k * sizeof(float)));
  float *C = static_cast<float *>(std::malloc(m * n * sizeof(float)));
  
  // initialize input
  for (index_t i = 0; i < m; ++i) {
    for (index_t j = 0; j < k; ++j) {
      index_t idx = i * k + j;
      A[i * k + j] = idx + 1;
    }
  }
  for (index_t i = 0; i < k; ++i) {
    for (index_t j = 0; j < n; ++j) {
      index_t idx = i * n + j;
      B[i * n + j] = idx + 1;
    }
  }
  
  // initiate the jitter and generate code
  std::shared_ptr<Jitter<float>> jitter = std::make_shared<Jitter<float>>();
  jitter->generate_code(B, m, k, n);

  // perform sgemm
  sgemm('N', 'N', m, n, k, 1.0, A, k, B, n, 0, C, n, jitter);

  // free
  std::free(A);
  std::free(B);
  std::free(C);
}

Build Instructions

The unit tests depend on Google test framework and is required to build the library. The code for Google test can be updated through a git submodule update.

git submodule init
git submodule update

After the code is available, run the make command to build the library. The binary libmarlin.so will be created in the build directory.

make

To verify whether everything's working as expected, build and run the unit tests using the following commands.

make test
./build/test

External Benchmarks

Inorder to test external benchmarks, the following variables in the makefile should be configured. The easiest way would be to build the respective libraries and export as environment variables.

EIGEN_INC_DIR        = ${EIGEN_INC}
LIBXSMM_LIB_DIR	     = ${LIBXSMM_LIB}
LIBXSMM_INC_DIR	     = ${LIBXSMM_INC}
MKL_LIB_DIR          = ${MKL_LIB}
MKL_INC_DIR          = ${MKL_INC}
OPENBLAS_INC_DIR.    = ${OPENBLAS_INC}
OPENBLAS_CFG_INC_DIR = ${OPENBLAS_CFG_INC}
OPENBLAS_LIB_DIR     = ${OPENBLAS_LIB}
ONEDNN_CFG_INC_DIR   = ${ONEDNN_LIB}/../include
ONEDNN_LIB_DIR       = ${ONEDNN_LIB}
ONEDNN_INC_DIR       = ${ONEDNN_INC}

Having verified the external libraries, compile with external support by using the following command.

make ext ENABLE_EXT=1