COMPSs Framework

Component Superscalar framework and programming model for HPC.

Website • Documentation • Releases • Slack

COMP Superscalar (COMPSs) is a programming model which aims to ease the development of applications for distributed infrastructures, such as Clusters, Grids and Clouds. COMP Superscalar also features a runtime system that exploits the inherent parallelism of applications at execution time.

Documentation

COMPSs documentation can be found at the COMPSs Webpage or at the doc/ folder.

• COMPSs_Installation_Manual.pdf
• COMPSs_User_Manual_App_Development.pdf
• COMPSs_User_Manual_App_Execution.pdf
• COMPSs_Supercomputers_Manual.pdf
• Tracing_Manual.pdf
• COMPSs_Developer_Manual.pdf

Packages

The COMP Superscalar Framework packages are available at the COMPSs Webpage or can be found on the builders/packages/ directory.

Sources Structure

• builders: Packages, scripts for local installations, scripts for supercomputers installation and package building scripts
• compss : COMPSs Runtime
• dependencies : COMPSs embeded dependencies
• doc : COMPSs documentation
• files : Dependency files (i.e. paraver configurations)
• tests : COMPSs integration tests
• utils : Misc utils (i.e. OVA scripts, Docker generation, Storage implementations)

Sample Applications

You can find extended information about COMPSs Sample applications at the Sample_Applications manual available at the COMPSs Webpage or at the doc/Sample_applications.pdf.

Building COMPSs

Follow the next steps to build COMPSs in your current machine.

1. Install dependencies

Install the listed dependencies for each component you wish to install. For a complete build please install all the dependencies.

• COMPSs Runtime dependencies
  • openjdk-8-jre
  • openssh-server
  • uuid-runtime
  • graphviz
  • xdg-utils
• Build dependencies
  • wget
  • openjdk-8-jdk
  • maven
  • curl
  • jq
• Bindings-common dependencies
  • build-essential
  • autoconf
  • automake
  • autotools-dev
  • libtool
• C-binding dependencies
  • libboost-all-dev
  • libxml2-dev
  • csh
• Python-binding dependencies
  • python-dev | python3-dev
  • python-pip | python3-pip
  • python-setuptools | python3-setuptools
  • libpython2.7
  • wheel
  • numpy
  • dill
  • guppy
• Extrae dependencies
  • libxml2
  • gfortran
  • libpapi-dev
  • papi-tools
• AutoParallel dependencies
  • libgmp3-dev
  • flex
  • bison
  • libbison-dev
  • texinfo
  • libffi-dev
  • astor
  • sympy
  • enum34
  • islpy
• Streaming dependencies
  • Gradle
• Testing dependencies
  • openmpi-bin
  • openmpi-doc
  • libopenmpi-dev
  • bc
  • decorator
  • mpi4py
  • redis-py-cluster
  • enum34
  • tabulate

2. Get GIT submodules

Before installing COMPSs you need to download the git submodules that contain its dependencies. To do that execute the following two commands at the root of the repository.

./submodules_get.sh
./submodules_patch.sh

3. Build COMPSs

Note: Remember to install the COMPSs dependencies and to get the GIT submodules before trying to build COMPSs from sources.

• Building COMPSs for all users

cd builders/
INSTALL_DIR=/opt/COMPSs/
sudo -E ./buildlocal [options] ${INSTALL_DIR}

• Building COMPSs for current user

cd builders/
INSTALL_DIR=$HOME/opt/COMPSs/
./buildlocal [options] ${INSTALL_DIR}

Many COMPSs modules can be activated/deactivated during the build using different options in the buildlocal command. You may check the available options by running the following command:

cd builders
./buildlocal -h

Running docker tests

1. Install Docker and docker-py

Follow these instructions

• Docker for Mac. Or, if you prefer to use Homebrew.
• Docker for Ubuntu.
• Docker for Arch Linux.

Add user to docker group to run docker as non-root user.

• Instructions.

2. Build the docker image

Run the following command at the root of the project to build the image that will used for testing. The command create an image named compss and install the current branch into the image.

docker build -t compss .

3. Run the tests

To run the tests inside the docker image use the script found in ./tests/scripts/docker_main. This command is a wrapper for the ./main test command so it has de the syntax and options. For example, you can run the first test without retrials as follows:

./docker_main -R local_1 local.cfg

The docker main command creates a new docker container each time you run it (replacing the last one used). It copies the current framework inside it and runs its tests. Note: the testing scripts assumes you have named the testing image compss.

Please be aware that:

• Code changes affecting the tests sources, config files (e.g. local.cfg, and scripts (like ./local) will be visible inside the newly created container.
• Code changes affecting the installation will not be visible in the installation because framework is not reinstalled. To do that rebuild the docker image as explained in step 3.
• If you run the command once, the container will be available for manual inspection (such as logs). You can log into in issuing docker exec --user jenkins -it compss_test bash and use the CLI as usual.
• To delete the created image issue docker rmi compss
• To delete the compss_test container use docker rm -f compss_test.

Contact

✉️ COMPSs Support support-compss@bsc.es ✉️

Workflows and Distributed Computing Group (WDC)

Department of Computer Science (CS)

Barcelona Supercomputing Center (BSC)