Home

Welcome to the ContainerHackathon wiki!

This page contains material for the Container Hackathon for Modellers of the ESiWACE2 project, held from Dec. 3 to Dec. 5, 2019, in Lugano, Switzerland. Until today, we have received applications from 16 people representing 7 different codes.

Agenda

The agenda has been planned in order to help the teams port their native codes into containers. For this reason, a dedicated mentor will be working together with each of the teams. The expected milestones are as follows:

• After the first day, a sane build environment is working and a Hello World type of container exists.
• By the end of the second day, the code should be containerised and the test case selected by the team should be passing.
• The third day is reserved for launching the containers at scale on Piz Daint and preparing the final report.

You will be assigned a username to access the Cray XC50 system Piz Daint and run the test of the container created during the first day: you will need to accept the ETH Zurich Acceptable Use Policy for Telematics Resources, a summary of which will be available for display with the form to be signed.

Reservation

A total of 160 Cray XC50 nodes will be reserved on Piz Daint during the second and third day of the Hackathon. Please have a quick look at the CSCS User Portal for basic information on how to run a Slurm batch job and how to use Sarus on Piz Daint. You can run your jobs within the reservations using the following Slurm option in your batch script:

#SBATCH --reservation=<reservation_name>

The names of the reservations will be the following:

1. esiwace_1 on Wednesday December 4th, from 09:00 to 19:00
2. esiwace_2 on Thursday December 5th, from 09:00 to 19:00

NOTICE When launching your containers on Piz Daint, configure the GPU environment and unload the XALT module, e.g.:

$ module load daint-gpu
$ module unload xalt/2.7.10

Please note that the usernames will expire Friday December 6th at midnight, therefore you must retrieve any important data before the deadline because it will be deleted.

For teams that are lagging behind, the third day will give them the opportunity of catching up with the second milestone and finish the hackathon with a working version of their code in a container.

Materials

Each team will deliver a short report about their experiences while containerising the codes. These reports will be used as the basis for the ESiWACE2 deliverable of this task. Additionally, this Git repository can be used to host non-licensed material that can be posted for everyone's benefit (e.g., Dockerfiles, step-by-step instructions, reports, etc.)

Reports

The teams are expected to do deliver a final report after the hackathon.

Please note that a report has to be delivered by each team before the end of the hackathon.

Here is a breakdown of the report with more details:

• A single paragraph abstract with the application description.
• A short overview of the application (without mathematics!)
• Break down of the containerisation approach:
  • Explain the starting point (e.g., CPU-code compiled with XXX compiler, GPU access using CUDA, etc)
  • Shortly describe the test case that verifies the code is functioning correctly
  • Steps you made porting the code to a Docker container (e.g., which components ported)
  • OPTIONAL Performance profile of the code running natively and from a container, e.g., speedup graph.
• A final conclusion including:
  • Short feedback about your experiences
  • Obstacles you encountered, and how you solved them
  • Lessons that you would like to share with other teams, e.g., suggestions on how to improve the process, better documentation, etc.

Last but not least, any general comments about this Container Hackathon for Modellers will be really useful for the organisers.

Slack Workspace

A Slack Worskspace has been created for the event, you can sign in here.

The teams

Software	Team Members	Mentor
EC-Earth	Pablo Echevaria Julian Berlin Uwe Fladrich	Alberto Madonna
Icon	Ralf Mueller Remo Dietlicher Andre Walser Sergey Kosukhin	Rafael Sarmiento
LFRIC	Iva Kavcic Simon Wilson	Luca Marsella
Nemo	Italo Epicoco Giorgio Micaletto	Jg Piccinali
NorESM/CESM	Anne Claire Fouilloux	Kean Mariotti
OpenIFS	Marcus Koehler	Theofilos Manitaras
Workflow for Frame Prediction by Parallel Deep Learning	Amirpasha Mozaffari Bing Gong Jan Vogelsang	Tomas Aliaga