What are these files

These are the code files for WarwickRSEs Parallelism Primer workshop. Here you will find the following:

• GNU Parallel
  • Simple scripts showing the basic use of GNU Parallel and how you can match jobs to arguments
• Pthread
  • A few simple example programs using threads
• OpenMP
  • Code from slides (whole day workshop version)
• MPI
  • Code from slides (whole day workshop version)
• Examples
  • Some simple programs implemented using several parallel approaches for comparing between them
• Suggestions pdfs
  • Guided GNU parallel examples for ensemble jobs
  • Guided questions about which parallelism scheme is best for which problem and some useful things to know if you're running parallel codes

Approaches to parallelism

For the exercises in this workshop, there are a few things you might want to keep in mind about the parallelism strategies available. We tend use the following terms rather interchangeably:

• GNU parallel versus ensemble
  • GNU parallel is a single workload manager which is able to run multiple copies of a program with variant inputs
  • This is an implementation of ensemble computing
• OpenMP/threads versus shared memory
  • OpenMP is a specific library for doing shared memory programming. While it can do most paradigms, it is most straightforward for parallelising loops
  • Threads are the basic technology behind OpenMP, allowing multiple "instruction streams" to work on the same data
  • PThreads is a library to use threading in C, C++11 adds its own threading library
  • Shared memory covers all of these - threads by default tend to share memory unless you specifically keep it local
• MPI versus distributed memory
  • Distributed memory refers to any programming technique where processes are separate and have to communicate
  • MPI is a library for providing this communication between separate processes, which is what lets you write useful programs

Workshops

This workshop has been run as two variations - a full day covering ideas, ensemble parallelism, threading and then OpenMP and MPI as the two most common parallel technologies, and a half day version coving mostly the ideas with some ensemble parallel examples, with a brief mention of how coupled paralellism is done.

For more details and exercises to work through on OpenMP and MPI if one of these is what you want or need, try WarwickRSEs IntroMPI and OpenMPExamples repos