Simple OpenMPI hello world program that initializes main and worker process on every node. The main process is started on node with world_rank == 0, while the worker is started on every other node.
Main and worker processes inherit from Process class. - Process class defines the world communicator, where the processes exists; this enables to distribute world_rank(s) to every process, thus distinguish main and worker process
Main and worker processes inherit from COMM_Groups class. - COMM_Groups class defines colors of communicators
The worker processes used traditional Scatter-Work-Gather strategy to split the work load. The Scatter is started by main process using MPI_Send to distribute the data.
The main process uses MPI_Barrier to synchronize with worker process. See the image below to see the synchronization in time.
NOTICE: For the matter of exercise, the worker nodes are actually structured, where the highlighted Node 1 has a leading role in data distribution. In practice, this role belongs to the main node.