PythonOpenMPI

A generalizable python-mpi utility for task-based parallel programming.

This implementation of task-based parallel programming consists of one root processor, and any number of worker processors. The root breaks a portion of a job into bite sized chunks (like a single file) which are then sent to the workers. While the workers... well... work, the root sits and waits. When a working finishes with its allotted chunk, it pings the root node and asks for another chunk, which the root node then provides. Therefore no worker is ever left without something to do.

This is fundamentally different and more efficient than data-based parallel processing in which an entire job is split into n equally sized chunks (where n is the number of processors) and sent to the worker processors. In this method, when a worker is done processing, it does not need to ask the root node for any more work (since everything has already been distributed to the workers). Therefore, although the task is being completed in parallel, there is a chance that workers will be left idle while they wait for other workers to finish.

Necessary packages

In addition to whichever packages you use for your process you wish to parallelize, you will need to ensure you have working installs of the following:

• python v3.X
• OpenMPI v4.X.X
• mpi4py python library

Ensure you have a properly configured and installed version of OpenMPI with your PATH and PYTHONPATH correctly pointed to the install directory before attempting to pip install mpi4py.

Examples using MPI

"Hello World"

1. cd into /examples
2. do mpiexec python hello_world.py

This will request all available processors to print "Hello, World!" to the screen along with their processor rank. This is a good way to determine how many cores your machine has and whether mpi4py/OpenMPI is installed correctly.

Read/write files

To run the test problem:

1. cd into the /examples directory.
2. do python gen_empty_files.py this will create a sub-directory /files within /read_write_parallel and will populate the directory with 10 blank text files of the form file_X.txt where X runs from 0 to 9. The number of test files created can be changed by editing gen_empty_files.py
3. do mpiexec -n {num_procs} python read_write_parallel.py while replacing num_procs with the number of processors you wish to parallelize the task across.

The output will reveal the communication between the supervisor processor and the worker processors: transferring "to-do" data around, sending completed messages back, etc. If you wish, you can edit the chunk size within read_write_parallel.py to see how the total work is broken up and how the individual processers handle different sizes of data.

Reducing array in-place

1. cd into /examples
2. do mpiexec python reduce_array.py

Each processor will build a (5,) numpy array with indicie [I] == I, where I is the unique processor rank (e.g. processor 2 will make [0 0 2 0 0]). Then, the root processor will collect all of the arrays and perform the MPI.SUM operation on them. The result (if using 4 processors) will be [0 1 2 3 4]. The MPI.SUM operatoin can be replaced with any other MPI op code.

Reducing dictionary in-place

1. cd into /examples
2. do mpiexec python reduce_dict.py

Each processor is given an empty dictionary and will insert a single key/value pair of str(rank):rank*100 (e.g. processor 2 dictionary is {'2':200}). The root processor then gathers all dictionaries from all processors and updates its own dictionary with the unique pairs.

---

The mpiexec calls used in the examples will only parallelize across a single node of any number of processors. If multiple nodes are needed, you will need to provide mpiexec with a hostfile.

How to parallelize your specific task

Upon investigating the perform_task_in_parallel() funciton within mpi_utilities.py you will notice that the "task" can be any function, with any number of input args and kwargs. If your task can be condensed into a single function with a discrete amount of input work, it can be parallelized with this routine. To utilize, call perform_task_in_parallel(your_func, [args], {kwargs}, input_data, chunk_size, rank, size, comm, root, debug=False) where rank, size, and comm are generated by the initialize_mpi() call. chunk_size is the amount of input work to be sent to each processor when it asks for something to do. [args] and {kwargs} are the arugments and keyword arguments for your_func.

This repository is meant to aid other graduate students in the Drexel University Physics Department, the University at large, and beyond.

Contributors:

• Sean C. Lewis (owner, scl63@drexel.edu)

• Evan Arena