MPI Programming Tutorials

A comprehensive collection of Message Passing Interface (MPI) programming examples for teaching parallel computing concepts. This repository is designed for students learning MPI programming in Windows environments with Microsoft MPI and Visual Studio.

Repository Structure

Each tutorial is a self-contained project with full CMake support, detailed documentation, and batch files for easy building and execution:

1. 01-mpi-basics - Introduction to MPI, environment setup, and simple communication
2. 02-point-to-point - Point-to-point communication patterns
3. 03-collective-communication - Collective communication operations
4. 04-derived-datatypes - Creating and using derived datatypes
5. 05-virtual-topologies - Cartesian and graph topologies
6. 06-one-sided - One-sided communication (Remote Memory Access)
7. 07-hybrid-parallelism - Hybrid MPI+OpenMP programming
8. 08-non-blocking - Non-blocking communications
9. 09-performance-optimization - Performance tuning and optimization
10. 10-mpi-io - Parallel I/O with MPI
11. 11-fault-tolerance - Fault tolerance and error handling
12. 12-process-manager - Process management and dynamic processes
13. 13-custom-communicators - Creating and managing custom communicators
14. 14-advanced-patterns - Advanced communication patterns and algorithms
15. 15-debugging - Debugging MPI applications

Prerequisites

• Windows 10/11
• Visual Studio 2022 Community Edition
• Microsoft MPI (MS-MPI)
• CMake 3.20 or higher
• C++17 compatible compiler

Getting Started

Setting Up Your Environment

1. Install Visual Studio 2022 Community Edition

   • During installation, select "Desktop development with C++"
   • Ensure MSBuild tools are installed

2. Install Microsoft MPI (MS-MPI)

   • Download both the runtime and SDK from Microsoft MPI
   • Install both packages
   • Add the MPI installation directory to your system PATH (typically C:\Program Files (x86)\Microsoft SDKs\MPI\Bin)
   • Verify the installation with mpiexec -help in a command prompt

3. Install CMake

   • Download from CMake
   • Add CMake to the system PATH during installation

Building and Running the Examples

Each project follows the same pattern for building and execution:

1. Navigate to the project directory

cd 01-mpi-basics

2. Configure the project

configure.bat

3. Build the project

build_all.bat

4. Run the example

run.bat

Advanced Run Options

Most examples support additional command-line parameters:

run.bat --np 4           # Run with 4 processes
run.bat --debug          # Run in debug mode with additional information
run.bat --verbose        # Show detailed output
run.bat --help           # Show all available options

Learning Path

For the best learning experience, follow the tutorials in numerical order:

1. Start with 01-mpi-basics to understand MPI fundamentals
2. Move through point-to-point and collective communication (02 through 03)
3. Learn advanced data handling with derived datatypes and virtual topologies (04 and 05)
4. Explore advanced communication models like one-sided and non-blocking (06 and 08)
5. Study performance optimization and parallel I/O (09 and 10)
6. Master advanced topics like fault tolerance, process management, and custom communicators (11 through 13)
7. Finally, learn about advanced patterns and debugging techniques (14 and 15)

Each directory contains its own detailed README.md with:

• Concepts explanation
• Code walkthrough
• Build and run instructions
• Expected output and interpretation
• Exercises and further exploration ideas

Common Issues and Troubleshooting

MS-MPI Installation Problems

• Make sure both the runtime and SDK are installed
• Verify the PATH environment variable includes the MPI bin directory
• Check installation by running mpiexec -help in a command prompt

Build Errors

• Check if CMake can find MPI with cmake .. -G "Visual Studio 17 2022" -A x64
• Ensure Microsoft MPI SDK is properly installed
• Check Visual Studio has the necessary C++ components installed

Runtime Errors

• Make sure mpiexec is available in your PATH
• Check that processes can be launched with mpiexec -n 2 hostname
• Try running examples with fewer processes if you encounter resource limitations

Additional Resources

• MPI Forum - Official MPI specification
• Microsoft MPI Documentation
• MPI Tutorial - Excellent resource for learning MPI
• Using MPI - Classic book on MPI programming
• Designing and Building Parallel Programs - Free online book

Contributing

Contributions to improve the tutorials are welcome! Please feel free to submit pull requests or open issues to suggest improvements.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

• MPI Forum for the MPI specification
• Microsoft for MS-MPI implementation
• The parallel computing community for sharing knowledge and best practices
• All contributors who help improve these teaching materials

---

Happy MPI Programming!