Parallelizing Deep Learning using MPI and GPU

Implemented a sequential and parallel neural network model using data based parallelism in Python using MPI and GPU computing. Measured the improvement in performance and speed up in training time.

• mnist-nn.py: 3-layer neural network as digit recognizer (MNIST).
• mnist-nn-gpu.py: mnist-nn with GPU computing.
• mnist-nn-data-parallelism.py: mnist-nn with data parallelism (MPI and GPU).