Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Brief Description of MaTEx TensorFlow (Old)
MaTEx-TensorFlow provides a set of scripts to provide a distributed memory implementation of TensorFlow using MPI.
The script is available in src/deeplearning/tensorflow_old/mpi-tensorflow.py
MaTEx TensorFlow supports data parallelism (input dataset is split in compute nodes and the model/gradients are averaged at each step using MPI All-to-all reduction). Please read the arxiv paper (Distributed TensorFlow with MPI - https://arxiv.org/abs/1603.02339) for more details.