MPI Tutorial

This example walks through using Latte in MPI mode by training a vgg-like model on the cifar10 dataset and the tiny imagenet dataset (from Stanford's cs231 project).

This tutorial expects the environment variable LATTE_ROOT to be set (i.e. export LATTE_ROOT=~/.julia/v0.4/Latte).

Dependencies

• MPI (>= 3)
• HDF5 (compiled with --enable-parallel)
• OpenCV (Optional for converting the tiny-imagenet dataset, unecessary for cifar10 example)

Build Latte libraries in MPI mode

# Build the IO library
cd $LATTE_ROOT/deps
cd IO
cmake -DLATTE_MPI=ON . && make
cp libLatteIO.so ../

# Build the communication library
cd ../communication
cmake . && make
cp libLatteComm.so ../

Notes

• For HDF5 related Cmake errors, try setting HDF5_ROOT=/path/to/HDF5/installation

cifar10

Prepare the Dataset

cd $LATTE_ROOT/examples/cifar10/data
./get-data.sh

Train a model

cd $LATTE_ROOT/examples/cifar10
LATTE_MPI=1 mpirun -np 2 julia vgg-mini.jl

Tiny-imagenet

Build imagenet convert

cd $LATTE_ROOT/utils/converter
cmake . && make
cp convert_imagenet $LATTE_ROOT/bin

Notes

• For HDF5 related Cmake errors, try setting HDF5_ROOT=/path/to/HDF5/installation
• If Cmake is having trouble finding OpenCV use the flag: -DOpenCV_DIR=/path/to/installation/share/OpenCV

Prepare the dataset

cd $LATTE_ROOT/examples/tiny-imagenet/data
./get-data.sh  # Optional argument: path to download/convert dataset (i.e. a scratch directory)

Train a model

cd $LATTE_ROOT/examples/tiny-imagenet
LATTE_MPI=1 mpirun -np 2 julia vgg-mini.jl