OCaml Distributed Data Processing
A distributed data processing system developed in OCaml
How to express arbitrary DAG? How to express loop? Apply function.
Interface to Irmin or HDFS to provide persistent storage
Test delay-bounded and error-bounded barrier
Split Context module into server and client two modules
Implement barrier control in parameter modules
Rename ... DataContext and ModelContext?
Implement Coordinate Descent in model parallel ...
Enhance Mapreduce engine, incorporate with owl.
Add techreport based on the barrier control.
How to compile & run it?
To compile and build the system, you do not have to install all the software yourself. You can simply pull a ready-made container to set up development environment.
docker pull ryanrhymes/actor
Then you can start the container by
docker run -t -i ryanrhymes/actor:latest /bin/bash
After the container starts, go to the home director, clone the git repository.
git clone https://github.com/ryanrhymes/actor.git
Then you can compile and build the system.
make oasis && make