Skip to content

pmundkur/odisco

master
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code
This branch is up to date with discoproject/odisco:master.

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
lib
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ODisco is a library for implementing Disco map-reduce jobs in OCaml.
It implements the Disco worker protocol, and provides the plumbing to
connect user-specified computations to Disco data sources and results.

Prerequisites:
--------------

The library has the following dependencies:

. ocaml 3.12.0 or later

. ocaml-findlib (Debian/Ubuntu/Fedora) / godi-findlib (Godi) / findlib (Gentoo, source)

. libzip-ocaml-dev (Debian/Ubuntu) / ocaml-zip (Fedora) / godi-zip (Godi) / camlzip (Gentoo, source)

. Disco 0.4 or later (http://discoproject.org)

. sonet

  This is a simple asynchronous networking library for OCaml.

  $ git clone git://github.com/pmundkur/sonet.git
  $ cd sonet && make && make install

  To uninstall:
  $ cd sonet && make uninstall

The tests have the following dependencies:

. atd    (https://github.com/MyLifeLabs/atd) / libatd-ocaml-dev (Debian/Ubuntu) / godi-atd (Godi)
. atdgen (https://github.com/MyLifeLabs/atdgen) / libatdgen-ocaml-dev (Debian/Ubuntu) / godi-atdgen (Godi)
. biniou (https://github.com/mjambon/biniou) / libbiniou-ocaml-dev (Debian/Ubuntu) / godi-biniou (Godi)

Building:
---------

 $ make

Installing:
-----------

 $ make install

Uninstalling:
-------------

 $ make uninstall

Using:
------

. Write a module implementing the lib/task.ml:TASK interface.  Pass
  this module to Worker.start in your main program.  See
  tests/test_install_task.ml for an example.

  Read the documentation in lib/task.mli and lib/worker.mli for more
  details.

. Build it and link to the ODisco library.  See tests/Makefile for an
  example build.

. Submit your compiled executable to Disco using 'disco job'.

  For example, to do a word count on a large text file:

  $ make -C tests
  $ wget -O /tmp/bigfile.txt http://discoproject.org/media/text/bigfile.txt
  $ ddfs push text /tmp/bigfile.txt
  $ ddfs blobs text \
    | disco job -m -r -p ocamlwc tests/word_count.opt \
    | xargs disco wait \
    | xargs ddfs cat

About

Library for implementing Disco jobs in ocaml.

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • OCaml 95.0%
  • Makefile 5.0%