Permalink
Browse files

ENH: Added Docker files

  • Loading branch information...
vojtechcima committed Apr 18, 2017
1 parent 888004a commit 7a9bf30e763a1717512d20a1fcdcee27456e1b8a
Showing with 77 additions and 0 deletions.
  1. +15 −0 Dockerfile
  2. +48 −0 README.md
  3. +14 −0 docker-compose.yml
@@ -0,0 +1,15 @@
FROM it4i/loombuilder:latest

RUN apt-get update -y && apt-get install -y git

RUN git clone https://code.it4i.cz/ADAS/loom.git && \
cd loom && \
mkdir _build && \
cd _build && \
cmake .. && \
make -j2 && \
cd ../python && \
sh generate.sh && \
python3 setup.py install

ENV PYTHONPATH "$PYTHONPATH:/loom/python/"
@@ -7,3 +7,51 @@ pipelines.
* High-level Python interface, back-end written in C++
* Peer-to-peer data sharing between workers
* Low latency & low task overhead to process hundred thousands of tasks

## Quickstart

Execute a Loom pipeline in 4 easy steps:

### 1. Deploy virtualized Loom infrastructure

```
docker-compose up
```

Note that before re-running `docker-compose up` you need to run `docker-compose down` to delete containers state.

### 2. Install Loom client (virtualenv)

```
virtualenv -p python3 loom_client_env
source loom_client_env/bin/activate
pip3 install cloudpickle protobuf
cd ./python
chmod +x generate.sh
./generate.sh
python3 setup.py install
```

### 3. Define a pipeline

Create a python file `pipeline.py` with the following content:

```
from loom.client import Client, tasks
task1 = tasks.const("Hello ") # Create a plain object
task2 = tasks.const("world!") # Create a plain object
task3 = tasks.merge((task1, task2)) # Merge two data objects together
client = Client("localhost", 9010) # Create a client object
future = client.submit_one(task3) # Submit task
result = future.gather() # Gather result
print(result) # Prints b"Hello world!"
```

### 4. Execute the pipeline

```
python3 pipeline.py
```
@@ -0,0 +1,14 @@
loom-master:
build: ./
dockerfile: Dockerfile
command: /loom/_build/src/server/loom-server
ports:
- 9010:9010

loom-worker:
build: ./
dockerfile: Dockerfile
links:
- loom-master
command: /loom/_build/src/worker/loom-worker loom-master 9010

0 comments on commit 7a9bf30

Please sign in to comment.