User_Using your own compute workers

In these instructions, we guide you step-by-step to create your own "compute worker" (a server to which submissions of challenge participants are sent to be executed).

Contents

• Setting up a server as compute worker
• Hooking up the server to a queue
• Assigning a queue to a competition

---

Setting up a server as compute worker

First you need to have access to a server connected to the Internet having an IP address and into which you can remote login via ssh. If you do not have one, you need to create one. We provide instructions for creating Ubuntu Virtual machines (VM):

• Creating a Windows Azure Ubuntu VM
• Creating an AWS Ubuntu VM

Then ssh into your machine and run the following commands:

• Get docker (quick and dirty way; for a full clean installation see Docker Ubuntu instructions):

  curl https://get.docker.com | sudo sh

• Adding yourself to the docker group (so you don't nee to run as root):

  sudo usermod -aG docker ubuntu

• Log out of your server and log back in. Then check your installation:

  docker run hello-world

• Run this command to be able to run many containers at once:

  sudo apt-get install docker-compose

• Clone the Codalab Competitions docker image:

  git clone https://github.com/codalab/codalab-competitions.git

• Create and edit the configuration file .env:

  cd codalab-competitions cp .env_sample .env vim .env

Note: vim is an editor coming with Ubuntu. If you rather use emacs, type first sudo apt-get install emacs.

• Uncomment BROKER_URL= and paste your worker queue key:

  BROKER_URL=pyamqp://61fdd17e-fb04-4296-9d30-6cd3a69c8aa8:8cc2cd6e-ce72-4592-9483-fe6b6c8b2495@13.82.145.119/030e877d-5f61-479c-8c87-7da347828287

To get your key, see next section.

• Get your worker to start computing (it will start listening to the queue via BROKER_URL and pick up jobs):

  docker-compose up -d worker_compute

Hooking up the server to a queue

Worker queues

Assigning a queue to a competition