Distributed Rex

Distibuted Rex is a addition to Rex for distributed Task execution with the help of gearman. So it is possible to scale rex.

Distributed Rex is setup of two components. One (or more) Worker and one Client. As more worker you run as more parallel tasks you are able to execute.

INSTALLATION

If you want to test Distributed Rex you have to follow these steps. But keep in mind, this is work in progress. It may crash your systems and eat all your jelly beans and pretzel sticks. And even worse - drink all your beer. You have been warned!

On the Client and the Worker Nodes

curl -L get.rexify.org | perl - --sudo -n Rex::Gearman

On the server (worker)

First you have to install gearman and start it.

After that you have to distribute your Rexfile (and all associated files (like templates, the "lib" directory, ssh keys, ...) to your worker machines. Please be carefull to name the directory where the Rexfile lives in exact the same way as on your client machine!

On your Client:

/home/user/projects/frontend
                       |-- Rexfile
                       |-- client.conf
                       +-- lib
                       |   +-- Rex
                       |       +-- Webserver
                       |       +-- Apache.pm
                       |           +-- templates
                       |               +-- httpd.conf
                       +-- worker.conf

On your Wroker:

/opt/rex/frontend
            |-- Rexfile
            |-- client.conf
            +-- lib
            |   +-- Rex
            |       +-- Webserver
            |       +-- Apache.pm
            |           +-- templates
            |               +-- httpd.conf
            +-- worker.conf

Now change into the directory where your Rexfile is located and create a file worker.conf.

Here you have to configure all your gearmand servers so that the Worker module can register its functions to them.

{
   job_servers => [
     "127.0.0.1:4730",
     "192.168.7.10:4730",
     "192.168.7.11:4730",
   ],
};

Now you are able to start the workerprocess.

rex-gearman-worker -d

RUNNING A TASK

Change to the directory where your Rexfile lives and create a file called client.conf. In this file you have to configure your gearmand servers.

{
   job_servers => [
     "127.0.0.1:4730",
     "192.168.7.10:4730",
     "192.168.7.11:4730",
   ],
};

Now you're able to run your tasks as known. But you have to replace the rex command with rex-gearman-client. The CLI parameter are the same. So you can use rex-gearman-client the same way as you used rex.

For example:

rex-gearman-client -h
rex-gearman-client -Tv
rex-gearman-client prepare

REPORTING BUGS

Please report the bugs in the Issue-Tracker.

GETTING HELP

If you need help, you can join us on irc on irc.freenode.net channel #rex.