README

DESCRIPTION

Define tasks for system administration or code deployment, then execute them on
one or many remote machines simultaneously. Strong logging creates a complete
audit trail of commands executed on remote machines in logs easily analyzed by
standard text manipulation tools. 

node-control depends only on OpenSSH and Node on the local control machine.
Remote machines simply need a standard sshd daemon.



QUICK EXAMPLE

If you want to control remote machines from individual scripts without the
tasks system, see QUICK EXAMPLE WITHOUT TASKS. Otherwise, to get the
current date from the two machines listed in the 'mycluster' config task as
the 'mylogin' user with a single command:

var control = require('control'),
    task = control.task;

task('mycluster', 'Config for my cluster', function () {
    var config = {
        'a.domain.com': {
            user: 'alogin'
        },
        'b.domain.com': {
            user: 'blogin',
            sshOptions: ['-p 44'] // sshd daemon on non-standard port
        }
    };

    return control.controllers(config);
});


task('date', 'Get date', function (controller) {
    controller.ssh('date');
});

control.begin();


If saved in a file named 'controls.js', run with:

node controls.js mycluster date


Each machine is contacted in parallel, date is executed, and the output from
the remote machine is printed to the console. Example console output:

 Performing mycluster
 Performing date for a.domain.com
a.domain.com:mylogin:ssh: date
 Performing date for b.domain.com
b.domain.com:mylogin:ssh: date
a.domain.com:stdout: Sun Jul 18 13:30:50 UTC 2010
b.domain.com:stdout: Sun Jul 18 13:30:51 UTC 2010
a.domain.com:exit: 0
b.domain.com:exit: 0
           

Each line of output is labeled with the address of the machine the command was
executed on. The actual command sent and the user used to send it is
displayed. stdout and stderr output of the remote process is identified
as well as the final exit code of the local ssh command. Each command, stdout,
stderr, and exit line also appears timestamped in a control.log file in the
current working directory.

See CODE DEPLOYMENT EXAMPLE for an example of deploying an application to
remote servers.



INSTALLATION

If you use npm:

npm install control

If you do not use npm, clone this repository with git or download the latest
version using the GitHub repository Downloads link. Then use as a standard Node
module by requiring the node-control directory.



EXAMPLE CONTROLS

As you read this documentation, you may find it useful to refer to the
example/controls.js file. Its work tasks cover a variety of advanced usage. The
config tasks use your local machine as a mock remote machine or cluster, so if
you run an sshd daemon locally, you can run the controls against your own
machine to experiment. 



CONFIG TASKS

When using tasks, you always identify two tasks on the command line for remote
operations. The first task is the config task and the second task is the work
task. Config tasks have a name, description, and function that will be called
once:

task('mycluster', 'Config for my cluster', function () {


The config task function must return an array of controllers (objects that
extend the control.controller prototype, described further in CONTROLLERS).
Each controller in the array controls a single machine and optionally has its
own properties. 

Config tasks enable definition of reusable work tasks independent of the
machines they will control. For example, if you have a staging environment with
different machines than your production environment, you can create two
different config tasks, each returning controllers for machines in the
respective environment, yet use the same deploy work task:

node controls.js stage deploy ~/myapp/releases/myapp-1.0.tgz

node controls.js production deploy ~/myapp/releases/myapp-1.0.tgz


If all the machines in a cluster share common properties, you can extend the
control.controller prototype and pass the new prototype into controllers() as
the second argument. For example, if all the machines in your cluster run sshd
on a non-standard port instead of just one as in QUICK EXAMPLE:

task('mycluster', 'Config for my cluster', function () {
    var shared = Object.create(control.controller),
        config = {
            'a.domain.com': {
                user: 'alogin'
            },
            'b.domain.com': {
                user: 'blogin'
            }
        };

    shared.sshOptions = ['-p 44'];

    return control.controllers(config, shared);
});


controllers() will return an array of controllers that prototypically inherit
from the shared prototype instead of the base prototype, each having
controller-specific properties as defined in the JSON notation. In this case,
both controllers will effectively have sshOptions = ['-p 44'].

If all machines in your cluster have the same properties, can you pass an array
of addresses as the first argument to controllers(). For example, if all the
machines your cluster run sshd on a non-standard port and you use the same
login on each:

task('mycluster', 'Config for my cluster', function () {
    var shared = Object.create(control.controller), 
        addresses = [ 'a.domain.com',
                      'b.domain.com',
                      'c.domain.com' ];
    shared.user = 'mylogin'; 
    shared.sshOptions = ['-p 44'];
    return control.controllers(addresses, shared);
});


You can also build up your list of controllers without the use of
controllers():

task('mycluster', 'Config for my cluster', function () {
    var controllers = [],
        shared = Object.create(control.controller), // Extend prototype 
        a, b;

    shared.sshOptions = ['p 44'];

    a = Object.create(shared); // Extend shared prototype
    a.address = 'a.domain.com';
    a.user = 'alogin'; 
    controllers.push(a);

    b = Object.create(shared);
    b.address = 'b.domain.com';
    b.user = 'blogin'; 
    controllers.push(b);

    return controllers;
});



WORK TASKS

Work tasks define logic to drive each controller returned by the config task.
They have a name, description, and a callback that will execute independently
and simultaneously for each controller:

task('date', 'Get date', function (controller) {


Arguments on the command line after the name of the work task become arguments
to the work task's function. With this task:

task('deploy', 'Deploy my app', function (controller, release) {


This command:

node controls.js stage deploy ~/myapp/releases/myapp-1.0.tgz


Results in: 

release = '~/myapp/releases/myapp-1.0.tgz'


More than one argument is possible:

task('deploy', 'Deploy my app', function (controller, release, tag) {



TASK EXECUTION

To execute the tasks identified on the command line, use the begin() method
after you have defined all your config and work tasks:

var control = require('control');
... // Define tasks
control.begin();


begin() calls the first (config) task identified on the command line to get the
array of controllers, then calls the second (work) task with each of the
controllers. If you run a control script and nothing happens, check if the
script calls begin().



CONTROLLERS

node-control provides a base controller prototype as control.controller, which
all controllers must extend. To create controllers, use the controllers()
method described in CONFIG TASKS or extend the base controller prototype and
assign the controller a DNS or IP address, user if not the same as the local
user, and any other properties required by work tasks or further logic:

var controller = Object.create(control.controller);
controller.address = 'a.domain.com'; // Machine to control
controller.user = 'mylogin'; // Username on remote machine if not same as local
controller.ips = [ // Example of property used by work task or further logic 
        '10.2.136.23',
        '10.2.136.24',
        '10.2.136.25',
        '10.2.136.26',
        '10.2.136.27' 
    ];


The base controller prototype provides ssh() and scp() methods for
communicating with a controller's assigned remote machine.

The ssh() method takes one argument - the command to be executed on the
remote machine. The scp method takes two arguments - the local file path and the
remote file path. 

Both ssh() and scp() methods are asynchronous and can additionally take a
callback function that is executed once the ssh or scp operation is complete.
This guarantees that the first operation completes before the next one begins
on that machine:

    controller.scp(release, remoteDir, function () {
        controller.ssh('tar xzvf ' + remotePath + ' -C ' + remoteDir, 
                function () {


You can chain callbacks as far as necessary.

If a command returns a non-zero exit code, the scp() and ssh() methods will log
the exit and exit code, but will not call the callback, ending any further
operations on that machine. This avoids doing further harm where a callback may
assume a successful execution of a previous command. However, you can specify
an exit callback that will be called and receive the exit code if a non-zero
exit occurs:

function callback() { ... }
function exitCallback(code) { ... }

controller.ssh('date', callback, exitCallback);


You can make both callbacks the same callback function if you want to check the
exit code and handle both zero and non-zero exits within a single callback.



CUSTOM LISTENERS

When running a command with ssh() on a remote device, controller objects listen
to the stdout and stderr of the process running on the remote device through
the local ssh process, printing what is heard to console and log. You can
attach your own listeners to these stdout and stderr streams to gather data to
use in your callback function:  

task('ondate', 'Different logic paths based on date', function (controller) {
    var datestring = '';

    controller.stdout.on('data', function (chunk) {
        datestring += chunk.toString();
    });

    controller.ssh('date', function () {
        console.log('  Date string is ' + datestring);
        // Further logic dependent on value of datestring
    });
});


Refer to Node's ReadableStream and EventEmitter documentation if the
stdout.on() pattern looks unfamiliar. Controllers also provide a stderr.on()
for attaching custom listeners to the stderr stream.

You can respond to prompts and errors as they happen in the remote process
through the remote process stdin stream similar to expect. An example of
responding to a prompt through the stdin of the remote process:

task('stdin', 'Test controller stdin usage', function (controller) {
    var stdout;

    controller.stdout.on('data', function (chunk) {
        chunk = chunk.toString(); // Assumes chunks come in full lines
        if (chunk.match('^Enter data')) { // Assumes command uses this prompt
            controller.stdin.write('hello\n');
        }
    });

    controller.ssh('acommand');
});


The controller only uses custom listeners for the next ssh() or scp() call.
Further ssh() or scp() calls will not attach the custom listener unless it is
reattached via controller.stdout.on() or controller.stderr.on() before the next
call. This avoids unanticipated usage of one-off listeners such as filling the
datestring variable in the first example with the output of every subsequent
ssh() command executed by the controller.



PERFORMING MULTIPLE TASKS

A task can call other tasks using perform() and optionally pass arguments to
them:

var perform = require('control').perform;

task('mytask', 'My task description', function (controller, argument) {
    perform('anothertask', controller, argument);


perform() requires only the task name and the controller. Arguments are
optional. If the other task supports it, optionally pass a callback function as
one of the arguments:

    perform('anothertask', controller, function () {


Tasks that support asynchronous performance should call the callback function
when done doing their own work. For example:

task('anothertask', 'My other task', function (controller, callback) {
    controller.ssh('date', function () {
        if (callback) {
            callback();
        }
    });
});


The peform() call can occur anywhere in a task, not just at the beginning.



LISTING TASKS

To list all defined tasks with descriptions:

node controls.js mycluster list



NAMESPACES

Use a colon, dash, or similar convention when naming if you want to group tasks
by name. For example:

task('bootstrap:tools', 'Bootstrap tools', function (controller) { 
...
task('bootstrap:compilers', 'Bootstrap compilers', function (controller) { 



SUDO

To use sudo, just include sudo as part of your command:

controller.ssh('sudo date');


This requires that sudo be installed on the remote machine and have requisite
permissions setup.



ROLES

Some other frameworks like Capistrano provide the notion of roles for different
machines. node-control does not employ a separate roles construct. Since
controllers can have any properties defined on them in a config task, a
possible pattern for roles if needed:

task('mycluster', 'Config for my cluster', function () {
    var dbs = Object.create(control.controller),
        apps = Object.create(control.controller);

    dbs = {
        user: 'dbuser',
        role: 'db'
    };

    apps = {
        user: 'appuser',
        role: 'app'
    };

    dbs = control.controllers(['db1.domain.com', 'db2.domain.com'], dbs);
    apps = control.controllers(['app1.domain.com', 'app2.domain.com'], apps);

    return dbs.concat(apps); 
});

task('deploy', 'Deploy my system', function (controller, release) {
    if (controller.role === 'db') {
        // Do db deploy work
    }

    if (controller.role === 'app') {
        // Do app deploy work
    }
});



LOGS

All commands sent and responses received are logged with timestamps (from the
control machine's clock). By default, logging goes to a control.log file in the
working directory of the node process. However, you can override this in your
control script:

task('mycluster', 'Config for my cluster', function () {
    var shared, addresses;
    shared = {
        user: 'mylogin',
        logPath: '~/mycluster-control.log'
    };
    addresses = [ 'a.domain.com',
                  'b.domain.com',
                  'c.domain.com' ];
    return control.controllers(addresses, shared); 
});


Since each controller gets its own log property, every controller could
conceivably have its own log fie. However, every line in the log file has a
prefix that includes the controller's address so, for example: 

grep a.domain.com control.log | less


Would allow paging the log and seeing only lines pertaining to
a.domain.com.


If you send something you do not want to get logged (like a password) in a
command, use the log mask:

controller.logMask = secret;
controller.ssh('echo ' + secret + ' > file.txt');


The console and command log file will show the masked text as asterisks instead
of the actual text.



SSH

To avoid repeatedly entering passwords across possibly many machines, use
standard ssh keypair authentication. 

Each controller.ssh() call requires a new connection to the remote machine. To
configure ssh to reuse a single connection, place this:

Host *
ControlMaster auto 
ControlPath ~/.ssh/master-%r@%h:%p


In your ssh config file (create if it does not exist):

~/.ssh/config


To pass options to the ssh command when using ssh(), add the option or options
as an array to the sshOptions property of the controller or controllers'
prototype:

controller.sshOptions = [ '-2', '-p 44' ];


Use scpOptions in the same manner for scp().



CONFIG TASK COMMAND LINE ARGUMENTS REWRITING

Config tasks receive a reference to the array of remaining arguments on the
command line after the config task name is removed. Therefore, config tasks
can rewrite the command line arguments other than the config task name. Example:

function configure(addresses) {
    var shared;
    shared = {
        user: 'mylogin'
    };
    return control.controllers(addresses, shared); 
}

task('mycluster', 'Config for my cluster', function () {
    var addresses = [ 'a.domain.com',
                      'b.domain.com',
                      'c.domain.com' ];
    return configure(addresses); 
});

task('mymachine', 'Config for one machine from command line', function (args) {
    return configure([args.shift()]); // From command line arguments rewriting
});



With this set of config tasks, if there is an ad hoc need to run certain tasks
against a single machine in the cluster, but otherwise have identical
configuration as when run as part of the cluster, the machine address can be
specified on the command line:

node controls.js mymachine b.domain.com mytask x


In that case, the mymachine config task receives as args:

['b.domain.com', 'mytask', 'x']


This is generally not necessary since you can edit the config task in the
control file at any time, but is available if config tasks need to have command
line arguments or rewrite the work task name and its arguments on the fly.



CODE DEPLOYMENT EXAMPLE

A task that will upload a local compressed tar file containing a release of a
node application to a remote machine, untar it, and start the node application.

var path = require('path');

task('deploy', 'Deploy my app', function (controller, release) {
    var basename = path.basename(release),
        remoteDir = '/apps/',
        remotePath = path.join(remoteDir, basename),
        remoteAppDir = path.join(remoteDir, 'myapp');
    controller.scp(release, remoteDir, function () {
        controller.ssh('tar xzvf ' + remotePath + ' -C ' + remoteDir, 
                function () {
            controller.ssh("sh -c 'cd " + remoteAppDir + " && node myapp.js'"); 
        });
    });
});

Execute as follows, for example:

node controls.js mycluster deploy ~/myapp/releases/myapp-1.0.tgz


A full deployment solution would shut down the existing application and have
different directory conventions. node-control does not assume a particular
style or framework. It provides tools to build a custom deployment strategy for
your application, system, or framework.



QUICK EXAMPLE WITHOUT TASKS

You can create scripts to run individually instead of through the tasks system
by using controllers() to create an array of controllers and then using
the controllers directly:

var control = require('../'),
    shared = Object.create(control.controller),
    i, l, controller, controllers;

shared.user = 'mylogin';
controllers = control.controllers(['a.domain.com', 'b.domain.com'], shared);

for (i = 0, l = controllers.length; i < l; i += 1) {
    controller = controllers[i];
    controller.ssh('date');
}


If saved in a file named 'controls.js', run with:

node controls.js


See example/taskless.js for a working example you can run against your local
machine if running a local sshd.



FEEDBACK

Welcome at node@thomassmith.com or the Node mailing list.