Skip to content
Switch branches/tags

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Minion Build Status Coverage Status

iobio server that easily turns a command line program into an iobio web service.


Minion is the server side code that powers all iobio web services. It wraps a command line program and hooks the input files/streams and stdout streams to websocket streams, enabling multiple web services to be piped together. It also handles converting the URL into a command and error handling. If you are wanting run a particular iobio webservice see the minion-services repo.

Web Services

iobio is made up of numerous micro web services that are each capable of supporting multiple apps. These web services can be mixed and matched to create a wide range of functionality. In this way a new web app can grab a set of relevant web services and then build a UI and visualization around them.

Public Web Services

You can develop apps against our public web services but due to resource constraints we can only run a set number of concurrent requests. Requests that can not be proccessed immediately are placed in a queue and processed in turn. If you need more power, you can install the web services yourself or contact us. All public services are located at

Local Install

If you need to install webservices locally there are two ways to do it


For testing it's often easier to install minion directly and place your required services in the service directory. First install minion then place your web service definition file in the minion/services/ directory. A list of web service definition files can be found in the minion-services repo. Lastly you'll need copy the tool you are turning into a web service into the minion/bin/ directory or it won't run (see create your own web service for more details).

// where someService is the service definition file for a tool (e.g. samtools) you want to turn into a webservice
cd minion/
mv someService.js minion/services/
// start service
node services/someService.js

For production you may want to use dockerized containers


If you want to improve minion or create your own webservice, you'll want to download and install minion itself.


To get going you need to clone the repo from github

git clone

Install Dependencies

This will install all needed node modules

cd minion; npm install

Run Tests

npm test

Config File

Minion services can take a config file on startup. Otherwise the default config file lib/config.js will be used

To use config file
node someService.js --config file.config
Example config file
// Minion Config file

var config = {
        platform : 'default',                   // 'default' returns nothing. 'amazon' will use amazon dns lookup to get hostname so you can use ssl
        maxConcurrentCommands: null,    // accepts a number or null. null allows unlimimited commands
        noQueueDomains: ['localhost', '', null], // lists domains that will never be queued
        cacheDir: 'cache'

module.exports = config;

Create Your Own Web Service

Minion wraps command line tools and converts them into iobio web services that are instantly pluggable into the iobio ecosystem.

Simple example

For this example we will create a service from the linux utility wc to count lines, words, and characters.

Create the iobio service wrapper file

Create a new file called wc-iobio.js in minion/services/ and paste the following js into it.

var port = 4003;
    minion = require('../index.js')(port);

// define tool
var tool = {
   apiVersion : "0.1",
   name : 'wc',
   path :  'wc',
   description : 'count lines, words, characters',
   exampleUrl : ""

// start minion socket
console.log('iobio server started on port ' + port);
Add tool into whitelisted directory

For security iobio only allows the execution of programs in the minion/bin directory so to enable execution of the wc utility a sym link needs to be created there. To create the symlink run the following command line. If wc is not at /usr/bin/ then you can find it's location with which wc.

$ cd minion/bin; ln -s /usr/bin/wc wc
Test new service

Start up server

node services/wc-iobio.js

To quickly test the new service, you can paste this url in the browser

This will run the file at through the wc web service. You should get back 1 6 28

Use new service

To seriously use this service, you'll want to use the iobio.js library which will create iobio urls and manage websocket connections. Here's an example

	<!DOCTYPE html>
		<script src='iobio.js'></script>

			// encode URL to tell the wc web service this is a input source and not a command line parameter
			// This is temporary and will be incoporated into the library
			var url = encodeURIComponent('');

			// Create command
			var cmd = new iobio.cmd(
		        [url], // parameters
		        { 'urlparams': {'protocol':'http'} } // use http option since input origin is a http server and not a iobio web service

		    cmd.on('data', function(results) {

			cmd.on('error', function(error) { console.log(error); });

To add parameters, add them as you would any iobio command. Here we add the -c option to only get back characters

  // Create command
  var cmd = new iobio.cmd(
    ['-c', url],
    { 'urlparams': {'protocol':'http'} } // use http sense input origin is a http server and not a iobio web service
Tool with sub-programs

./sometool subprogram --flag1 arg1

var tool = {
   apiVersion : "0.1",
   name : 'sometool',
   path: 'sometool/' // path to directory where subprograms live

cacheTransform allows a stream to be piped through a transform pipe before being cached. Here's an example of using the last-record-stream utility to cache only the last record coming back from bamstatsalive

var tool = {
   apiVersion : "0.1",
   name : 'bamstatsalive',
   path :  'bamstatsalive',
   cacheTransform: lastRecord


Server wrapper that turns command line tools into web services




No packages published

Contributors 4