Network System Reactive Framework - Distributed STDOUT Grepping
The big idea is that every process in a network, by writing to STDOUT, can be thought of as an event generator, where an event is a line of output. If we want to react to these events (logging, monitoring, elastic computing), we simply need to make the STDOUT more programmable.
LOBE is publish/subscribe broker built on top of STDOUT. When a process is controlled by LOBE, each write to STDOUT is published as an event, which is then relayed to the subscribes by the LOBE process. This allows dynamic tracing of the STDOUTs of the set of processes controlled by LOBE.
Inspired by DTrace, LOBE opens up system level tracing, which may discover facts not visible by looking at a single process.
LOBE is a glorified grep.
LOBE has a HTTP API. The basic idea is to expose a pipe by spawning a process with LOBE. Then multiple clients can subscribe to the exposed pipe (or pipes, if the subscriber is multiplexing over multiple processes).
Start a lobe
> $PORT=5678 > node lobe.js $PORT Server running at http://127.0.0.1:5678
Spawn processes. We'll run a simple counter script. We must give every process a name.
> curl 'localhost:5678/spawn?name=counter1&command=ruby%20count.rb' counter1 > curl 'localhost:5678/spawn?name=counter2&command=ruby%20count.rb' counter2
We can list the processes controlled by a LOBE,
> curl 'localhost:5678/list' counter1 counter2
If we want to see all the output of all the processes,
> curl 'localhost:5678/attach' pid(12872) 13 pid(12874) 13 pid(12872) 14 pid(12874) 14 pid(12872) 15 pid(12874) 15
You'll notice that we get the output from both processes. If you want to attach only to processes matched by a name (a regexp),
> curl 'localhost:5678/attach?process=counter1' pid(12872) 76 pid(12872) 77 pid(12872) 78 pid(12872) 79 pid(12872) 80 pid(12872) 81
Or if you want to grep for lines matching a pattern,
> curl 'localhost:5678/attach?data=6' pid(12872) 168 pid(12874) 168 pid(12872) 169 pid(12874) 169 pid(12872) 176 pid(12874) 176
Or you can do both. To kill a process, do,
> curl 'localhost:5678/kill?name=counter1'
A LOBE controls local processes only. To access remote pipes, we can arrange LOBE instances into parent-child relationship. A parent LOBE, then, would act as proxy to the child nodes.
We'll first run two instances of LOBEs, on different machines, perhaps. For now, we'll just run on localhost,
> node lobe.js 5000 > node lobe.js 5001
and we spawn processes in each LOBE,
> curl 'localhost:5000/spawn?name=counter1&command=ruby%20count.rb' counter1 > curl 'localhost:5001/spawn?name=counter2&command=ruby%20count.rb' counter2
then we spawn a third LOBE, which acts as the parent of the previous two,
> node lobe.js 5002 > curl 'localhost:5002/child?name=child1&addr=localhost:5000' child1 > curl 'localhost:5002/child?name=child2&addr=localhost:5001' child2
and if we list, we'll see all the processes controlled the child LOBES,
> curl 'localhost:5002/list' child1/counter1 child2/counter2
if we attach to the parent lobe, we get all the output from the child lobes,
> curl 'localhost:5002/attach' pid(13086) 223 pid(13090) 220 pid(13086) 224 pid(13090) 221 pid(13086) 225 pid(13090) 222 pid(13086) 226 pid(13090) 223
the list of names is kept synchronized. So if we kill a process,
> curl 'localhost:5000/kill?name=counter1' > curl 'localhost:5002/list' child2/counter2