Home
DTK Arbiter is a ruby based service that is run on DTK nodes. DTK Arbiter reports the state of the nodes and executes workers on them concurrently. Communication between DTK Server and DTK Arbiter is entirely STOMP based. The design of arbiter is such that allows horizontal scaling where DTK Server's commands are broadcast to all owned nodes.
Above diagram shows the order of actions that happen when DTK Server provisions AWS node. As well as DTK Arbiter role in setup and configuration of node.
We differentiate between four types of messages used by DTK Arbiter
- Heartbeat message, sent by DTK Arbiter to report that node has been started
- Pong message, sent from DTK Arbiter to keep STOMP connection alive
- Request message, sent from DTK Server to DTK Arbiter with instruction on what to run
- Response message, sent from DTK Arbiter to DTK Server with results of request run
For the security, we are using a combination of public key encryption and symmetric key algorithm . Simply put message due to their size will be encrypted advanced encryption standard, ciphers will be sent as well encrypted with SSH key. This means that each message travelling over the web will not be encrypted the same, and even if messages are intercepted they can only be decrypted with SSH key.
It is very easy to extend current functionality of DTK Arbiter. Workers are run after a message is received, and for us to add new functionality we essentially create new worker or action under workers. Workers have been separated by their responsibility e.g. docker, ssh, puppet, system, and they allow different actions to be run. Meaning one message can trigger worker (default action), or specific action on a worker. The good example of this is the system worker which supports multiple actions encapsulated in one worker.
Request message, mentioned before, specifies :agent and :action, if :action is not provided default action is used. We would specify request
{
agent: :system_agent,
action: :get_ps
}
to trigger action specified action on system worker.
We've tried to bring inner workings of DTK Arbiter close with explanations but to best understand it, it would be easiest to use it. You can start DTK Arbiter in development mode by running.
bundle install --with development
ruby start.rb --development
You would also message bus with STOMP supported to communicate with running instance of DTK Arbiter. More information on how to setup this connection can be found here. Also available is cheatsheet for debugging potential DTK Arbiter issues.