Event based config generator for Marathon tasks
Go Shell
Latest commit 0f2fad6 Jun 26, 2015 @byxorna Merge pull request #11 from byxorna/handle-health-status
Handle health status
Failed to load latest commit information.
example rename vhost to name and add example for named SRV zone May 20, 2015
.gitignore rename to goji Feb 1, 2015
LICENSE Initial commit Jan 28, 2015
README.md Merge branch 'master' of github.com:byxorna/goji Jun 26, 2015
config.go use /v2/tasks instead of app specific task route Jun 26, 2015
main.go support -app-required flag to allow missing apps in marathon Jun 25, 2015
util.go gofmt Feb 5, 2015


Goji: Marathon Task Proxy Config Generator

goji is a server that registers with a Marathon instance, consumes events, and emits templated configs containing information about running tasks for a set of apps that you care about.

Build Status Build Status


One-shot mode

Just hit up marathon for the task list, generate a config, and write it out.

./goji -conf myconfig.json


Generates a config like one-shot mode, but goji will then listen on http-port for events from Marathon.

./goji -conf myconfig.json -server


An automated build of master is available at registry.hub.docker.com/u/byxorna/goji. The container is more useful when run in single shot mode, as opposed to -server mode. You can use dockers -v argument to mount directories into the container to provide a config or output directory.

# docker run -it byxorna/goji -h
Usage of ./goji:
  -app-required=false: Require marathon applications to exist (assumes no tasks for missing apps if false)
  -conf="": Config JSON file
  -server=false: Start a HTTP server listening for Marathon events
  -target="": Target file to write to


goji takes a config file, formatted in json, as the -conf option. It tells goji information about your marathon instance, what services (app IDs) to query for tasks, and where and with what template to write configs out.

  "services": [
    { "app-id": "/sre/byxorna/site", "name": "pipefail.service.iata.tumblr.net" },
    { "app-id": "/sre/byxorna/app1", "name": "app1.service.iata.tumblr.net", "protocol":"TCP" },
    { "app-id": "/sre/byxorna/webapp", "name": "web.service.iata.tumblr.net", "protocol":"HTTP", "health-check":"/_health" },
    { "app-id": "/sre/byxorna/appwithopts", "name": "myapp", "protocol":"HTTP", "health-check":"/_health",
      "options": { "acl-match": "-m beg", "healthcheck-rate":"100"}}
  "template": "templates/haproxy.tmpl",
  "target": "/tmp/haproxy.cfg",
  "command": "/usr/bin/check_haproxy_config /tmp/haproxy.cfg && cp /tmp/haproxy.cfg /etc/haproxy/haproxy.cfg && service haproxy reload",
  "http-port": 8000,
  "delay": 5
  • marathon-host: Hostname of the marathon instance to connect to (required)
  • marathon-port: Port the marathon service is running on (optional, default: 8080)
  • template: Template config file to feed services and tasks into, text/template format (required)
  • target: Write templated configuration to this location (required)
  • http-port: What port to start an HTTP event listener on to register and receive event messages from marathon (optional, default: 8000)
  • delay: Coalesce events within this window before triggering a task get and config emit (optional, default: 0)
  • command: Run a script after writing out the config (optional, default: empty)
  • services: List of Services. A service is an object with a app-id key of the marathon app ID you want tasks from, and a name that will be passed into your template for each service. See below.

Service Configuration

  // marathon app id for your application
  "app-id": "/sre/byxorna/webapp",
  // a name that is associated with the service. Useful for doing nginx vhosting for http apps, or service name for DNS SRV records. Just a string
  "name": "web.service.iata.tumblr.net",
  // TCP or HTTP, defaults to HTTP
  // if the protocol of the service is HTTP, you can specify a health check URI here
  // what service port to use. Defaults to 80, but you can override for TCP services
  // options is an optional map[string]string of arbitrary options you can switch on in your templates
  // these are useful to specify behavior logic per service
  options: { "acl-match": "-m beg", "healthcheck-rate":"100" }


The comand field in the config json specifies a command to run upon successful creation of a new config. This can be anything you want, but here are some useful examples:

HAProxy Reloading

"command":"cp /etc/haproxy/haproxy.cfg /etc/haproxy/haproxy.cfg.bak && cp ./haproxy.cfg /etc/haproxy/haproxy.cfg && service haproxy check && service haproxy reload || (mv /etc/haproxy/haproxy.cfg.bak /etc/haproxy/haproxy.cfg && exit 1)",


You can use goji to emit whatever configs you care about. Common usecases would be HAproxy or Nginx configurations. The name attribute could be useful for doing nginx/apache vhosting to identify a request by Host: header.

Here is an example:

This is a test template
{{ range $index, $service := . }}AppId {{ $service.AppId }} at {{ $service.Name }}{{ range $i, $task := $service.Tasks }}
{{ range $j, $port := $task.Ports }}  task: {{ $task.Id }} {{ $task.Host }}:{{ $port }}{{ end }}{{ end }}{{ end }}

A much more useful template is available in example/haproxy.tmpl, example/nginx.tmpl, and example/named.tmpl

You may use arbitrary keys and values in $service.Options to do clever things per service. For example, this snippet will allow you to modify how the haproxy acl rule works with options stored per service. acl {{ $service.EscapeAppIdColon }}-aclrule hdr(host) {{with index $service.Options "acl-opts"}}{{index $service.Options "acl-opts"}} {{end}}{{ $service.Name }}

DNS SRV Records

You can generate a named zone with SRV records from marathon backends trivially, given the template:

@ IN SOA ns1.example.com. admin.example.com. (
  12345      ; serial
  600        ; refresh
  1800       ; retry
  604800     ; expire
  300        ; minimum

  IN NS ns1.example.com.
  IN NS ns2.example.com.

$ORIGIN goji.example.com.
; service.proto.owner-name     ttl   class   rr    pri   weight    port    target
; _http._tcp.goji.example.com. 60    IN      SRV   0     5         301234  ct-12345.iata.example.com.
{{ range $index, $service := . }}
; SRV for {{$service.Name}}
{{ range $i, $task := $service.Tasks }}{{ range $j, $port := $task.Ports }}_{{$service.Name}}._{{$service.Protocol}}  {{with index $service.Options "ttl"}}{{index $service.Options "ttl"}}{{else}}60{{end}}  IN SRV 0 5 {{$port}} {{$task.Host}}.{{end}}

And config:

  "services": [
    { "app-id": "/sre/website", "name": "http", "protocol":"HTTP"},
    { "app-id": "/sre/tracker", "name": "tracker","protocol":"UDP","options":{"ttl":"120"}},
    { "app-id": "/sre/tcpservice", "name": "myservice", "protocol":"TCP"}
  "template": "example/named.tmpl",
  "target": "./named.zone.cfg",
  "command": "named-checkzone goji.example.com ./named.zone.cfg && cp ./named.zone.cfg /var/named/services.zone && service named reload",
  "delay": 5


$ go build
$ ./goji -help
Usage of ./goji:
  -app-required=false: Require marathon applications to exist (assumes no tasks for missing apps if false)
  -conf="": Config JSON file
  -server=false: Start a HTTP server listening for Marathon events
  -target="": Target file to write to

More information

Similar in function to https://github.com/QubitProducts/bamboo, but with fewer features, and not HAproxy specific. https://mesosphere.github.io/marathon/docs/event-bus.html


  • event coalescing is a bit wonky and fires on very first event
  • Write test suite
  • Setup travis-ci builds