Kafka integration

[jamilbk]

Backend component of real-time alerts

[jamilbk]

The way this could work is as follows:

1. Register an ES JSON query to a mock devo http api
2. Kafka topic will be id or name of this query
3. Expose a websockets endpoint through Node referencing the kafka topic

So the Kibana will consume alerts from this endpoint with a socket.io connection to something like http://localhost:5000/search?realtime=true&query=foobar. The query parameter will uniquely reference which kafka topic to stream alerts from. We can probably persistent this in Kafka itself, so when a dashboard is created, the query is converted to Elasticsearch JSON on the fly, a kafka topic is created or used (if exists), then the websockets endpoint is created and the Kibana will be able to stream results from it.

[rammie]

Stale.