UDP-to-Websockets-to-ZMQ JSON data collector for Bootic, written in Go
Go Ruby
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
fanout
firehose
udp
.gitignore
README.md
client.rb
main.go

README.md

Data collector

This is a realtime events/statistics recording and distribution backend written in Go. The backend is designed to receive messages via UDP and publish them via Websockets and a ZMQ pub/sub socket:

Incoming events/stats are broadcast out to any listening websocket clients.

Start

$ go run main.go --zmqsocket=tcp://:6000 --udphost=host:5555 --wshost=host:5555 --ssehost=localhost:5556 --accesstoken=foobar

Options

Option Description example
zmqsocket ZMQ socket to publish events to. tcp://6000
udphost UDP host:port to listen for incoming events on someserver:5555
wshost host:port for Websockets endpoint somepublicserver.com:80
ssehost host:port for Server Sent Events endpoint somepublicserver:5555
accesstoken Access token to authenticate public endpoints "foobar123"

Consumers

Consumer programs can subscribe to all or some event types on the ZMQ socket. Ruby example:

require 'zmq'
require 'msgpack'

context = ZMQ::Context.new
chans   = %w(order pageview)
sub     = context.socket ZMQ::SUB

sub.connect "tcp://127.0.0.1:6000"

chans.each { |ch| sub.setsockopt ZMQ::SUBSCRIBE, ch }

puts "Listening to events"

while line = sub.recv
  chan,  msg = line.split ' ', 2
  event = MessagePack.unpack(msg)
  puts "##{chan}: #{event['data']['account']} tz #{event['data']['tz']}"
end