User friendly implementation of ideas behind "Kafka".
Download (change "darwin" to "linux" as needed):
curl -Os https://storage.googleapis.com/peakunicorn/bin/amd64/darwin/hbuf
chmod 0755 hbuf
Run:
./hbuf
Produce message:
curl localhost:8080/topics/foo -d'bar'
Consume message:
curl localhost:8080/topics/foo/next
Stats:
curl localhost:8080/stats
Get more info (this will show all available routes and info about them):
curl localhost:8080 | jq .
hbuf -h
built in tools:
- producer
- consumer
- load generator ("stress")
I see Kafka not as a message queue, but as an append-only file; a place where data (usually logs of some kind) gets dumped, and then read and re-read at leisure. A semi-permanent data store; a buffer. I find this hugely useful, especially if messages can be assigned unique IDs, which allows for idempotent consumption.