JOCKO

Kafka/distributed commit log service in Go.

Goals of this project:

• Implement Kafka in Go
• Protocol compatible with Kafka so Kafka clients and services work with Jocko
• Make operating simpler
• Distribute a single binary
• Use Serf for discovery, Raft for consensus (and remove the need to run ZooKeeper)
• Smarter configuration settings
  • Able to use percentages of disk space for retention policies rather than only bytes and time kept
  • Handling size configs when you change the number of partitions or add topics
• Learn a lot and have fun

TODO

• Producing
• Fetching
• Partition consensus and distribution
• Protocol
  • Produce
  • Fetch
  • Metadata
  • Create Topics
  • Delete Topics
  • Consumer group
• Discovery
• Replication [current task]

Reading

• How Jocko's built-in service discovery and consensus works
• How Jocko's (and Kafka's) storage internals work

License

MIT

---

• travisjeffery.com
• GitHub @travisjeffery
• Twitter @travisjeffery
• Medium @travisjeffery