Skip to content
A full ecosystem that is built around kafka powered by golang
Go Shell Makefile Smarty Perl Java Other
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
cmd
ctx
diagnostics/agent
mpool
registry
sla
telemetry
zk
.dockerignore
.gitignore
.travis.yml
CODE_OF_CONDUCT.md
Changelog.md
Dockerfile
LICENSE.md
Makefile
README.md
TODO.md
build.sh
manifest.json
ver.go

README.md

gafka Build Status

                   __   _            
                  / _| | |           
   __ _    __ _  | |_  | | __   __ _ 
  / _` |  / _` | |  _| | |/ /  / _` |
 | (_| | | (_| | | |   |   <  | (_| |
  \__, |  \__,_| |_|   |_|\_\  \__,_|
   __/ |                             
  |___/                              

A full ecosystem for kafka/redis/PubSub/ElasticSearch/Zookeeper/haproxy.

Hope it can help you.

Components

  • ehaproxy

    Elastic haproxy that sits in front of kateway.

  • kateway

    A fully-managed real-time secure and reliable RESTful Cloud Pub/Sub streaming message/job service.

  • actord

    kateway job scheduler and webhook dispatcher.

  • gk

    Unified multi-datacenter multi-cluster kafka swiss-knife management console.

  • zk

    A handy zookeeper CLI that supports recursive operation without any dependency.

  • es

    ElasticSearch console.

  • kguard

    Kafka clusters body guard that emits health info to InfluxDB and exports key warnings to zabbix for alerting.

Install

export PATH=$PATH:$GOPATH/bin

#========================================
# install go-bindata and go annotations
#========================================
go install github.com/jteeuwen/go-bindata/go-bindata
go install github.com/funkygao/goannotation

#========================================
# install gafka
#========================================
go get github.com/funkygao/gafka
cd $GOPATH/src/github.com/funkygao/gafka
./build.sh -a # build all components

#========================================
# try the gafka command 'gk'
#========================================
gk -h

Status

Currently gafka manages:

  • 4 data centers
  • 50+ kafka clusters
  • 100+ kafka brokers
  • 500+ kafka topics
  • 2000+ kafka partitions
  • 10Billion messages per day
  • peak load
    • 1Million message per second
    • 8TB transfered per hour
  • 5TB redis
You can’t perform that action at this time.