Go kit

Go kit is a distributed programming toolkit for microservices in the modern enterprise. We want to make Go a viable choice for application (business-logic) software in large organizations.

• Mailing list: go-kit
• Slack: gophers.slack.com #go-kit (invite)

Motivation

Go has emerged as the language of the server, but it remains underrepresented in large, consumer-focused tech companies like Facebook, Twitter, Netflix, and SoundCloud. These organizations have largely adopted JVM-based stacks for their business logic, owing in large part to libraries and ecosystems that directly support their microservice architectures.

To reach its next level of success, Go needs more than simple primitives and idioms. It needs a comprehensive toolkit, for coherent distributed programming in the large. Go kit is a set of packages and best practices, leveraging years of production experience, and providing a comprehensive, robust, and trustable platform for organizations of any size.

In short, Go kit brings Go to the modern enterprise.

For more details, see the motivating blog post and the video of the talk. See also the Go kit talk at GopherCon 2015.

Goals

• Operate in a heterogeneous SOA — expect to interact with mostly non-Go-kit services
• RPC as the primary messaging pattern
• Pluggable serialization and transport — not just JSON over HTTP
• Operate within existing infrastructures — no mandates for specific tools or technologies

Non-goals

• Supporting messaging patterns other than RPC (in the initial release) — MPI, pub/sub, CQRS, etc.
• Re-implementing functionality that can be provided by wrapping existing packages
• Having opinions on deployment, orchestration, process supervision, etc.
• Having opinions on configuration passing — flags, env vars, files, etc.

Component status

• API stability — adopted
• package log — implemented
• package metrics — implemented
• package endpoint — implemented
• package transport — implemented
• package circuitbreaker — implemented
• package loadbalancer — implemented
• package ratelimit — implemented
• package tracing — prototyping
• Client patterns — prototyping
• Service discovery — pending
• Example addsvc — implemented

Dependency management

Users who import Go kit into their package main are responsible to organize and maintain all of their dependencies to ensure code compatibility and build reproducibility. Go kit makes no direct use of dependency management tools like Godep.

We will use a variety of continuous integration providers to find and fix compatibility problems as soon as they occur.

Contributing

Please see CONTRIBUTING.md. Thank you, contributors!

API stability policy

The Go kit project depends on code maintained by others. This includes the Go standard library and sub-repositories and other external libraries. The Go language and standard library provide stability guarantees, but the other external libraries typically do not. The API Stability RFC proposes a standard policy for package authors to advertise API stability. The Go kit project prefers to depend on code that abides the API stability policy.

Related projects

Projects with a ★ have had particular influence on Go kit's design.

Service frameworks

• go-micro, a microservices client/server library ★
• gocircuit, dynamic cloud orchestration
• gotalk, async peer communication protocol & library
• Kite, a micro-service framework

Individual components

• afex/hystrix-go, client-side latency and fault tolerance library
• armon/go-metrics, library for exporting performance and runtime metrics to external metrics systems
• codahale/lunk, structured logging in the style of Google's Dapper or Twitter's Zipkin
• eapache/go-resiliency, resiliency patterns
• FogCreek/logging, a tagged style of logging
• grpc/grpc-go, HTTP/2 based RPC
• inconshreveable/log15, simple, powerful logging for Go ★
• mailgun/vulcand, programmatic load balancer backed by etcd
• mattheath/phosphor, distributed system tracing
• pivotal-golang/lager, an opinionated logging library
• rubyist/circuitbreaker, circuit breaker library
• Sirupsen/logrus, structured, pluggable logging for Go ★
• sourcegraph/appdash, application tracing system based on Google's Dapper
• spacemonkeygo/monitor, data collection, monitoring, instrumentation, and Zipkin client library
• streadway/handy, net/http handler filters
• vitess/rpcplus, package rpc + context.Context
• gdamore/mangos, nanomsg implementation in pure Go

Web frameworks

• Beego
• Gin
• Goji
• Gorilla
• Martini
• Negroni
• Revel (considered harmful)

Additional reading

• Architecting for the Cloud — Netflix
• Dapper, a Large-Scale Distributed Systems Tracing Infrastructure — Google
• Your Server as a Function (PDF) — Twitter