Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

boilerplate codegen first cut #75

Closed
wants to merge 82 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
05bdeb7
1
Jul 10, 2015
81f4fef
first (very rought) cut of go-kit boilerplate generation
Jul 10, 2015
f7ac8bd
typo
Jul 10, 2015
70dcbf1
fixed a problem with multiple functions in an single interface
Jul 10, 2015
8ef2f10
readme: minor cleanup
Jul 11, 2015
e0392cf
proper error handling
Jul 11, 2015
76f4175
endpoints
Jul 13, 2015
c4e9a01
clients
Jul 13, 2015
6fa0b42
Readme, fixed paths
Jul 13, 2015
10bb207
removed old experimantal file
Jul 13, 2015
6cbb095
removed unused blueprint
Jul 13, 2015
466a2e7
removed commented out stuff
Jul 13, 2015
ddf4de4
fixed a premature return in endpoint code
Jul 13, 2015
4df96fb
use struct instead of map in endpoints
Jul 13, 2015
e0d3a63
generated clinet code
Jul 14, 2015
4c01e71
added an example that builds (addsvc2)
Jul 14, 2015
ffd36d0
-binary
Jul 14, 2015
800c67c
sample client
Jul 14, 2015
060f348
README: updates and rationales
peterbourgon Jun 14, 2015
d6e519f
Update main README
peterbourgon Jun 14, 2015
90dee1f
Uniform 'Go kit'
peterbourgon Jun 14, 2015
f3b3571
loadbalancer: do not expose sync.Mutex as public field
rogpeppe Jul 10, 2015
3f58641
Document up an example of how to wire up a statsd connection.
joho Jul 9, 2015
c08cb9a
log: first cut of new API, incomplete
peterbourgon Jul 11, 2015
f988c7b
Rename withLogger to Context and export With method.
ChrisHines Jul 12, 2015
136de4e
Add Context example.
ChrisHines Jul 13, 2015
26d5089
Rename log.With to log.NewContext and remove keyvals vargs.
ChrisHines Jul 13, 2015
0644355
Fix spelling in comment.
ChrisHines Jul 13, 2015
26e9b7f
Move leveled logging into its own package and update API to improve c…
ChrisHines Jul 14, 2015
bd48c15
log/levels: clean up some comments and var names
peterbourgon Jul 14, 2015
e7ebf25
log2: rm
peterbourgon Jul 14, 2015
6361292
log: fix some comments
peterbourgon Jul 14, 2015
01364cb
Combine config and Levels structs.
ChrisHines Jul 14, 2015
6552a82
Document implementation tradeoff and reason for choosing to favor a c…
ChrisHines Jul 15, 2015
6cbc7fa
Remove package-global Log for parameterized logger
peterbourgon Jul 21, 2015
962f9ae
Drop, rather than accumulate, failed batches
peterbourgon Jul 21, 2015
3bb6b71
Add support for a fixed sampling rate
peterbourgon Jul 21, 2015
68faf95
log/levels: improve test coverage
peterbourgon Jul 21, 2015
dfef1ac
metrics/expvar: improve test coverage
peterbourgon Jul 21, 2015
822f940
metrics: improve test coverage
peterbourgon Jul 21, 2015
042cf04
Fix bad merge
peterbourgon Jul 21, 2015
b5d567d
fixing reference to mysvcPublisher (fooPublisher)
andrewwatson Jul 28, 2015
074c735
README: link to GopherCon video
peterbourgon Jul 28, 2015
fa748d6
Split out loadbalancer components to packages
peterbourgon Aug 1, 2015
c2842d6
Big re-org
peterbourgon Aug 2, 2015
0ccdaa2
Add README and Retry component
peterbourgon Aug 2, 2015
decfaa6
More user-friendly behavior in DNSSRVPublisher
peterbourgon Aug 3, 2015
ce870e1
Fix typo in README
peterbourgon Aug 3, 2015
e4fa5f9
s/lift/makeEndpoints/g
peterbourgon Aug 4, 2015
f7aa49f
Remove redundant and wrongly named test.
ChrisHines Aug 7, 2015
6105757
Append ErrMissingValue to odd length keyvals rather than panic.
ChrisHines Aug 7, 2015
8793c00
Recover from panics caused by dereferencing a nil pointer to a value …
ChrisHines Aug 7, 2015
5dc7b1e
Code review updates.
ChrisHines Aug 7, 2015
0c5f674
fixed a typo in the Individual components section
Aug 9, 2015
addae31
circuitbreaker: improve test reporting
peterbourgon Aug 15, 2015
fce5487
circuitbreaker: temp fix for new hystrix behavior
peterbourgon Aug 15, 2015
a9aba50
Adaptation of @shore's proposal from #95
peterbourgon Aug 14, 2015
f62003b
Fix resource link for commit messages
xla Aug 16, 2015
17810ba
Fixed example code for metrics
wari Aug 19, 2015
b25f286
README: update language re: dependencies/vendoring
peterbourgon Aug 21, 2015
09c4463
Use NopLogger in tests.
ChrisHines Aug 24, 2015
7161daa
Take care that log.Context.Log has a consistent function stack depth …
ChrisHines Aug 24, 2015
fa79ec4
Completely rework examples and documentation.
peterbourgon Aug 11, 2015
20c3932
README: update
peterbourgon Aug 25, 2015
9aff2f7
examples/README: update
peterbourgon Aug 25, 2015
6303863
examples/README: update
peterbourgon Aug 25, 2015
12b75df
Update examples README.md
Aug 25, 2015
dd036fc
update log README
sbl Aug 25, 2015
0b25b11
transport/http: {Encode,Decode}{Request,Response}Func
peterbourgon Aug 27, 2015
daf995b
transport/http: bonus: ErrorEncoder
peterbourgon Aug 27, 2015
8ddc828
Attempt to get Coveralls coverage via Circle
peterbourgon Aug 27, 2015
df4968d
Add coverage badge to README
peterbourgon Aug 27, 2015
125da47
It's circle.yml, not .circle.yml
peterbourgon Aug 27, 2015
4a30fe1
Try to fix Circle
peterbourgon Aug 27, 2015
3861489
Try to fix Circle
peterbourgon Aug 27, 2015
7cf2b1b
Give up on Circle, try Drone
peterbourgon Aug 27, 2015
dcddbee
coverage.bash: set -e only, no -x
peterbourgon Aug 27, 2015
c37aeb7
Fix errors in coverage.bash
peterbourgon Aug 27, 2015
88fbc30
.travis.yml: test on go1.5
peterbourgon Aug 27, 2015
c00e4f8
transport/http: minor improvements
peterbourgon Aug 28, 2015
ab1ecc2
endpoints
Jul 13, 2015
66e6752
post-merge
Aug 30, 2015
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
addsvc/addsvc
addsvc/client/addcli/addcli
cover.out
examples/addsvc/addsvc
examples/addsvc/client/addcli/addcli
examples/stringsvc1/stringsvc1
examples/stringsvc2/stringsvc2
examples/stringsvc3/stringsvc3
gover.coverprofile

# Compiled Object files, Static and Dynamic libs (Shared Objects)
*.o
Expand Down
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@ language: go

go:
- 1.4.2
- 1.5
- tip


2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,5 +18,5 @@ Before submitting major changes, here are a few guidelines to follow:
[issues]: https://github.com/go-kit/kit/issues
[prs]: https://github.com/go-kit/kit/pulls
[squash]: http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html
[message]: https://github.com/go-kit/kit/issues
[message]: http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html

182 changes: 151 additions & 31 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,66 +1,186 @@
# gokit [![Circle CI](https://circleci.com/gh/go-kit/kit.svg?style=svg)](https://circleci.com/gh/go-kit/kit) [![Drone.io](https://drone.io/github.com/go-kit/kit/status.png)](https://drone.io/github.com/go-kit/kit/latest) [![Travis CI](https://travis-ci.org/go-kit/kit.svg?branch=master)](https://travis-ci.org/go-kit/kit) [![GoDoc](https://godoc.org/github.com/go-kit/kit?status.svg)](https://godoc.org/github.com/go-kit/kit)
# Go kit [![Circle CI](https://circleci.com/gh/go-kit/kit.svg?style=svg)](https://circleci.com/gh/go-kit/kit) [![Drone.io](https://drone.io/github.com/go-kit/kit/status.png)](https://drone.io/github.com/go-kit/kit/latest) [![Travis CI](https://travis-ci.org/go-kit/kit.svg?branch=master)](https://travis-ci.org/go-kit/kit) [![GoDoc](https://godoc.org/github.com/go-kit/kit?status.svg)](https://godoc.org/github.com/go-kit/kit) [![Coverage Status](https://coveralls.io/repos/go-kit/kit/badge.svg?branch=master&service=github)](https://coveralls.io/github/go-kit/kit?branch=master)

**Go kit** is a **distributed programming toolkit** for microservices in the modern enterprise. We want to make Go a viable choice for application (business-logic) software in large organizations.
**Go kit** is a **distributed programming toolkit** for building microservices
in large organizations. We solve common problems in distributed systems, so
you can focus on your business logic.

- Mailing list: [go-kit](https://groups.google.com/forum/#!forum/go-kit)
- Slack: [gophers.slack.com](https://gophers.slack.com) **#go-kit** ([invite](http://bit.ly/go-slack-signup))

## Documentation

### Examples

Perhaps the best way to understand Go kit is to follow along as we build an
[example service][examples] from first principles. This can serve as a
blueprint for your own new service, or demonstrate how to adapt your existing
service to use Go kit components.

[examples]: https://github.com/go-kit/kit/tree/master/examples

### Endpoint

Go kit primarily deals in the RPC messaging pattern. We use an abstraction
called an **[endpoint][]** to model invidivual RPCs. An endpoint can be
implemented by a server, and called by a client. It's the fundamental building
block of many Go kit components.

[endpoint]: https://github.com/go-kit/kit/tree/master/endpoint/endpoint.go

#### Circuit breaker

The [circuitbreaker package][circuitbreaker] provides endpoint adapters to
several popular circuit breaker libraries. Circuit breakers prevent thundering
herds, and improve resiliency against intermittent errors. Every client-side
endpoint should be wrapped in a circuit breaker.

[circuitbreaker]: https://github.com/go-kit/kit/tree/master/circuitbreaker

#### Rate limiter

The [ratelimit package][ratelimit] provides endpoint adapters to rate limiting
packages. Rate limiters are equally applicable to both server- and client-side
endpoints. Use rate limiters to enforce upper thresholds on incoming or
outgoing request throughput.

[ratelimit]: https://github.com/go-kit/kit/tree/master/ratelimit

### Transport

The [transport package][transport] provides helpers to bind endpoints to
specific serialization mechanisms. At the moment, Go kit just provides helpers
for simple JSON over HTTP. If your organization uses a fully-featured
transport, bindings are typically provided by the Go library for the
transport, and there's not much for Go kit to do. In those cases, see the
examples to understand how to write adapters for your endpoints. For now, see
the [addsvc][addsvc] to understand how transport bindings work. We'll soon
have specific examples for Thrift, gRPC, net/rpc, and JSON over HTTP. Avro and
JSON/RPC support is planned.

[transport]: https://github.com/go-kit/kit/tree/master/transport
[addsvc]: https://github.com/go-kit/kit/tree/master/examples/addsvc

### Logging

Services produce logs to be consumed later, either by humans or machines.
Humans might be interested in debugging errors, or tracing specific requests.
Machines might be interested in counting interesting events, or aggregating
information for offline processing. In both cases, it's important that the log
messages be structured and actionable. Go kit's [log package][log] is designed
to encourage both of these best practices.

[log]: https://github.com/go-kit/kit/tree/master/log

### Metrics (Instrumentation)

Services can't be considered production-ready until they're thoroughly
instrumented with metrics that track counts, latency, health, and other
periodic or per-request information. Go kit's [metrics package][metrics]
provides a robust common set of interfaces for instrumenting your service.
Bindings exist for common backends, from [expvar][] to [statsd][] to
[Prometheus][].

[metrics]: https://github.com/go-kit/kit/tree/master/metrics
[expvar]: http://golang.org/pkg/expvar
[statsd]: https://github.com/etsy/statsd
[Prometheus]: http://prometheus.io

### Request tracing

As your infrastructure grows, it becomes important to be able to trace a
request, as it travels through multiple services and back to the user. Go
kit's [tracing package][tracing] provides enhancements for your endpoints and
transport bindings to capture information about requests and emit them to
request tracing systems. (Currently, [Zipkin][] is supported; [Appdash][]
support is planned.)

[tracing]: https://github.com/go-kit/kit/tree/master/tracing
[Zipkin]: https://github.com/twitter/zipkin
[Appdash]: https://github.com/sourcegraph/appdash

### Service discovery and load balancing

If your service calls another service, it needs to know how to find it, and
should intelligently spread its load among those discovered instances. Go
kit's [loadbalancer package][loadbalancer] provides client-side endpoint
middleware to solve that problem, whether your organization uses static hosts
or IPs, [DNS SRV records][dnssrv], Consul, etcd, or Zookeeper. And if you use
a custom system, it's very easy to write your own [Publisher][] and use Go
kit's load balancing strategies. (Currently, static hosts and DNS SRV are
supported; Consul, etcd, and Zookeeper support is planned.)

[loadbalancer]: https://github.com/go-kit/kit/tree/master/loadbalancer
[dnssrv]: https://github.com/go-kit/kit/tree/master/loadbalancer/dnssrv
[Publisher]: https://github.com/go-kit/kit/tree/master/loadbalancer/publisher.go

## Motivation

See [the motivating blog post](http://peter.bourgon.org/go-kit) and [the video of the talk](https://www.youtube.com/watch?v=iFR_7AKkJFU).
Go has emerged as the language of the server, but it remains underrepresented
in large, consumer-focused tech companies like Facebook, Twitter, Netflix, and
SoundCloud. These organizations have largely adopted JVM-based stacks for
their business logic, owing in large part to libraries and ecosystems that
directly support their microservice architectures.

To reach its next level of success, Go needs more than simple primitives and
idioms. It needs a comprehensive toolkit, for coherent distributed programming
in the large. Go kit is a set of packages and best practices, leveraging years
of production experience, and providing a comprehensive, robust, and trustable
platform for organizations of any size.

In short, Go kit brings Go to the modern enterprise.

For more details, see
[the motivating blog post](http://peter.bourgon.org/go-kit) and
[the video of the talk](https://www.youtube.com/watch?v=iFR_7AKkJFU).
See also the
[Go kit talk at GopherCon 2015](https://www.youtube.com/watch?v=1AjaZi4QuGo).

## Goals

- Operate in a heterogeneous SOA — expect to interact with mostly non-Go-kit services
- RPC as the primary messaging pattern
- Pluggable serialization and transport — not just JSON over HTTP
- Zipkin-compatible request tracing
- Operate within existing infrastructures — no mandates for specific tools or technologies

## Non-goals

- Supporting messaging patterns other than RPC (in the initial release) — pub/sub, CQRS, etc.
- Supporting messaging patterns other than RPC (in the initial release) — MPI, pub/sub, CQRS, etc.
- Re-implementing functionality that can be provided by wrapping existing packages
- Having opinions on deployment, orchestration, process supervision, etc.
- Having opinions on configuration passing — flags, env vars, files, etc.

## Component status

- [API stability](https://github.com/go-kit/kit/blob/master/rfc/rfc007-api-stability.md) — **adopted**
- [`package log`](https://github.com/go-kit/kit/tree/master/log) — **implemented**
- [`package metrics`](https://github.com/go-kit/kit/tree/master/metrics) — **implemented**
- [`package endpoint`](https://github.com/go-kit/kit/tree/master/endpoint) — **implemented**
- [`package transport`](https://github.com/go-kit/kit/tree/master/transport) — **implemented**
- [`package circuitbreaker`](https://github.com/go-kit/kit/tree/master/circuitbreaker) — **implemented**
- [`package loadbalancer`](https://github.com/go-kit/kit/tree/master/loadbalancer) — **implemented**
- [`package ratelimit`](https://github.com/go-kit/kit/tree/master/ratelimit) — **implemented**
- [`package tracing`](https://github.com/go-kit/kit/tree/master/tracing) — prototyping
- Client patterns — prototyping
- Service discovery — pending
- Example [addsvc](https://github.com/go-kit/kit/tree/master/addsvc) — **implemented**

## Contributing

Please see [CONTRIBUTING.md]. Thank you, [contributors]!

[CONTRIBUTING.md]: /CONTRIBUTING.md
[contributors]: https://github.com/go-kit/kit/graphs/contributors

### Dependency management
## Dependency management

Go kit is a library, designed to be imported into a binary package. Vendoring
is currently the best way for binary package authors to ensure reliable,
reproducible builds. Therefore, we strongly recommend our users use vendoring
for all of their dependencies, including Go kit. To avoid compatibility and
availability issues, Go kit doesn't vendor its own dependencies, and doesn't
recommend use of third-party import proxies.

Users who import Go kit into their `package main` are responsible to organize
and maintain all of their dependencies to ensure code compatibility and build
reproducibility. Gokit makes no direct use of dependency management tools like
[Godep](https://github.com/tools/godep).
There are several tools which make vendoring easier, including [gb][],
[govendor][], and [Godep][]. And Go kit uses a variety of continuous
integration providers to find and fix compatibility problems as soon as they
occur.

We will use a variety of continuous integration providers to find and fix
compatibility problems as soon as they occur.
[gb]: http://getgb.io
[govendor]: https://github.com/kardianos/govendor
[Godep]: https://github.com/tools/godep

### API stability policy
## API stability policy

The Go kit project depends on code maintained by others.
This includes the Go standard library and sub-repositories and other external libraries.
The Go language and standard library provide stability guarantees, but the other external libraries typically do not.
[The API Stability RFC](https://github.com/go-kit/kit/tree/master/rfc/rfc007-api-stability.md)
proposes a standard policy for package authors to advertise API stability.
[The API Stability RFC](https://github.com/go-kit/kit/tree/master/rfc/rfc007-api-stability.md)
proposes a standard policy for package authors to advertise API stability.
The Go kit project prefers to depend on code that abides the API stability policy.

## Related projects
Expand All @@ -83,7 +203,7 @@ Projects with a ★ have had particular influence on Go kit's design.
- [FogCreek/logging](https://github.com/FogCreek/logging), a tagged style of logging
- [grpc/grpc-go](https://github.com/grpc/grpc-go), HTTP/2 based RPC
- [inconshreveable/log15](https://github.com/inconshreveable/log15), simple, powerful logging for Go ★
- [mailgun/vulcand](https://github.com/mailgun/vulcand), prorammatic load balancer backed by etcd
- [mailgun/vulcand](https://github.com/mailgun/vulcand), programmatic load balancer backed by etcd
- [mattheath/phosphor](https://github.com/mattheath/phosphor), distributed system tracing
- [pivotal-golang/lager](https://github.com/pivotal-golang/lager), an opinionated logging library
- [rubyist/circuitbreaker](https://github.com/rubyist/circuitbreaker), circuit breaker library
Expand Down
11 changes: 7 additions & 4 deletions addsvc/README.md → addsvc2/README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,21 @@
# addsvc
# addsvc2

addsvc is an example service, used to illustrate the mechanics of gokit.
addsvc2 is an example service, used to illustrate the mechanics of gokit.
It exposes simple functionality on a variety of transports and endpoints.

## Server

To build and run addsvc,
To build and run addsvc2,

```
$ go install
$ addsvc
$ addsvc2
```

## Client
TODO: make it work with codegenrator.

Everything below is not working.

addsvc comes with an example client, [addcli][].

Expand Down
9 changes: 9 additions & 0 deletions addsvc2/add/add.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
package add

import "golang.org/x/net/context"

//go:generate gokit-gen -package github.com/sasha-s/kit/addsvc2/add -type Adder -w -binding=rpc,http

type Adder interface {
Add(ctx context.Context, a int64, b int64) (v int64)
}
34 changes: 34 additions & 0 deletions addsvc2/add/adder_add_http_binding.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
// Do not edit! Generated by gokit-generate
package add

import (
"encoding/json"
"net/http"

"golang.org/x/net/context"

"github.com/go-kit/kit/endpoint"
httptransport "github.com/go-kit/kit/transport/http"
)

func MakeAdderAddHTTPBinding(ctx context.Context, e endpoint.Endpoint, before []httptransport.RequestFunc, after []httptransport.ResponseFunc) http.Handler {
decode := func(r *http.Request) (interface{}, error) {
defer r.Body.Close()
var request AdderAddRequest
if err := json.NewDecoder(r.Body).Decode(&request); err != nil {
return nil, err
}
return request, nil
}
encode := func(w http.ResponseWriter, response interface{}) error {
return json.NewEncoder(w).Encode(response)
}
return httptransport.Server{
Context: ctx,
Endpoint: e,
DecodeRequestFunc: decode,
EncodeResponseFunc: encode,
Before: before,
After: append([]httptransport.ResponseFunc{httptransport.SetContentType("application/json; charset=utf-8")}, after...),
}
}
61 changes: 61 additions & 0 deletions addsvc2/add/adder_add_http_client.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
// Do not edit! Generated by gokit-generate
package add

import (
"bytes"
"encoding/json"
"net/http"

"golang.org/x/net/context"

"github.com/go-kit/kit/endpoint"
httptransport "github.com/go-kit/kit/transport/http"
)

// NewClient takes a URL that should point to an instance of an addsvc. It
// returns an endpoint that makes a request to that URL.
func NewAdderAddHTTPClient(method, url string, before ...httptransport.RequestFunc) endpoint.Endpoint {
return func(ctx0 context.Context, request interface{}) (interface{}, error) {
var (
ctx, cancel = context.WithCancel(ctx0)
errs = make(chan error, 1)
responses = make(chan interface{}, 1)
)
defer cancel()
go func() {
var buf bytes.Buffer
if err := json.NewEncoder(&buf).Encode(request); err != nil {
errs <- err
return
}
req, err := http.NewRequest(method, url, &buf)
if err != nil {
errs <- err
return
}
for _, f := range before {
ctx = f(ctx, req)
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
errs <- err
return
}
defer resp.Body.Close()
var response AdderAddResponse
if err := json.NewDecoder(resp.Body).Decode(&response); err != nil {
errs <- err
return
}
responses <- response
}()
select {
case <-ctx.Done():
return nil, context.DeadlineExceeded
case err := <-errs:
return nil, err
case response := <-responses:
return response, nil
}
}
}
Loading