API gateway example

[peterbourgon]

This is an extension of #91.

My thinking has evolved a bit on this after watching Chris Richardson speak about his work (http://microservices.io/). In particular my understanding of what an API gateway is has shifted, and I think there is definitely a role for Go kit there. I find it pretty straightforward how to create such a thing:

• Use transport/foo.Clients to establish endpoint.Endpoints for all services in your infra (backends)
• Use regular net/http or whatever to bind /api/foosvc/{param} handlers to those endpoints (frontend)

Let's add an example/apigateway that gateways to an example/profilesvc, example/addsvc, and example/stringsvc. Assume:

• frontend as plain net/http + gorilla/mux
• Consul for service discovery
• gRPC for communication with addsvc
• transport/http for communication with profilesvc and stringsvc

[basvanbeek]

Step 2 of this could be adding Zipkin tracing to showcase tracing across these connected services and across different transports... that would be a little more challenging but I might be able to contribute after the above piece has been taken care of

[marshauf]

I am new to go-kit and I am working on this to get to know the library.

[peterbourgon]

@basvanbeek Sounds good to me. A docker-compose-style infra-in-a-box, with those Go kit services + Consul + Zipkin + Prometheus would make a compelling demo!

@marshauf Great! If you'd like, feel free to make a work-in-progress style PR, and ask questions as you go. It's the easiest way to learn :)

[marshauf]

@peterbourgon I am discovering the services on-demand through http requests. Should the services be discovered on startup or demand? I am also not sure how to handle http/json on the frontend and a grpc service like addsvc, since the data requires a conversion. The simple solution would be to import the generated protobuf package and use jsonpb to convert the json data to protobuf messages. Which I don't like, because it would require recompiling the apigateway when a protobuf message changes. Any thoughts?

[peterbourgon]

@marshauf re: service discovery, definitely defined on startup. I'm still pretty firmly convinced an API gateway that does dynamic service discovery and route wiring is an antipattern.

re: transport conversion, since you're doing predefined service wiring, you know what the various endpoints are & what their request/response types should be, so you're free to deterministically unmarshal the specific requests from e.g. HTTP+JSON and re-marshal them to e.g. protobuf. I can't think of a way to build an API gateway that supports this kind of transcoding without this knowledge. An API gateway without request transcoding is I think indistinguishable from a content-agnostic reverse proxy; not the domain of Go kit.

[peterbourgon]

Thanks to @marshauf's great contribution #203, this issue is now closed. Merci!