WORK IN PROGRESS
Reusable template for building REST Web Services in Golang. Uses gorilla/mux as a router/dispatcher and Negroni as a middleware handler. Tested against Go 1.4 and 1.5.
After writing many REST APIs with Java Dropwizard, Node.js/Express and Go, I wanted to distil my lessons learned into a reusable template for writing REST APIs, in the Go language (my favourite).
It's mainly for myself. I don't want to keep reinventing the wheel and just want to get the foundation of my REST API 'ready to go' so I can focus on the business logic and integration with other systems and data stores.
Just to be clear: this is not a framework, library, package or anything like that. This tries to use a couple of very good Go packages and libraries that I like and cobbled them together.
The main ones are:
- gorilla/mux for routing
- negroni as a middleware handler
- testify for writing easier test assertions
- godep for dependency management
- render for HTTP response rendering
Whilst working on this I've tried to write up as much as my thought process as possible. Everything from the design of the API and routes, some details of the Go code like JSON formatting in structs and my thoughts on testing. However, if you feel that there is something missing, send a PR, raise an issue or contact me on twitter @leeprovoost.
If you're new to programming in Go, I would highly recommend you to read the following two resources:
You can work your way through those in two to three days. I wouldn't advise buying any books right now. I unfortunately did and I have yet to open them. If you really want to get some books, there is a github repo that tries to list the main ones.
I've tried many different editors and it seems like it's a common Go newbie frustration. Coming from Java, I've tried "proper" IDEs like LiteIDE and IntelliJ with the golang plugin but never really fell in love with those.
I've used Sublime Text with the GoSublime plugin for a long time, but have now settled on Atom with the go-plus plugin and that is working very well. It provides the "lightness" of an editor but with some of the power you'd expect from an IDE. It has code completion, linting, formatting and even shows you what parts of the code are covered by tests.
The choice of an editor is a personal one, so the only advice I can give you is to try different ones. It looks like currently the following seem to be the most mature:
- Sublime Text 3 with GoSublime. I've been made aware that there is a more recent Go plugin for Sublime Text called GoTools
- Atom with go-plus. There is a good article describing a full Go setup here.
- vim with vim-go
- LiteIDE
go run main.go works fine if you have a single file you're working on, but once you have multiple files you'll have to start using the proper go build tool and run the compiled executable.
go build && ./go-rest-api-template
The app will bind itself by default to port 3009. If you want to change it (e.g. bind it to the default http port 80), then use a command line flag. Same for the location of the fixtures.json model.
go build && ./go-rest-api-template -port=80 -fixtures=/tmp/fixtures.json
Manually stopping and restarting your server can get quite annoying after a while, so let's set up a task runner that automatically restarts the server when it detects changes (similar to Grunt for the JavaScript / Node developers).
Install fresh:
go get github.com/pilu/fresh
And during development, you can now just type in the following command in your project root directory:
fresh
You should see following output if all goes well:
Loading settings from ./runner.conf
8:41:18 runner | InitFolders
8:41:18 runner | mkdir ./tmp
8:41:18 watcher | Watching .
8:41:18 watcher | Watching codedeploy-scripts
8:41:18 main | Waiting (loop 1)...
8:41:18 main | receiving first event /
8:41:18 main | sleeping for 600 milliseconds
8:41:19 main | flushing events
8:41:19 main | Started! (12 Goroutines)
8:41:19 main | remove tmp/go-rest-api-template.log: no such file or directory
8:41:19 build | Building...
8:41:20 runner | Running...
8:41:20 main | --------------------
8:41:20 main | Waiting (loop 2)...
8:41:20 app | Location of fixtures.json file: ./fixtures.json
8:41:20 app | Starting server on port: 3009
8:41:20 app | [negroni] listening on :3009
Fresh should work without any configuration, but to make it more explicit you can add a runner.conf file in your project root:
root: .
tmp_path: ./tmp
build_name: go-rest-api-template
build_log: go-rest-api-template.log
valid_ext: .go, .tpl, .tmpl, .html
build_delay: 600
colors: 1
log_color_main: cyan
log_color_build: yellow
log_color_runner: green
log_color_watcher: magenta
log_color_app:
As you can see, it creates a tmp directory in your project root and a log file. You can tell .gitignore to stop checking it into your git repository by adding the following lines in your .gitignore file:
# fresh
tmp
go-rest-api-template.log
Main server file (bootstrapping of http server and router):
main.go
Route handlers:
handlers.go
Data model descriptions:
passport.go
user.go
Mock database with operations:
database.go
Tests and test data:
database_test.go
fixtures.json
Configuration file for [fresh](go get github.com/pilu/fresh):
runner.conf
TO DO talk about the layers of the applications
TO DO
We are going to use a travel Passport for our example. I've chosen Id as the unique key for the passport because (in the UK), passport book numbers these days have a unique 9 character field length (e.g. 012345678). A passport belongs to a user and a user can have one or more passports.
type User struct {
Id int `json:"id"`
FirstName string `json:"firstName"`
LastName string `json:"lastName"`
DateOfBirth time.Time `json:"dateOfBirth"`
LocationOfBirth string `json:"locationOfBirth"`
}
type Passport struct {
Id string `json:"id"`
DateOfIssue time.Time `json:"dateOfIssue"`
DateOfExpiry time.Time `json:"dateOfExpiry"`
Authority string `json:"authority"`
UserId int `json:"userId"`
}
The first time you create a struct, you may not be aware that uppercasing and lowercasing your field names have a meaning in Go. It's similar to public and private members in Java. Uppercase = public, lowercase = private. There are some good discussions on Stackoverflow about this. The gist is that field names that start with a lowercase letter will not be visible to json.Marshal.
You may not want to expose your data to the consumer of your web service in this format, so you can override the way your fields are marshalled by adding json:"firstName" to each field with the desired name. I admit that in the past I had the habit of using underscores for my json field names, e.g. first_name. However after reading this excellent presentation on API design, I got reminded that the JS in JSON stands for JavaScript and in the JavaScript world, it's common to use camelCasing so the preffered way of writing the same fieldname would be: firstName.
Note the use of time.Time for the dates instead of using a standard string type. We'll discuss the pain of marshalling and unmarshalling of JSON dates a bit later.
I wanted to create a template REST API that didn't depend on a database, so started with a simple in-memory database that we can work with. The good thing is that this will be the start of a so-called data access layer that abstracts away the underlying data store. We can achieve that by starting with creating an interface (which is a good practice in Go anyway). Note the use of the -er at the end of the interface name, as per Go convention.
type DataStorer interface {
List() (map[string]User, error)
Get(i int) (User, error)
Add(u User) (User, error)
Update(u User) (User, error)
Delete(i int) (bool, error)
}
This allows us to define a set of operations on the data as a contract, without people having to worry about the actual implementation of how the data is stored and accessed. I've added the basic operations to list, retrieve, create, update and delete data, so the standard CRUD-style operations (accepting that CRUD has some subtle differences with REST).
Let's have a look at the type signature of the Get operation:
Get(i int) (User, error)
What this tells us is that it is expecting an integer as an argument (which will be the User id in our case), and returns a pair of values: a user object and an error object. Returning pairs of values is a nice Go feature and is often used to return information about errors.
An example of how this could be used is the following:
user, err := db.Get(uid)
if err == nil {
Render.JSON(w, http.StatusOK, user)
} else {
Render.JSON(w, http.StatusNotFound, err)
}
We check whether the error object is nil. If it is, then we return a HTTP 200 OK, if not then we return HTTP 404 NOT FOUND. Let's go into more detail when we talk about our API handlers.
Let's have a look at the actual mock in-memory database. We need to create a Database struct that will hold the data:
type Database struct {
UserList map[int]User
MaxUserID int
}
The UserList will hold a list of User structs and the MaxUserID holds the latest used integer. MaxUserID mimicks the behaviour of an autogenerated ID in conventional databases. In this case MaxUserID represents the highest used database ID.
We will now create a global database variable so that it's accessible across our whole API:
var db *Database
Which gets initalised in our main.go init function:
func init() {
// ...
list := make(map[int]User)
list[0] = jsonObject["users"][0]
list[1] = jsonObject["users"][1]
db = &Database{
UserList: list,
MaxUserID: 1,
}
}
This creates a map of User objects with an integer key, creates two items and then creates the Database object with the list and sets the new user ID to 1.
In order to make it a bit more useful, we will initialise it with some user objects. Luckily, we can make use of the init function that gets automatically called when you start the application. This init() function will be in our main.go file when you start up the server:
func init() {
// read JSON fixtures file
var jsonObject map[string][]User
file, err := ioutil.ReadFile("./fixtures.json")
if err != nil {
log.Fatalf("File error: %v\n", err)
}
err = json.Unmarshal(file, &jsonObject)
if err != nil {
log.Fatal(err)
}
// load data in database
list := make(map[int]User)
list[0] = jsonObject["users"][0]
list[1] = jsonObject["users"][1]
db = &Database{list, 1}
}
We are first going to load the data from a fixtures.json file:
{
"users": [
{
"dateOfBirth": "1985-12-31T00:00:00Z",
"firstName": "John",
"id": 0,
"lastName": "Doe",
"locationOfBirth": "London"
},
{
"dateOfBirth": "1992-01-01T00:00:00Z",
"firstName": "Jane",
"id": 1,
"lastName": "Doe",
"locationOfBirth": "Milton Keynes"
}
]
}
When we can't load the file, we will stop the bootstrapping of the application. This is taken care of by Go's log handler, which fires off a fatal error:
file, err := ioutil.ReadFile("./fixtures.json")
if err != nil {
log.Fatalf("File error: %v\n", err)
}
The fixtures.json file contains a JSON representation of a Go map where the key is a string (i.e. "users") and the map value is a string of User objects. In Go, this would be respresented as: map[string][]User. We load the fixtures file, marshal it into the type we just defined and then load it into our database.
The date string looks a bit odd. Why not just use 31-12-1985 or 1985-12-31? The first is discouraged altogether because that's an European way of writing dates and will cause confusion around the world. Not this particular example, but imagine you have 3-4-2015. Is it third of April or fourth of March? Unfortunately there isn't an "enforced standard" for dates in JSON, so I've tried to use one that is commonly used and also understood by Go's json.Marshaler and json.Unmarshaler to avoid that we have to write our own custom marshaler/unarshaler.
If you have a look at Go's time/format code then you'll see on line 54:
54 RFC3339 = "2006-01-02T15:04:05Z07:00"
That's the one we need. It has the following format:
year-month-day
T (for time)
hour:minutes:seconds
Z (for time zone)
offset from UTC
We now need to implement the various methods from our DataStore interface.
TO DO document implemented methods
Now that we have defined the data access layer, we need to translate that to a REST interface:
- Retrieve a list of all users:
GET /users-> TheGETjust refers to the HTTP action you would use. If you want to test this in the command line, then you can use curl:curl -X GET http://localhost:3009/usersorcurl -X POST http://localhost:3009/users - Retrieve the details of an individual user:
GET /users/{uid}-> {uid} allows us to create a variable, named uid, that we can use in our code. An example of this url would beGET /users/1 - Create a new user:
POST /users - Update a user:
PUT /users/{uid} - Delete a user:
DELETE /users/{uid}
We now need to do the same for handling passports. Don't forget that a passport belongs to a user, so to retrieve a list of all passports for a given user, we would use GET /users/{uid}/passports.
When we want to retrieve an specific passport, we don't need to prefix the route with /users/{uid} anymore because we know exactly which passport we want to retrieve. So, instead of GET /users/{uid}/passports/{pid}, we can just use GET /passports/{pid}.
Once you have the API design sorted, it's just a matter of creating the code that gets called when a specific route is hit. We implement those with Handlers.
router.HandleFunc("/users", makeHandler(env, ListUsersHandler)).Methods("GET")
router.HandleFunc("/users/{uid:[0-9]+}", makeHandler(env, GetUserHandler)).Methods("GET")
router.HandleFunc("/users", makeHandler(env, CreateUserHandler)).Methods("POST")
router.HandleFunc("/users/{uid:[0-9]+}", makeHandler(env, UpdateUserHandler)).Methods("PUT")
router.HandleFunc("/users/{uid:[0-9]+}", makeHandler(env, DeleteUserHandler)).Methods("DELETE")
router.HandleFunc("/users/{uid}/passports", makeHandler(env, PassportsHandler)).Methods("GET")
router.HandleFunc("/passports/{pid:[0-9]+}", makeHandler(env, PassportsHandler)).Methods("GET")
router.HandleFunc("/users/{uid}/passports", makeHandler(env, PassportsHandler)).Methods("POST")
router.HandleFunc("/passports/{pid:[0-9]+}", makeHandler(env, PassportsHandler)).Methods("PUT")
router.HandleFunc("/passports/{pid:[0-9]+}", makeHandler(env, PassportsHandler)).Methods("DELETE")
In order to make our code more robust, I've added pattern matching in the routes. This [0-9]+ pattern says that we only accept digits from 0 to 9 and we can have one or more digits. Everything else will most likely trigger an HTTP 404 Not Found status code being returned to the client.
Most Go code that show HandleFunc examples, will show something slightly different, something more like this:
router.HandleFunc("/users", ListUsersHandler).Methods("GET")
So Go's HandleFunc function takes two arguments, one string that defines the route and a second argument of the type http.HandleFunc(w http.ResponseWriter, r *http.Request). This works very well, but doesn't allow you to pass any extra data to your handlers. So for instance when you want to use the unrolled/render package, then you'd have to define in your main.go file a global variable like this: var Render *render.Render, then initialise that in your func main() so that your handlers can access this global variable later on. Using global variables in Go is not a good practice (there are exceptions like certain database drivers, but that's a different discussion).
So our initial handler function for returning a list of users was:
func ListUsersHandler(w http.ResponseWriter, req *http.Request) {
Render.JSON(w, http.StatusOK, db.List())
}
Where the Render variable is a global variable. We'd prefer to pass the Render variable to that function so will rewrite it to:
func ListUsersHandler(w http.ResponseWriter, req *http.Request, render Render) {
render.JSON(w, http.StatusOK, db.List())
}
Perfect. Or, is it? What if we want to pass more variables to the handler function? Like Metrics? Or some environment variables? We'd continuously have to change ALL our handlers and the type signature will become quite long and hard to maintain. An alternative is to create a server or an environment struct and use that as a container for our variables we want to pass around the system.
In our main.go we'd add:
type env struct {
Metrics *stats.Stats
Render *render.Render
}
And in our func main() function we initialise that struct:
func main() {
env := env{
Metrics: stats.New(),
Render: render.New(),
}
// ...
}
Our handler looks now like this:
func ListUsersHandler(w http.ResponseWriter, req *http.Request, env env) {
env.Render.JSON(w, http.StatusOK, db.List())
}
The only problem is that this handler's type signature is not http.ResponseWriter, *http.Request but http.ResponseWriter, *http.Request, env so Go's HandleFunc function will complain about this. That's why we are introducing a helper function makeHandler that takes our environment struct and our handlers with the special type signature and converts it to func(w http.ResponseWriter, r *http.Request):
func makeHandler(env env, fn func(http.ResponseWriter, *http.Request, env)) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
fn(w, r, env)
}
}
When you look at the overview of the handlers in main.go, you will notice a couple of special routes:
router.HandleFunc("/", makeHandler(env, HomeHandler))
router.HandleFunc("/healthcheck", makeHandler(env, HealthcheckHandler)).Methods("GET")
router.HandleFunc("/metrics", makeHandler(env, MetricsHandler)).Methods("GET")
When someone hits our API, without a specified route, then we can handle that with either a standard 404 (not found), or any other type of feedback.
We also want to set up a health check that monitoring tools like Sensu can call: GET /healthcheck. The health check route can return a 204 OK when the serivce is up and running, including some extra stats. A 204 means "Hey, I got your request, all is fine and I have nothing else to say". It essentially tells your client that there is no body content.
func HealthcheckHandler(w http.ResponseWriter, req *http.Request, env env) {
env.Render.Text(w, http.StatusNoContent, "")
}
This health check is very simple. It just checks whether the service is up and running, which can be useful in a build and deployment pipelines where you can check whether your newly deployed API is running (as part of a smoke test). More advanced health checks will also check whether it can reach the database, message queue or anything else you'd like to check. Trust me, your DevOps colleagues will be very grateful for this. (Don't forget to change your HTTP status code to 200 if you want to report on the various components that your health check is checking.)
We will skip the /metrics route for a second and keep that for the end of the article.
Let's have a look at interacting with our data. Returning a list of users is quite easy, it's just showing the UserList:
func ListUsersHandler(w http.ResponseWriter, req *http.Request, env env) {
env.Render.JSON(w, http.StatusOK, db.List())
}
BTW, notice the Render.JSON? That's part of "github.com/unrolled/render" and allows us to render JSON output when we send data back to the client.
So, this will return the following to the client:
{
"users": [
{
"dateOfBirth": "1992-01-01T00:00:00Z",
"firstName": "Jane",
"id": 1,
"lastName": "Doe",
"locationOfBirth": "Milton Keynes"
},
{
"dateOfBirth": "1985-12-31T00:00:00Z",
"firstName": "John",
"id": 0,
"lastName": "Doe",
"locationOfBirth": "London"
}
]
}
It may surprise you that we are returning a JSON object that holds an array with multiple JSON objects, rather than an array with multiple JSON objects, as seen in the example below:
{
[
{
"dateOfBirth": "1992-01-01T00:00:00Z",
"firstName": "Jane",
"id": 1,
"lastName": "Doe",
"locationOfBirth": "Milton Keynes"
},
{
"dateOfBirth": "1985-12-31T00:00:00Z",
"firstName": "John",
"id": 0,
"lastName": "Doe",
"locationOfBirth": "London"
}
]
}
They're both valid and there are lots of views and opinions (as always in developer / architecture communities!), but the reason why I prefer to wrap the array in a JSON object is because later on we can easily add more data without causing significant changes to the client. What if we want to add the concept of pagination to our API?
Example:
{
"offset": 0,
"limit": 25,
"users": [
{
"dateOfBirth": "1992-01-01T00:00:00Z",
"firstName": "Jane",
"id": 1,
"lastName": "Doe",
"locationOfBirth": "Milton Keynes"
},
{
"dateOfBirth": "1985-12-31T00:00:00Z",
"firstName": "John",
"id": 0,
"lastName": "Doe",
"locationOfBirth": "London"
}
]
}
Another example is the retrieval of a specific object:
func GetUserHandler(w http.ResponseWriter, req *http.Request, env env) {
vars := mux.Vars(req)
uid, _ := strconv.Atoi(vars["uid"])
user, err := db.Get(uid)
if err == nil {
env.Render.JSON(w, http.StatusOK, user)
} else {
env.Render.JSON(w, http.StatusNotFound, err)
}
}
This reads the uid variable from the route (/users/{uid}), converts the string to an integer and then looks up the user in our UserList by ID. If the user does not exit, we return a 404 and an error object. If the user exists, we return a 200 and a JSON object with the user.
Example:
{
"dateOfBirth": "1992-01-01T00:00:00Z",
"firstName": "Jane",
"id": 1,
"lastName": "Doe",
"locationOfBirth": "Milton Keynes"
}
We add the attributes of the user object to the root of the JSON response, rather than wrapping it up in an explicit JSON object. I can quite easily add extra data to the response, without breaking the existing data.
Let's start with some simple curl tests. Open your terminal and try the following curl commands.
Retrieve a list of users:
curl -X GET http://localhost:3009/users | python -mjson.tool
That should result in the following result:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 216 100 216 0 0 14466 0 --:--:-- --:--:-- --:--:-- 15428
{
"users": [
{
"dateOfBirth": "1992-01-01T00:00:00Z",
"firstName": "Jane",
"id": 1,
"lastName": "Doe",
"locationOfBirth": "Milton Keynes"
},
{
"dateOfBirth": "1985-12-31T00:00:00Z",
"firstName": "John",
"id": 0,
"lastName": "Doe",
"locationOfBirth": "London"
}
]
}
The | python -mjson.tool at the end is for pretty printing (formatting). It essentially tells to pipe the output of the curl command to the SJON formatting tool. If we only typed curl -X GET http://localhost:3009/users then we'd have something like this:
{"users":[{"id":0,"firstName":"John","lastName":"Doe","dateOfBirth":"1985-12-31T00:00:00Z","locationOfBirth":"London"},{"id":1,"firstName":"Jane","lastName":"Doe","dateOfBirth":"1992-01-01T00:00:00Z","locationOfBirth":"Milton Keynes"}]}
So not that easy to read as the earlier nicely formatted example.
Get a specific user:
curl -X GET http://localhost:3009/users/0 | python -mjson.tool
Results in:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 104 100 104 0 0 6625 0 --:--:-- --:--:-- --:--:-- 6933
{
"dateOfBirth": "1992-01-01T00:00:00Z",
"firstName": "Jane",
"id": 1,
"lastName": "Doe",
"locationOfBirth": "Milton Keynes"
}
Adding a user:
TO DO
Deleting a user:
TO DO
Updating an existing user:
TO DO
TO DO
There are lots of opinions on testing, how much you should be testing, which layers of your applications, etc. When I'm working with micro services, I tend to focus on two types of tests to start with: testing the data access layer and testing the actual HTTP service.
In this example, we want to test the List, Add, Get, Update and Delete operations on our in-memory document database. The data access code is stored in the database.go file, so following Go convention we will create a new file called database_test.go.
In the database_test.go file, we have two sections:
First, we're going to create a common initialiser, this code sets up our database, inserts a couple of test records and then fires off the tests:
func TestMain(m *testing.M) {
list := make(map[int]User)
list[0] = User{0, "John", "Doe", "1985-12-31T00:00:00Z", "London"}
list[1] = User{1, "Jane", "Doe", "1992-01-01T00:00:00Z", "Milton Keynes"}
db = &Database{list, 1}
retCode := m.Run()
os.Exit(retCode)
}
Once this is ready, we can start writing tests. Let's have a look at the easiest one where we list the elements in our database:
func TestList(t *testing.T) {
list := db.List()
count := len(list["users"])
assert.Equal(t, 2, count, "There should be 2 items in the list.")
}
This first calls the db.List() function, which returns a list of users. We then count the number of elements and last but not least we then check whether that count equals 2.
In standard Go, you would actually write something like:
if 2 != count {
t.Errorf("Expected 2 elements in the list, instead got %v", count)
}
However there is a neat Go package called testify that gives you assertions like Java and that's why we can write cleaner test code like:
assert.Equal(t, 2, count, "There should be 2 items in the list.")
The TestList is only testing for a positive result, but we really need to test for failures as well.
This is our test code for the Delete functionality:
func TestDeleteSuccess(t *testing.T) {
ok, err := db.Delete(1)
assert.Equal(t, true, ok, "they should be equal")
assert.Nil(t, err)
}
func TestDeleteFail(t *testing.T) {
ok, err := db.Delete(10)
assert.Equal(t, false, ok, "they should be equal")
assert.NotNil(t, err)
}
The first test function TestDeleteSuccess tries to delete a known existing user, with Id 1. We're expecting that the error object is Nil. The second test function TestDeleteFail tries to look up a non-existing user with Id 10, and as expected, this should return an actual Error object.
How do we run the tests?
Simple:
go test
If you want it more verbose, then:
go test -v
Which will give you:
=== RUN TestList
--- PASS: TestList (0.00s)
=== RUN TestGetSuccess
--- PASS: TestGetSuccess (0.00s)
=== RUN TestGetFail
--- PASS: TestGetFail (0.00s)
=== RUN TestAdd
--- PASS: TestAdd (0.00s)
=== RUN TestUpdateSuccess
--- PASS: TestUpdateSuccess (0.00s)
=== RUN TestUpdateFail
--- PASS: TestUpdateFail (0.00s)
=== RUN TestDeleteSuccess
--- PASS: TestDeleteSuccess (0.00s)
=== RUN TestDeleteFail
--- PASS: TestDeleteFail (0.00s)
PASS
ok github.com/leeprovoost/go-rest-api-template 0.008s
Do you want to get some more info on your code coverage? No worries, Go has you covered (no pun intended):
go test -cover
This will give you:
PASS
coverage: 34.9% of statements
ok github.com/leeprovoost/go-rest-api-template 0.009s
The app binds itself by default to port 3009 and assumes that the fixtures.json file is in the project root directory. If that is not the case and you want to change that, then you can use some command line flags (or just change the code obviously).
This is how you would start the app with command line flags:
./go-rest-api-template -port=80 -fixtures=/tmp/fixtures.json
We achieve that by adding a flag parsers in the main.go init section:
fixturesLocation := flag.String("fixtures", "./fixtures.json", "location of fixtures.json file")
port = flag.String("port", "3009", "serve traffic on this port")
flag.Parse()
The flag.String function takes three arguments: the command-flag name, the default value and a description. Don't forget to add the flag.Parse() call after you've defined all the flags. Otherwise your system won't read the flag values.
Something that is often overlooked is ensuring that you have a deep insight in your application metrics. I admit that in the past I was mainly looking at server metrics like memory consumption, CPU usage, swap, etc. Once I started building micro-services using Coda Hale's excelent Java Dropwizard framework, I got to know his metrics library that gave me insight in the application and JVM metrics as an engineer.
As a start, you could look into two metrics: average response times for either your whole APU (or for a specific route) and the various HTTP status codes being returned. The important thing here is that you hook this up with a monitoring tool (e.g. Sensu) so that the tool periodically pings your metrics endpoint (in this example: http://localhost:3009/metrics) and look at trends. We're less interested in an individual snapshot. We will always have the occasional HTTP 404 being returend. However, if after a deployment you start seeing a spike in HTTP 404 codes being returned, then that should give you an indication that something might be wrong.
I experimented a bit with the [thoas/stats])https://github.com/thoas/stats) package but admit that I haven't used it in anger in a production enviornment.
We will first add the stats as a variable, named Metrics, to our environment struct:
type env struct {
Metrics *stats.Stats
Render *render.Render
}
Then initialise the variable in our main function (in main.go):
func main() {
env := env{
Metrics: stats.New(),
Render: render.New(),
}
router := mux.NewRouter()
// ...
}
The actual handler is pretty simple. In handlers.go, we add a handler called MetricsHandler and that just gets the data from our environment, renders it into a JSON format and returns an HTTP 200 OK status:
func MetricsHandler(w http.ResponseWriter, req *http.Request, env env) {
stats := env.Metrics.Data()
env.Render.JSON(w, http.StatusOK, stats)
}
If you curl the Metrics endpoint:
curl -X GET http://localhost:3009/metrics | python -mjson.tool
Then you should receive the following response:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 392 100 392 0 0 27724 0 --:--:-- --:--:-- --:--:-- 28000
{
"average_response_time": "93.247\u00b5s",
"average_response_time_sec": 9.324700000000001e-05,
"count": 0,
"pid": 71093,
"status_code_count": {},
"time": "2015-09-16 08:27:26.958487627 +0100 BST",
"total_count": 15,
"total_response_time": "1.398705ms",
"total_response_time_sec": 0.001398705,
"total_status_code_count": {
"200": 14,
"404": 1
},
"unixtime": 1442388446,
"uptime": "3m53.321206056s",
"uptime_sec": 233.321206056
}
You can start monitoring the response codes for instance. Let's say you get all of a sudden a spike in HTTP 404 messages, that might point to a failed deployment or a failing back-end service:
"total_status_code_count": {
"200": 14,
"404": 1
},
This is how you could run your app on a server:
First, you copy the binary and the fixtures.json files into a directory, e.g. /opt/go-rest-api-template.
Then start the app as a service. Store the app's PID in a text file so we can kill it later.
#!/bin/bash
sudo nohup /opt/go-rest-api-template/go-rest-api-template -fixtures=/opt/go-rest-api-template/fixtures.json -port=80 >> /var/log/go-rest-api-template.log 2>&1&
echo $! > /var/log/go-rest-api-template-pid.txt
When you want to kill your app later during a redeployment or a server shutdown, then you can kill the app bu looking up the previously stored PID:
#!/bin/bash
if [ -f /var/log/go-rest-api-template-pid.txt ]; then
kill -9 `cat /var/log/go-rest-api-template-pid.txt`
rm -f /var/log/go-rest-api-template-pid.txt
fi
- Structs and JSON formatting
- JSON and Go
- Design beautiful REST + JSON APIs
- Use render for generating JSON for use of global variable
- Read JSON POST body
- How to pass a parameter to a Http handler function
- Go and datetime parsing/formatting: ISO 8601, the International Standard for the representation of dates and times, Go by Example: Time Formatting / Parsing, JSON datetime formatting, src/time/format.go