todo-api is a small API web server built with Go. It supports user account
creation, login, and basic CRUD functions for tasks via a simple RESTful API.
While application functionality itself is contrived to serve as an example, this repository does demonstrate a few practical patterns for developing with Go:
- Use Docker Compose to manage service dependencies (Postgres, Redis) for local development
- Providing fast testing and linting feedback with
modd, which is critical to developer productivity and confidence - Using environment vars as the source for application configuration whether in deployed testing environments
- An integration testing strategy where the API is treated as a "black box," where the only interface is via HTTP request/response
Ultimately, every application has different needs and requirements—there is no one-true-way™ to build applications.
The features demonstrated by this application were converged upon after shipping dozens of small to medium APIs in Go over the past several years and constantly re-evaluating tooling choices and patterns.
We think it might be helpful reference point when you need a starting point for a new app, or as an answer to "how are others doing this" when you find yourself questioning your own approach.
We believe the curation of tools and patterns around the development of an application is important to the long term health and success of a project. They can greatly help or hinder aspects like developer onboarding, speed of feature development, and confidence around making code changes.
Docker Compose is used to run ephemeral dependencies without having to install them directly on the host machine.
We love make, but this project uses a Rakefile and thus requires on Ruby
installed on the host machine.
We've found over time that a Makefile can get unwieldy when managing a complex
application (e.g. database seeding, translation sync, code generation,
image/file asset management, etc).
Using rake and having the full power of Ruby for development tasks seems to
grow alongside a long living and complex application a bit better.
Postgres is used for application persistence, storing user accounts and tasks in a straight-forward relational data model.
Redis is used to store and persist user login sessions.
In practice, it would be completely acceptable to store sessions in memory or the database (depending on your application requirements), but we have added Redis as dependency mostly for demonstration purposes.
Package api is the API server backed by
jsonrest-go. It contains routing,
handlers, and middleware.
Package protocol translates domain objects to their API response format.
This requires you to be deliberate about exposing attributes of your domain
objects and allows domain objects to evolve without implicitly affecting their
response format.
Package conf holds all of the configuration for the application: e.g. database
connection strings, port to listen on, external credentials, environment. By
locating all configuration in one place, it's easy to see all parameters at a
glance. This approach implies that no other packages should access environment
variables directly.
Package domain contains the domain models of our application. It does not
contain any persistence logic; instead, that belongs in the repo package.
Our services are generally microservices, and therefore have a small enough scope for all of our models to live in one package. If you're building something with a larger scope (i.e. more monolith than microservice), it may make sense to break this into subpackages. However, make this decision very thoughtfully, and ensuring that your subpackages truly are independent, or else you may find yourself running into circular dependencies (an indication your packages were more coupled than you thought, and potentially shouldn't have been split up).
Expect this package to have few unit tests. For the most part, these are just models, without much logic. When do you do have logic in here (e.g. custom JSON marshaling, sort functions), consider writing unit tests.
The apicmd package hoists configuration and application startup together. This
allows the application entrypoint to be remain lean
and makes it easier to configure and run the application from selftest for
integration testing.
We treat the pkg directory as a "staging ground" for self-contained,
extractable packages. Once a package here is needed by multiple projects, it's a
signal that the package may be worth elevating to its own repo with a strong
commitment to semantic versioning.
Package repo contains a database persistence strategy for our domain models.
All interactions with the database should be implemented in this package; no
higher-level concept (e.g. request handlers) should execute database queries
directly.
The repo package contains unit tests that are executed against an actual
database instance, provided by Docker Compose. This gives us confidence that our
repo code actually works, ensuring that our queries are valid, and our model
mapping is correct.
The service directory is where you should find self-contained, but
application-specific functionality, particularly chunks which can be tested in
isolation.
Package selftest implements an automated integration test strategy that
launches the application under as realistic conditions as possible:
- It provides a real Redis and database connection
- It starts the actual API server listening on a port in the same manner as the application's package main
- It provides configuration via environmental variables
- It may provide external resources (e.g. 3rd party APIs) as fake implementations (not demonstrated by this project)
Once the API server is running, it's primarily tested as a black box; requests are made via HTTP, and responses are validated. The black box is only penetrated for test setup (e.g. seeding) and teardown (e.g. wiping the database in between tests).
This approach gives us extremely high confidence that our application will behave properly in production, since it was tested with the full launch process (including parsing env vars!) and real dependencies, no mocks. Because there's absolutely no dependency on internal implementation details, you can refactor at will without changing your tests so long as you maintain your API contracts.
A glaring omission from this project is both application tracing and logging (!). We may layer it in in a future update.