go.slide

# Go Development Best Practices

Lam Tran
Software Engineer
tranngoclam288@gmail.com

## Agenda

1. Code
2. Test
3. Build
4. Deploy
5. Observe

## Go coding best practices

## [code] Common structure

- `cmd` package to store multiple binaries.
- `internal` packages since Go 1.4, see [doc](https://go.dev/doc/go1.4#internalpackages) and [design](https://docs.google.com/document/d/1e8kOo3r51b2BWtTs_1uADIA5djfXhPT36s6eHVRIvaU/edit).

References:
- [Standard Go Project Layout](https://github.com/golang-standards/project-layout)

## [code] Coding style and convention

Effective Go (Names), see [doc](https://go.dev/doc/effective_go#names), package names, see [blog](https://go.dev/blog/package-names)

Use `test` suffix for testing package

	package httptest # import net/http/httptest

Avoid repetition

    package http

    type Server struct {} # usage: http.Server

    type Client struct {} # usage: http.Client

## [code] Coding style references

- [Effective Go](https://go.dev/doc/effective_go)
- [Go Code Review Comments](https://go.dev/wiki/CodeReviewComments)
- [Uber Go Style Guide](https://github.com/uber-go/guide/blob/master/style.md)
- [Google Go Style Guide](https://google.github.io/styleguide/go)
- [Thanos Coding Style Guide](https://github.com/thanos-io/thanos/blob/main/docs/contributing/coding-style-guide.md)
- [Mattermost Go Style Guide](https://developers.mattermost.com/contribute/more-info/server/style-guide)
- [Gruntwork Go Style Guide](https://docs.gruntwork.io/guides/style/golang-style-guide)
- [CockroachDB Go Coding Guidelines](https://cockroachlabs.atlassian.net/wiki/spaces/CRDB/pages/181371303/Go+Golang+coding+guidelines)

## [code] Code generation

`//go:generate` directive since Go 1.4, see [blog](https://go.dev/blog/generate) and [design](https://go.googlesource.com/proposal/+/refs/heads/master/design/go-generate.md).

Generated code example:

    // Code generated by mockery. DO NOT EDIT.

    package mocks

DO NOT format the generated code.

## [code] Go Modules

Go mod provides access to operations on modules, see [blog](https://go.dev/blog/using-go-modules)

Organizing a Go module, see [docs](https://go.dev/doc/modules/layout)

Usage:

	go mod <command> [arguments]

The commands are:

	download    download modules to local cache
	edit        edit go.mod from tools or scripts
	graph       print module requirement graph
	init        initialize new module in current directory
	tidy        add missing and remove unused modules
	vendor      make vendored copy of dependencies
	verify      verify dependencies have expected content
	why         explain why packages or modules are needed

## [code] Go Multi-module workspace

See [blog](https://go.dev/blog/get-familiar-with-workspaces) and [tutorial](https://go.dev/doc/tutorial/workspaces).

Usage:

	go work <command> [arguments]

The commands are:

	edit        edit go.work from tools or scripts
	init        initialize workspace file
	sync        sync workspace build list to modules
	use         add modules to workspace file

## [code] Dependency management

- Follow Go [Release Policy](https://go.dev/doc/devel/release#policy)

    Each major Go release is supported until there are two newer major releases. For example, Go 1.5 was supported until the Go 1.7 release, and Go 1.6 was supported until the Go 1.8 release. We fix critical problems, including critical security problems, in supported releases as needed by issuing minor revisions (for example, Go 1.6.1, Go 1.6.2, and so on).

- Use official, popular, zero/less-dependencies, in-active and stable maintenance modules.
- Use dependabot/renovate to keep dependencies up-to-date as soon as possible.
- "A little copying is better than a little dependency." - [Go Proverbs](https://go-proverbs.github.io)

## [code] DevTools - golangci-lint

    $ curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh \
        -s -- -b $(go env GOPATH)/bin v1.55.2

List of linters

    $ golangci-lint linters
    errcheck        errcheck is a program for checking for unchecked errors in Go code
    govet           vet examines Go source code and reports suspicious constructs
    unused          checks Go code for unused constants, variables, functions and types
    depguard        checks if package imports are in a list of acceptable packages
    revive          fast, configurable, extensible, flexible, and beautiful linter for Go
    gci             controls Go package import order and makes it always deterministic
    gosec           inspects source code for security problems
    ...

- Run with `--fix` for auto-fixing found issues (if linter supports)
- Use `enable-all: true` and disable unnecessary/deprecated linters

## [code] DevTools - mockery

Use [package configuration](https://vektra.github.io/mockery/latest/features/#packages-configuration)

    with-expecter: True
    packages:
      github.com/vektra/mockery/v2/pkg:
        interfaces:
          RequesterVariadic:
            config:
              with-expecter: False
      io:
        interfaces:
          Writer:
            config:
              with-expecter: False

Use [expecter structs](https://vektra.github.io/mockery/latest/features/#expecter-structs)

    requesterMock := mocks.NewRequester(t)
    requesterMock.EXPECT().Get("some path").Return("result", nil)

## [code] DevTools Management

[How can I track tool dependencies for a module?](https://go.dev/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module)

    $ cat tools.go
    // +build tools
    package tools

    import (
    	_ "golang.org/x/tools/cmd/stringer"
    )

    $ cat go.mod
    module go.tools
    go 1.21.6

    require golang.org/x/tools v0.18.0

    $ go install golang.org/x/tools/cmd/stringer
    $ which stringer
    /Users/lam/go/bin/stringer

[Proposal: Adding tool dependencies to go.mod](https://go.googlesource.com/proposal/+/refs/changes/55/495555/5/design/48429-go-tool-modules.md)

## [code] Configuration

Use URI for dependency which has connection string.

    MYSQL_URI=mysql://user:password@tcp(host:port)/dbname?query
    // from "github.com/go-sql-driver/mysql"
    func ParseDSN(dsn string) (cfg *Config, err error) { }

    REDIS_URI=redis://user:password@localhost:6789/3?dial_timeout=3&db=1&read_timeout=6s&max_retries=2
    // from "github.com/redis/go-redis/v9"
    func ParseURL(redisURL string) (*Options, error) { }

    RABBITMQ_URI=amqp://user:pass@host:10000/vhost
    // from "github.com/rabbitmq/amqp091-go"
    func ParseURI(uri string) (URI, error) { }

Use standard configuration environment variables from external sdk

    AWS_ENDPOINT_URL_DYNAMODB="https://dynamodb.me-central-1.amazonaws.com"
    OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic 123456abc123456abc"
    NEW_RELIC_APP_NAME="checkout-service"

## [code] Configuration (cnt.)

Avoid redundant data type

    type Config struct {
        // use `slog.Level` instead of `string`
        LogLevel slog.Level     `env:"LOG_LEVEL" yaml:"log_level" json:"log_level"`

        // use `netip.Addr` instead of `string`
        IP netip.Addr           `env:"IP"        yaml:"ip"        json:"ip"`

        // use `time.Duration` instead of `int`
        Timeout time.Duration   `env:"TIMEOUT"   yaml:"timeout"   json:"timeout"`
    }

Only provide configurations when needed.

## [code] Misc

Execute remote cli with `go run` since Go 1.17

    $ go run golang.org/x/tools/cmd/deadcode@latest .

    # main.go (executing with go generate)
    //go:generate go run github.com/bufbuild/buf/cmd/buf@v1.29.0 generate
    package main

Use built-in packages since Go 1.21: [slices](https://pkg.go.dev/slices), [maps](https://pkg.go.dev/maps), [cmp](https://pkg.go.dev/cmp)

Use [gonew](https://go.dev/blog/gonew) to start a new project from a template, see examples: [1](https://github.com/GoogleCloudPlatform/go-templates) and [2](https://github.com/ServiceWeaver/template)

Find unreachable functions with [deadcode](https://go.dev/blog/deadcode)

Vulnerability Management for Go with [govulncheck](https://go.dev/blog/vuln)

## [develop] Local development with Docker and Docker Compose

- Centralize dependencies in a single compose.yaml file including application + infrastructure (mysql, redis, rabbitmq, …) + testing (playwright, cucumber, …) + monitoring (prometheus, grafana) to:
  * extend usability for integration, e2e tests.
  * unify experience between local and CI/Preview environments.
- Use [develop.watch](https://www.docker.com/blog/announcing-docker-compose-watch-ga-release) stanza for local development with hot reload feature (reference)
- Use [depends_on](https://github.com/compose-spec/compose-spec/blob/master/spec.md#depends_on) and [healthcheck](https://github.com/compose-spec/compose-spec/blob/master/spec.md#healthcheck) stanza to express the startup and shutdown dependencies between services.
- Use [build.context](https://docs.docker.com/compose/compose-file/build/#attributes) stanza with remote git repositories to deploy multiple containers in polyrepo (reference).
- Use [delve](https://github.com/go-delve/delve) for debugging in microservices.
- Use [devcontainer](https://containers.dev) to develop inside container.

## Go testing best practices

## [test] Best practices

- Use table-driven tests, see [wiki](https://go.dev/wiki/TableDrivenTests)
- Use `t.Parallel()` for parallel testing.
- Use `t.Skip()` to skip a test.
- Use `t.Setenv()` to set environment variables (since Go 1.17).
- Use `t.Cleanup()` to clean up after a test or benchmark has finished (since Go 1.14), see [article](https://ieftimov.com/posts/testing-in-go-clean-tests-using-t-cleanup)
- Use `go test -tags integration` to separate integration vs unit tests.
- Use `testdata` to manage test fixtures.
- Use `go test -race` to detect data race in tests.
- Use Uber's [goleak](https://github.com/uber-go/goleak) to detect leaked goroutines.

## [test] Best practices (cnt.)

- Code coverage for Go integration tests, see [blog](https://go.dev/blog/integration-test-coverage).
- Testable examples in Go, see [blog](https://go.dev/blog/examples)
- References:

  - [Go Testing By Examples (Russ Cox)](https://www.youtube.com/watch?v=X4rxi9jStLo)
  - [Advanced Testing with Go (Mitchell Hashimoto)](https://www.youtube.com/watch?v=8hQG7QlcLBk)
  - [Testing Techniques (Andrew Gerrand)](https://www.youtube.com/watch?v=ndmB0bj7eyw)
  - [Learn Go with tests](https://quii.gitbook.io/learn-go-with-tests)

## Go building best practices

## [build] Embedding build metadata and assets

Embedding one file into a string or a slice of bytes:

    import _ "embed"

    //go:embed hello.txt
    var s string

    //go:embed hello.txt
    var b []byte

Embedded one or more files into a file system:

    import "embed"

    //go:embed migrations
    var assets embed.FS

## [build] Dockerfile best practices

- Get started with docker init command: [reference](https://www.docker.com/blog/docker-init-initialize-dockerfiles-and-compose-files-with-a-single-cli-command)
- Use multi-stage builds: [reference](https://docs.docker.com/build/building/multi-stage)
- Use RUN –mount=type=cache: [reference](https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/reference.md#run---mounttypecache)
- Use RUN –mount=type=bind: [reference](https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/reference.md#run---mounttypebind)
- Use RUN –mount=type=ssh: [reference](https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/reference.md#run---mounttypessh)
- Use distroless image: [Chainguard](https://edu.chainguard.dev/chainguard/chainguard-images/getting-started/getting-started-go) | [Google Distroless](https://github.com/GoogleContainerTools/distroless)
- Pin image tags to digests: [reference](https://github.com/step-security/secure-repo?tab=readme-ov-file#4-pin-image-tags-to-digests-in-dockerfiles)
- Avoid copying files from host to container image.
- Keep Go binary as small as possible: [reference](https://blog.howardjohn.info/posts/go-binary-size)
- COPY from remote registry: [reference](https://github.com/openfga/openfga/blob/53deb4c165fbca3260b4342e714580d10a316dcf/Dockerfile#L23)
- Adopt OCI labels: [reference](https://github.com/opencontainers/image-spec/blob/main/annotations.md) | [docker-metadata action](https://github.com/docker/metadata-action)

## [build] Dockerfile best practices (cnt.)

    FROM golang:1.22.0@sha256:7b297d9... AS build

    ENV GOPRIVATE=gitlab.com/org/go-module

    WORKDIR /src

    RUN mkdir -p -m 0700 ~/.ssh && \
        ssh-keyscan gitlab.com >> ~/.ssh/known_hosts && \
        git config --global url.ssh://git@gitlab.com/.insteadOf https://gitlab.com/

    RUN --mount=type=ssh \
        --mount=type=cache,target=/go/pkg/mod/ \
        --mount=type=bind,source=go.sum,target=go.sum \
        --mount=type=bind,source=go.mod,target=go.mod \
        ssh -q -T git@gitlab.com 2>&1 | go mod download

    RUN --mount=type=cache,target=/go/pkg/mod \
       --mount=type=cache,target=/root/.cache/go-build \
       --mount=type=bind,target=. \
       CGO_ENABLED=0 go build --ldflags "-s -w" -o /bin/server ./cmd/server && \
       go build --ldflags "-s -w" -o /bin/migration ./cmd/migration

## [build] Dockerfile best practices (cnt.)

    FROM cgr.dev/chainguard/static@sha256:43f8701...

    COPY --from=build /bin/server /server
    COPY --from=build /bin/migration /migration
    COPY --from=ghcr.io/grpc-ecosystem/grpc-health-probe:v0.4.24 /ko-app/grpc-health-probe /probe

    CMD ["/server"]

## Go deployment best practices

## [deploy] Health checks

Use shallow health check: `GET /healthz` which returns `200 OK`

References:

  - [Amazon Builder - Implementing health checks](https://aws.amazon.com/builders-library/implementing-health-checks)
  - [Better Stack - Getting Started with Server Health Checks](https://betterstack.com/community/guides/monitoring/health-checks)
  - [Kubernetes Deep Health Check](https://encore.dev/blog/horror-stories-k8s)

## [deploy] Database schema migration

- Database schema migration should be a separate task, run before deploying applications.
- Both old/new app versions must be compatible with old/new schema versions.
- Use [golang-migrate/migrate](https://github.com/golang-migrate/migrate) to handle database schema migration.
- Use migration directory integrity file, see [blog](https://atlasgo.io/concepts/migration-directory-integrity)

## [deploy] Graceful shutdown

- Use `signal.NotifyContext(parent context.Context, signals ...os.Signal)` to listen unix signals (terminate, interrupt, kill, ...)

References:

  - [Graceful shutdown and zero downtime deployments in Kubernetes](https://learnk8s.io/graceful-shutdown)

## Go observability best practices

## [observe] Log

Use `log/slog` package to handle structured logging, see [blog](https://go.dev/blog/slog), [talk](https://www.youtube.com/watch?v=8rnI2xLrdeM), [guide](https://github.com/golang/example/blob/master/slog-handler-guide/README.md)

Example: getting trace data from context

    type loggingHandler struct {
        slog.Handler
    }

    func (h *loggingHandler) Handle(ctx context.Context, record slog.Record) error {
        if txn := newrelic.FromContext(ctx); txn != nil {
            record.AddAttrs(slog.String("trace.id", txn.GetTraceMetadata().TraceID))
        }

        return h.Handler.Handle(ctx, record)
    }

## [observe] Log (cnt.)

Example: setup slog

    func SetupLogging(cfg Config, appRole string) {
        opts := &slog.HandlerOptions{
            Level: cfg.LogLevel,
            ReplaceAttr: func(groups []string, a slog.Attr) slog.Attr {
                if a.Key == slog.MessageKey {
                    a.Key = "message"
                }
                if a.Key == slog.TimeKey {
                    a.Key = "timestamp"
                }
                return a
            },
        }
        attrs := []slog.Attr{slog.String("app_role", appRole)}
        jsonHandler := slog.NewJSONHandler(os.Stdout, opts).WithAttrs(attrs)
        slog.SetDefault(slog.New(&loggingHandler{jsonHandler}))
        slog.Info("setting up logging with slog", slog.String("log.level", cfg.LogLevel.String()))
    }

## [observe] OpenTelemetry

- Getting started with [documentation](https://opentelemetry.io/docs)
- See OpenTelemetry in action: [demo](https://github.com/open-telemetry/opentelemetry-demo)

References:

  - [What Makes a Service Observable?](https://www.codereliant.io/what-makes-a-service-observable)