Skip to content
Permalink
Branch: master
Find file Copy path
Find file Copy path
2 contributors

Users who have contributed to this file

@sean- @jclulow
2482 lines (2187 sloc) 95.4 KB
authors state
Sean Chittenden <seanc@joyent.com>
abandoned

RFD 106 Engineering Guide - Go Best Practices

Introduction

Joyent is embracing of Go’s contribution to distributed, enterprise, production-grade computing and celebrates the software engineering ethos held by many in the Go community because their beliefs are aligned with our General Principles.

In this document we will define and articulate Joyent’s evolving set of best-practices with regards to how to Go.

Each item has a recommendation and a rationale. Over time these recommendations and rationale are apt to evolve and be refined to further reflect our experiences. Omissions or gaps in the list of items, the recommendations, or their rationale is almost certainly unintentional. At times several of the recommendations or rationale become pithy due to time constraints or the perceived lack of a need to provide a more robust explanation, and similarly, this is unintentional. If an issue warrants additional discussion, please open an issue.

To provide feedback, bug fixes, request clarification, or initiate a discussion, please open an RFD issue and reference RFD 106 along with the necessary specifics. For issues where there is a debate or discussion, this document will include a link back to the individual issues where a particular item was addressed or debated.

This RFD makes use of terminology defined in BCP 14 (RFC 2119 and RFC 8174, i.e. "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL").

Core Beliefs

At Joyent we have the following beliefs regarding software:

  • Build robust, trusted systems

    • Build secure systems

    • Data integrity

  • Operability

  • Maintainability

  • Availability

  • Debuggablity

It’s not possible to prescribe a fool-proof set of best-practices but it is possible to influence the outcome. To wit, in order to satisfy these beliefs, we take principled stances on the following (in no particular order, but roughly ordered from philosophical to practical):

  • Version of Go

  • Project Structure

  • Workflow

  • Style

  • $EDITOR Integration

  • vendor/ Management

  • Naming Conventions

  • Repository Names

  • API stability

  • Static Analysis

  • Explicit Types

  • return Parameters

  • Interface Receivers

  • const

  • Constructing and Using Bitmasks

  • Documentation

  • Use of context

  • Deadline Timers and Timeouts

  • pprof

Beliefs

Build Robust, Trusted Systems
  • Data Integrity

  • Predictability

  • Security

Operability
Maintainability
Availability
Debuggablity
Principles

1. Version of Go

Recommendation

  1. The latest released version or a current release candidate of Go SHOULD be used for day-to-day development.

  2. A released version of Go or a release candidate , until the next version becomes available, MUST be used for building released binaries.

  3. The latest version of Go SHOULD be used for release engineering.

  4. master MAY be used for development and is RECOMMENDED for Continuous Integration (CI) testing.

  5. Use of gobin(1) is RECOMMENDED in order to keep $GOPATH/bin's binaries in sync with that of the Go compiler.

  6. Every time the version of go(1) changes the contents of $GOPATH/pkg MAY be cleaned out:

    $ rm -rf $GOPATH/pkg/

Rationale

The Go team and project are consistently advancing the state of the Go compiler and standard library. Released versions and release candidates are considered fit for production and should therefore be used in production. Upgrade early and often.

When release candidates are available, they should be used until the next finalized release is made available.

The Go team and contributors have a good track record of advancing the state of the compiler in terms of stability and performance. Tendencies toward risk aversion frequently have a higher cost (e.g. security, correctness, performance, or stability) than absorbing the cost of any incremental upgrade.

Use of master is not peril free, however it does provide a good way of staying current and doesn’t typically cause much grief during development. Use of master MAY improve correctness and quality.

The version of go(1) may be tightly coupled with binaries located in $GOPATH/bin (especially those binaries who either walk the Go AST or interact with other go(1) tooling). gobin(1) automatically rebuilds $GOPATH/bin binaries who are out of date with respect to the version of the go(1) compiler used. Because Go developer tools tend to be written in Go and necessarily coupled to the version of go(1), rebuilding $GOPATH/bin binaries every time go(1) is updated is RECOMMENDED.

The cached archive files in $GOPATH/pkg are explicitly tightly coupled with go(1) but starting in Go 1.5, go(1) now maintains a package version in order to automatically detect and trigger the rebuild packages.[1]

Use of tools like vg(1) MAY be useful or considered in the future but are NOT REQUIRED: most of these recommendations are intended to aid productivity and decrease friction, not constrain how someone goes about working.

2. Project Structure

Recommendation

  1. $GOPATH SHOULD be set to $HOME/go and SHOULD be incorporated into your shell’s environment.

  2. All development SHOULD be done within $GOPATH/src.

  3. $GOPATH/bin SHOULD be part of PATH and before /usr/local or /opt/local (i.e. before system or package manager managed binaries).

    $ export GOPATH=$HOME/go
    $ export PATH=$GOPATH/bin:$PATH
  4. Where appropriate, it is RECOMMENDED to make use of monolithic repositories (mono-repo).

  5. Publicly consumable libraries or programs SHOULD be pushed to a distinct canonical public location and automatically synchronized to the internal codebase.

Rationale

Starting in Go 1.8, go(1) defaulted to $HOME/go as its default GOPATH. It is not strictly necessary to set GOPATH, however it is still advised to make this implicit default explicit. Many tools or pieces of software test for the environment variable GOPATH instead of using using go env GOPATH.

In Go 1.8, the Go project defaulted to $HOME/go as the default value for $GOPATH. Use of one-workspace per project is counter-productive and establishes a workflow that is orthogonal to the ethos of the prevailing Go ecosystem. This isn’t to say there are times where this is necessary (i.e. clean-room verification or maintenance of vendor/), but the default practice SHOULD be to work inside of a single $GOPATH workspace [2].

Go’s tooling makes it especially productive to move all libraries and programs into the same codebase so that refactoring can commence in atomic units of work. In particular, making sweeping changes via gofmt(1) -r is easy to accomplish in a single repository and commit. Breaking apart individual libraries into discrete repositories fragments the codebase with no isolation guarantees that Version Control System (VCS) doesn’t already provide. Contrast that with having all libraries and programs in the same codebase, it is now possible to move the entire codebase forward in an atomic transaction [3]. Additional arguments in support of monorepos include:

Publicly reusable components, however, SHOULD be discretely usable.

3. Workflow

Recommendation

Engineer workflow changes based on whether or not you have write-privileges to the target repository.

If you HAVE write access to a repository and it is Github-like:

  1. Checkout the repository:

    $ go get -d my.git.server/my_org/my_project
  2. Create a branch for your change:

    $ cd $GOPATH/src/my.git.server/my_org/my_project
    $ git checkout -b my-branch-name
  3. Commit your change(s):

    $ git commit
  4. Push your change to origin:

    $ git push -u origin my-branch-name
  5. Submit a Pull Request (PR).

  6. You SHOULD obtain a review. For all changes deemed to be trivial this is not necessary, however the change MUST be made through a PR in order to to aid in a quick backout commit.

  7. Automated regression tests MUST complete and pass.

  8. If the velocity of change for the repository is low enough, a CHANGELOG entry for the project SHOULD be committed to the PR as the final step before merging the PR. If the velocity of the repository is too high, the CHANGELOG entry for the project MAY be added after the PR has been merged.

  9. Merge the PR. If the history of the PR is messy with unhelpful commits (e.g. "fix typo", "update test"), perform a squash merge with a detailed, high-quality commit message that has been approved by the rest of the team. Detail that can’t be expressed in the commit message should be outlined in code comments.

  10. Pull the latest changes:

    $ git checkout master && git pull origin master
  11. Delete your local branch:

    $ git branch -d my-branch-name
  12. Delete your branch from the server (e.g. my-branch-name).

If you do NOT HAVE write access to a repository the workflow is largely the same except you MUST create a fork of the repository:

  1. Checkout the original repository:

    $ go get -d -v my.git.server/my_org/my_project
  2. Fork the upstream repository to your individual user account.

  3. Add the remote for your repository:

    $ cd $GOPATH/src/my.git.server/my_org/my_project
    $ git remote add me my.git.server/my_user/my_project
  4. Create a branch for your change:

    $ git checkout -b my-branch-name
  5. Commit your change(s)

  6. Push your change to me:

    $ git push -u me my-branch-name
  7. A CHANGELOG entry SHOULD be incorporated into the PR unless the upstream project will write the CHANGELOG entry for you.

  8. Submit a Pull Request (PR).

  9. Wait for the upstream provider to merge your PR.

  10. Pull the latest changes:

    $ git checkout master
    $ git pull origin master
  11. Delete your local branch:

    $ git branch -d my-branch-name
Important

Work MUST be completed within the same directory as the upstream source and not the path to your fork of an upstream module (i.e. CORRECT: $GOPATH/src/my.git.server/upstream_org/my_project WRONG: $GOPATH/src/my.git.server/my_user/my_project). Instead use the path of your upstream source, but use a different git remote URL.

If you HAVE write access to a repository and it is Gerrit:

  1. Checkout the repository:

    $ git clone --origin gerrit https://my.git.server/my_org/my_project.git
  2. Create a branch for your change:

    $ git checkout -b my-branch-name
  3. Commit your change(s):

    $ git commit
  4. Push your change to origin:

    $ git push gerrit HEAD:refs/for/master
  5. You MUST obtain a review.

  6. Automated regression tests MUST complete and pass.

  7. A CHANGELOG entry MUST be committed to the PR as the final step before merging the PR.

  8. Merge the PR. If the history of the PR is messy with unhelpful commits (e.g. "fix typo", "update test"), perform a squash merge with a detailed, high-quality commit message that has been approved by the rest of the team. Detail that can’t be expressed in the commit message should be outlined in code comments.

  9. Pull the latest changes:

    $ git checkout master
    $ git pull origin master
  10. Delete your local branch:

    $ git branch -d my-branch-name

4. Style

Recommendation

  1. All code MUST pass through gofmt(1). gofmt(1) SHOULD be executed with the -s flag.

  2. Lines SHOULD wrap at 80 characters.

Rationale

The particular brand of tribal fascism that extends from gofmt(1) increases the overall productivity of the entire Go community by creating a single dialect of Go that is universal across projects, teams, and organizations. Being able to drop into any arbitrary Go project, regardless of the copyright, and be able to understand the codebase quickly is a universal boon.

The only observable consequence to adhering to `https://golang.org/cmd/gofmt/[gofmt(1)]’s set of style norms is the cost of shedding the sentimental attachment to a preference for "my way of doing things". Developing a personal or project-wide coding style takes discipline to adhere to, an understanding of the style guide’s rules (including their rationale), and an eagle-eye to enforce. Investment in such skills and the pride attached to that skill-set is near-zero in the Go community. Shedding personal preference - justified or not - in favor of a prescribed doctrine is a tangible hurdle to overcome.

Note

The computing industry has been well served by project-wide style guidelines in part because this created a sufficiently high barrier to entry which acted as a litmus-test to ensure tribal norms were understood and communicated to new members of the tribe. With many of the original industrial programming languages being riddled with undefined behavior (e.g. C or C++), style guides helped communities of engineers ship more reliable code and with fewer bugs because project-wide idioms had a tendency to be put in place for a reason.

Even before go(1) adopted gofmt(1) to enforce Go’s single-style guideline, ident(1) existed as a crude tool for enforcing style (crude to the point that ident(1) was eschewed because it was unable to perform at the levels required for a developer tool). In no way should clang-format(1) or clang-tidy(1) be lumped into the same league of correctness as ident(1) because clang-format(1) and clang-tidy(1) [4] recreate the AST before rewriting code (vs the brute-force text-level tokenization performed by most ident(1) implementations).

The value and merit of individual or project preferences with regards to the artistry stemming from style guides has been eclipsed by the value generated from participating in the open, code-sharing world of the Go Open Source ecosystem. Go came into the world with a lack of legacy, fragmentation, or tribalism and has largely remained an unfragmented community in large part due to its fungability of both Go developers and code that can be readily shared across either projects or organizations.

gofmt(1) SHOULD be used in place of the go(1) tool’s fmt command because:

  1. gofmt(1) supports the -s flag to simplify code where possible.

  2. go fmt calls gofmt(1): src/cmd/go/internal/fmtcmd/fmt.go L42-L71

  3. gofmt(1) supports programmatic rewriting of the code base via the -r flag.

  4. Code SHOULD be fungible. Go’s simple syntax, emphasis on readability, and "side-effect"-free code largely make this a reality.

5. Developer Tools

Recommendation

The following tools are RECOMMENDED for development:

  1. goimports(1):

    $ go get -u golang.org/x/tools/cmd/goimports
  2. guru(1):

    $ go get -u golang.org/x/tools/cmd/guru
  3. godoc(1):

    $ go get -u golang.org/x/tools/cmd/godoc
  4. gorename(1):

    $ go get -u golang.org/x/tools/cmd/gorename
  5. gomvpkg(1):

    $ go get -u golang.org/x/tools/cmd/gomvpkg
  6. gobin(1):

    $ go get -u github.com/rjeczalik/bin/cmd/gobin

Rationale

goimports(1)

Manually maintaining import declarations is a tedious waste of time. goimports(1) gets this right 99% of the time and increases productivity significantly once integrated into your $EDITOR. goimports(1) does periodically get the package wrong when there is ambiguity, but with a nudge in the right direction it doesn’t go off the rails again for a particular source file.

guru(1)

guru(1) SHOULD be integrated into your $EDITOR because it enables quick, authoritative traversal of codebases. guru(1) is a huge productivity bump and can’t have enough good things said about it. Watch Navigating Unfamiliar Code with the Go Guru and read Using Go Guru: an editor-integrated tool for navigating Go code. Spiritually guru(1) could probably attribute a material portion of its inspiration with ctags(1), cscope(1), and LXR, however guru(1) is much more sophisticated.

The list of guru(1)s functionality includes (as of July 2017, and taken from https://golang.org/x/tools/cmd/guru):

Identifier Queries
what

The what query describes the current source position as rapidly as possible. It is not intended to be invoked directly by the user, but it allows editors to provide immediate feedback in the UI whenever the cursor position changes. It can be used to highlight all identifiers that are equivalent to current one.

definition

The definition query finds the declaration of the selected identifier. In some editors, it may jump the cursor directly to that location.

referrers

The referrers query finds references to the selected identifier, scanning all necessary packages within the workspace.

freevars

The freevars query enumerates the free variables of the selection. "Free variables" is a technical term meaning the set of variables that are referenced but not defined within the selection, or loosely speaking, its inputs.

Type Queries
describe

The describe query shows various properties of the selected syntax: its syntactic kind, the type of an expression, the value of a constant expression, the size, alignment, method set, and interfaces of a type, the declaration of an identifier, and so on. You may describe almost any piece of syntax, and guru(1) will print all the useful information it can.

implements

The implements query shows interfaces that are implemented by the selected type and, if the selected type is itself an interface, the set of concrete types that implement it. An implements query on a value reports the same information about the expression’s type. An implements query on a method shows the set of abstract or concrete methods that are related to it.

Call Graph Queries
callees

The callees query shows the possible call targets of the selected function call site. The cursor or selection must be within a function call expression; the selection need not be exact.

callers

The callers query shows the possible callers of the function containing the selection.

callstack

The callstack query shows an arbitrary path from the root of the call graph to the function containing the selection. This may be useful to understand how the function is reached in a given program.

Alias Queries
pointsto

The pointsto query shows the set of possible objects to which a pointer may point. It also works for other reference types, like slices, functions, maps, and channels.

whicherrs

The whicherrs query reports the set of possible constants, global variables, and concrete types that may appear in a value of type error. This information may be useful when handling errors to ensure all the important cases have been dealt with.

peers

The peers query shows the set of possible sends/receives on the channel operand of the selected send or receive operation; the selection must be a <- token.

godoc(1)

godoc(1) SHOULD be installed in order to have quick access to formatted documentation. Before committing a new body of work, the documentation for the package should be inspected. A strong cofactor in determining the reusable value of software is its documentation (see also: network effect).

gorename(1) and gomvpkg(1)

gorename(1) SHOULD be used for renaming package, function, and method members (i.e. const, func, var, and type).

While less commonly needed, gomvpkg(1) SHOULD be used when moving packages around because it updates the necessary import declarations in a given scope.

gobin(1)

Use gobin(1) to rebuild $GOPATH/bin is RECOMMENDED after every upgrade of go(1). Rebuilding $GOPATH/bin by hand is equally acceptable but tedius and error prone.

$ gobin -u
ok	/Users/seanc/go/bin/goflymake	(github.com/dougm/goflymake)	2.674s
ok	/Users/seanc/go/bin/http-echo	(github.com/hashicorp/http-echo)	3.133s
ok	/Users/seanc/go/bin/codecgen	(github.com/ugorji/go/codec/codecgen)	3.827s
ok	/Users/seanc/go/bin/misspell	(github.com/client9/misspell/cmd/misspell)	4.531s
ok	/Users/seanc/go/bin/licenseok	(github.com/golang/dep/hack/licenseok)	5.001s
ok	/Users/seanc/go/bin/go-erd	(github.com/gmarik/go-erd)	1.871s
ok	/Users/seanc/go/bin/humanlog	(github.com/aybabtme/humanlog/cmd/humanlog)	5.879s
ok	/Users/seanc/go/bin/goconst	(github.com/jgautheron/goconst/cmd/goconst)	1.804s
ok	/Users/seanc/go/bin/git-codereview	(golang.org/x/review/git-codereview)	3.299s
ok	/Users/seanc/go/bin/stringer	(golang.org/x/tools/cmd/stringer)	6.001s
ok	/Users/seanc/go/bin/tomll	(github.com/golang/dep/vendor/github.com/pelletier/go-toml/cmd/tomll)	3.838s
ok	/Users/seanc/go/bin/smoke	(github.com/MediaMath/keryxlib/smoke)	9.568s
ok	/Users/seanc/go/bin/gocyclo	(github.com/fzipp/gocyclo)	1.493s
ok	/Users/seanc/go/bin/cmd	(github.com/golang/dep/vendor/github.com/pelletier/go-toml/cmd)	4.757s
ok	/Users/seanc/go/bin/unused	(honnef.co/go/tools/cmd/unused)	12.658s
ok	/Users/seanc/go/bin/keyify	(honnef.co/go/tools/cmd/keyify)	9.188s
ok	/Users/seanc/go/bin/structcheck	(github.com/opennota/check/cmd/structcheck)	9.794s
ok	/Users/seanc/go/bin/varcheck	(github.com/opennota/check/cmd/varcheck)	13.938s
ok	/Users/seanc/go/bin/gotype	(golang.org/x/tools/cmd/gotype)	8.968s
ok	/Users/seanc/go/bin/gorename	(golang.org/x/tools/cmd/gorename)	10.470s
ok	/Users/seanc/go/bin/set	(github.com/ncw/gotemplate/examples/set)	2.191s
ok	/Users/seanc/go/bin/vendorfmt	(github.com/magiconair/vendorfmt/cmd/vendorfmt)	2.557s
ok	/Users/seanc/go/bin/ineffassign	(github.com/gordonklaus/ineffassign)	2.097s
ok	/Users/seanc/go/bin/unparam	(github.com/mvdan/unparam)	15.957s
ok	/Users/seanc/go/bin/aligncheck	(github.com/opennota/check/cmd/aligncheck)	11.086s
ok	/Users/seanc/go/bin/gox	(github.com/mitchellh/gox)	3.246s
ok	/Users/seanc/go/bin/deadcode	(github.com/remyoudompheng/go-misc/deadcode)	10.044s
ok	/Users/seanc/go/bin/gotemplate	(github.com/ncw/gotemplate)	1.678s
ok	/Users/seanc/go/bin/tomljson	(github.com/golang/dep/vendor/github.com/pelletier/go-toml/cmd/tomljson)	3.557s
ok	/Users/seanc/go/bin/interfacer	(github.com/mvdan/interfacer/cmd/interfacer)	17.489s
ok	/Users/seanc/go/bin/lll	(github.com/walle/lll/cmd/lll)	2.962s
ok	/Users/seanc/go/bin/godo	(gopkg.in/godo.v2/cmd/godo)	22.895s
ok	/Users/seanc/go/bin/dep	(github.com/golang/dep/cmd/dep)	7.762s
ok	/Users/seanc/go/bin/cover	(golang.org/x/tools/cmd/cover)	8.697s
ok	/Users/seanc/go/bin/safesql	(github.com/stripe/safesql)	10.935s
ok	/Users/seanc/go/bin/gocode	(github.com/nsf/gocode)	4.424s
ok	/Users/seanc/go/bin/staticcheck	(honnef.co/go/tools/cmd/staticcheck)	16.404s
ok	/Users/seanc/go/bin/goimports	(golang.org/x/tools/cmd/goimports)	5.515s
ok	/Users/seanc/go/bin/golint	(github.com/golang/lint/golint)	8.446s
ok	/Users/seanc/go/bin/gosimple	(honnef.co/go/tools/cmd/gosimple)	15.178s
ok	/Users/seanc/go/bin/unconvert	(github.com/mdempsky/unconvert)	16.471s

Time spent in user mode   (CPU seconds) : 113.222s
Time spent in kernel mode (CPU seconds) : 52.636s
Total time                              : 0:49.03s
CPU utilization (percentage)            : 338.2%
Times the process was swapped           : 0
Times of major page faults              : 8981
Times of minor page faults              : 8560363
# Update go1.9rc1 to go1.9rc2 but *forgot* to copy bin/go{,fmt} to ~/go/bin/
$ gobin -u
fail	/Users/seanc/go/bin/goconst	(github.com/jgautheron/goconst/cmd/goconst)
	error: exit status 2
	# github.com/jgautheron/goconst
	compile: version "go1.9rc2" does not match go tool version "go1.9rc1"

fail	/Users/seanc/go/bin/goflymake	(github.com/dougm/goflymake)
	error: exit status 2
	# github.com/dougm/goflymake
	compile: version "go1.9rc2" does not match go tool version "go1.9rc1"
[snip]
$ cp -pf ~/go1.9/bin/* $GOPATH/bin/
$ gobin -u
ok	/Users/seanc/go/bin/set	(github.com/ncw/gotemplate/examples/set)	1.723s
ok	/Users/seanc/go/bin/vendorfmt	(github.com/magiconair/vendorfmt/cmd/vendorfmt)	1.758s
ok	/Users/seanc/go/bin/gocyclo	(github.com/fzipp/gocyclo)	1.793s
[snip]

6. $EDITOR Integration

Recommendation

This section is NOT making a recommendation regarding any particular $EDITOR.[5] This section is, however making a strong recommendation that your $EDITOR include the following interrogations in order to aid in maximal productivity:

  1. goimports(1) is SHOULD be added as a save hook. $EDITOR instructions are found at: https://godoc.org/golang.org/x/tools/cmd/goimports.

  2. guru(1) SHOULD be integrated into your $EDITOR as a plugin. Binding "jump-to-definition" to an easy-to-access keybinding is strongly RECOMMENDED. Instructions can be found at https://golang.org/s/using-guru. See guru(1) rationale.

  3. gorename(1) SHOULD be integrated into your $EDITOR as a plugin (instructions for emacs, vim).

Specific editor integrations (alphabetically sorted):

  • emacs users SHOULD look at go-mode.el and MAY OPTIONALLY investigate integrating gocode. With guru(1) installed, enabling go-guru-hl-identifier-mode is RECOMMENDED when navigating code.

  • JetBrains users SHOULD look at Gogland.

  • vim users SHOULD look at vim-go and MAY OPTIONALLY investigate integrating gocode.

Rationale

$EDITOR preferences and configuration is an intensely personal subject. Integrating gofmt -s -w $FILE may be a benefit to your individual workflow. Some people who use emacs really like flymake or flycheck or go-guru-hl-identifier-mode, whereas others don’t like the extra background CPU they incur. The list above is both lightweight and common. Additional suggestions or tips to improve $EDITOR productivity are welcome.

7. vendor/ Management

Recommendation

  1. Forked and cached libraries in vendor/ MUST be managed via the dep(1) tool:

    $ go get -u github.com/golang/dep/cmd/dep   // (1)
    $ dep status                                // (2)
    $ dep ensure                                // (3)
    $ dep ensure -update                        // (4)
    $ dep init                                  // (5)
  2. In a monorepo, whomever wants to update the bits in vendor/ and Gopkg.lock MAY update the version, however they MUST:

    1. take responsibility for making the change (and updating code as necessary).

    2. testing the change.

    3. communicate the change with consumers of the library.

    4. receive approval from teams receiving the update.

  3. Gopkg.toml SHOULD NOT lock a version to a specific version without reason. Valid reasons include:

    1. Upstream did in-fact change something that requires local attention and the cost of fixing the change locally is currently too high.

    2. Upstream did in-fact commit something that is materially broken and the cost of fixing the bug upstream is too high.

  4. Release CI runs MUST use the version specified in Gopkg.lock.

  5. Non-release CI SHOULD be able to report vendor drift via dep status. If the CI environment is safely isolated to the extent that you’re willing to run uninspected code from upstream, dep ensure -update SHOULD be run and report breakage caused by upstream changes.

  6. vendor/ and Gopkg.lock SHOULD be updated regularly in order to prevent forklift upgrades.

  7. Understanding the difference between the Gopkg.toml and Gopkg.lock files is REQUIRED.

Rationale

As of Gophercon 2017, dep(1) is on track to becoming the community defacto vendor/ management tool (this is also on track according to their roadmap). If a project is using either godep(1) or govendor(1), please make plans to upgrade to dep(1).

See also https://github.com/golang/dep/issues/281 and the dep(1) FAQ. Gopkg.toml documentation is RECOMMENDED reading, too.

CI automatically updating dependencies in non-release builds provides a motivation for vendor/ to more closely track the upstream’s most recently tagged version or master. Tracking more frequent, small changes is less error prone than large "#yolo updates."

Warning
Confusingly named, dep(1) is not the same as godep(1). dep(1) is the future, not godep(1).
Caution
CI systems can only run dep ensure -update if the CI systems are capable of running untrusted, foreign code (or some other compensating control is in place).

8. Naming Conventions

Recommendations

  1. Go software SHOULD conform to the recommendations outlined in the following resources:

  2. Package authors MAY deviate from these conventions IF they have sought feedback from engineers who have sufficient experience writing Go libraries.

  3. When working with a forked copy of a package, package import paths MUST continue to use the canonical, public import path. See comments in workflow.

  4. Package aliases SHOULD be used when necessary and there are two libraries with the same package name.

  5. Programs SHOULD NOT explicitly import a package into the current namespace (i.e. do not use import . "lib/math" Sin).

  6. Programs MAY import a package for their side effects using the black identifier (i.e. a package’s init() MUST run). For example:

    link:examples/import_test.go[]
    1. imports database/sql

    2. Invoke’s github.com/lib/pq's init() method because it registers itself with the sql package.

Rationale

Naming is one of the hard things in software. The package semantics of Go help with this dilemma and minimize the blast-radius of poorly chosen names. With tools like guru(1) commonly in use, the practice of encoding extraneous type and package information into variable names is non-idiomatic and frowned upon. The burden for good naming and exported functions falls on library authors.

9. Repository Names

Recommendations

  1. Repositories that contain Go libraries that wrap APIs or where there are competing libraries written in different langauges MUST be prefixed with go-.

  2. Repositories that house libraries whose only consumers are internal to Go SHOULD be prefixed with go-.

  3. Go repositories whose primary consumer is a library, but who also contain a binary, SHOULD conform to the go-mylib/cmd/mycmdname.

Rationale

Repositories where the distinct possibility of a name conflict may arise should be prefixed with the primary language in order to create a unique namespace. Examples of this at Joyent include:

In a different example where a library also contains a binary, such as go-sockaddr. go-sockaddr the repository contains:

  1. Two libraries/packages: sockaddr and its template package.

  2. One binary: sockaddr(1)

Standalone binaries such as those referenced in the Static Analysis section are examples of standalone commands that do not need to be nested under a cmd/ subdirectory nor do they need to be prefixed with go- because of their domain.

10. API Stability

Recommendation

  1. APIs within a single project SHOULD use tightly-coupled function signatures.

  2. Refactoring APIs within a single project SHOULD use gofmt(1)'s -r flag to migrate function signatures.

  3. External APIs that are loosely coupled across projects AND potentially unstable SHOULD use struct inputs. For example:

    package mypkg
    struct MyFuncInputs {
      ArgA string
      ArgB int
      ArgC bool
    }
    func MyFunc(args MyFuncInputs) {
      // ...
    }

    on the caller’s side:

    mypkg.MyFunc(MyFuncInput{
      ArgA: "foo",
      ArgB: 0xba72,
      Argc: true,
    })
  4. Required arguments SHOULD be extracted from the input struct.

  5. Optional arguments or parameters that are subject to change by the authors of the library SHOULD be included in the input struct in order to provide loose coupling between the library and its consumers.

  6. Where input arguments are not reused across API calls, use of stack-initialized (e.g. MyFuncInput{}) input structs SHOULD be used (vs heap initialized, e.g. &MyFuncInput{}).

Rationale

Tightly coupled interfaces within the same project SHOULD be treated as local where possible. The onus for maintaining the API MUST be on the author changing the function signature. Tools that programmatically rewrite the codebase SHOULD be employed to make the change. The entire change SHOULD be merged as a single operation. Sweeping mechanical changes SHOULD be committed independent of either functional or behavioral changes.

External APIs that are loosely coupled where consumers of a library are apt to not update all of their call sites need to acknowledge that it is a maintenance cost to enforce tight coupling between a project and an external library. Use of struct input arguments allows:

  1. library maintainer to advance the functionality of their library independently

  2. consumers of the library to update without fear of breaking their API

Note

This recommendation stems from the following hypothetical:

Imagine a function signature is:

func MyFunc(a string, b int) { /* ... */ }

and the authors of MyFunc() decide the function signature needs to be updated to:

func MyFunc(a string, b int, c bool) { /* ... */ }

All consumers of MyFunc() must update to the new signature. In some cases this compile-time breakage may be desirable in order for MyFunc()'s authors to communicate a breaking change or semantic change that requires some level of understanding by the consumer. In other cases, the authors of MyFunc() may have added new functionality without changing the semantic meaning of the contract API. In the latter case, adding functionality to MyFunc() requires source-code level API flexibility with a permissive interfaces in order to minimize the maintenance cost incurred by consumers.

This could be achieved by adding an additional variadic function argument:

MyFunc(a string, b int, args ...interface{}) { /* ... */ }

but that approach would require runtime checking of the variadic argument, args, and would eschew compile-time safety guarantees (and subsequent optimizations). If the consumers of MyFunc() span team or organizational boundaries, it is effectively impossible to force callers to update their interface to match the new function signature.

This recommendation is to introduce a static function signature with an "append-only input structure":

type MyFuncInputs struct {}
func MyFunc(ctxt context, dnode uint64, MyFuncInputs{}) {
  //
}

The function signature for MyFunc() can now be effectively frozen and stable from the perspective of the consumers of the library. MyFunc()'s new signature acknowledges that both ctxt and dnode are required arguments yet still allows the library author to extend the API in the future by appending members to the MyFuncInputs struct.

If an API is performance sensitive, this approach MAY NOT be appropriate. Use of this technique is an exercise in forethought where the cost of maintenance burdened by the author is weighed against the runtime performance impact of passing an optional struct input to a function. It is difficult to imagine the case where the execution cost of thousands of requests per second would outweigh the engineering burden of maintaining a frequently updated or loosely coupled interface that spans repositories.

This technique must adhere to similar rules as those suggested when updating a protobuf message type, notably:

  • *Input struct member names are permanent and MUST NOT change or have their meaning altered in a way that changes their contract.

  • Obsolete *Input struct member names MUST:

    1. be automatically mapped to an updated struct member(s)

    2. ignored (a discouraged practice)

    3. removed thereby explicitly breaking any existing code

    4. never be reused for the life of the interface (and therefore the *Input struct.

      A phased approach to evolving a *Input struct is an acceptable strategy.

Again, this is a recommended technique for providing stable interfaces where the runtime and diminished readability has been weighed against the cost of maintenance (most notably engineering time or runtime breakage).

11. Static Analysis

Recommendations

  1. Use and integration of "baseline static analysis checks" SHOULD be integrated into the CI.

  2. An inventory of "optional static analysis checks" is RECOMMENDED but not necessary for a second tier of checks to be added to list of suggested static analysis checks (e.g. "noisy, but useful" or "mostly accurate, but still throws false-positives").

Rationale

reviewdog stands out as a pragmatic way to programmatically raise the bar of quality within a given Go project by automatically executing and providing inline annotations in PRs with the results of baseline checks. If a particular type of error occurs more than a few times, write a static analysis check and incorporate it into reviewdog.

For offline development, use of gometalinter(1) is RECOMMENDED:

$ go get -u github.com/alecthomas/gometalinter
$ gometalinter --install

Regardless of the tool, incorporating a baseline of static analysis of commonly identified issues frees up reviewers to focus on the content of change versus the mechanics of the change. Time invested in static analysis checks usually pays dividends with respect to preventing bugs (e.g. scopelint, go tool vet --shadow, errcheck, safesql, staticcheck), reducing sub-optimal code (e.g. ineffassign, unparam), or reducing engineering time wasted pointing out nits that could be identified consistently by bots (e.g. go vet, lll (long line linter), misspell).

Several recommended static analysis checks include (most come from gometalinter(1), alphabetically sorted):

Several optional linters include (alphabetically sorted):

12. Explicit Types

Recommendation

  1. Explicit types SHOULD be used within a project.

  2. Libraries or public APIs MAY export types where it helps readability.

  3. Where the meaning or intent of a fundamental type would benefit from explicit type checking by the compiler, explicit types SHOULD be used.

  4. Type Conversions SHOULD be deferred as long as reasonable.

  5. Where explicitly typed variables are employed, the lifecycle of identifiers referencing underlying types SHOULD be reduced to the smallest reasonable scope possible.

  6. Use of gorename(1) to maintain type names is RECOMMENDED. The RECOMMENDED use of gorename(1) extends to all package, function, and method members (i.e. const, func, var, and type).

Rationale

Go is an explicitly typed language. The compiler does not perform any implicit type conversions of named types. Exported functions, interfaces, and types SHOULD make use of explicit types in order to enable the compiler to detect and enforce a package’s specified type system. It is NOT RECOMMENDED to deprive the compiler of the necessary type information it requires in order to prevent developers from incorrectly and abusively overloading Go’s underlying types (e.g. string vs RandomStringID, or uint64 vs inode).

As an example, a string SHOULD be thought of as an immutable slice of runes that is missing its type information (i.e. a string is a container, not a type).

Go’s fundamental or underlying types (e.g. string, int*, []byte) are containers that crudely answer the question "how is a variable going to be stored efficiently." Use of underlying types do not answer the question "what bits are in a given container." Go does not permit any implicit type conversions of named types.

Go’s explicit type system prevents variables backed by the same underlying type from fraternizing. Use of fundamental types at formal interface boundaries is discouraged because use of variable names to indicate the intended use of a variable is only enforced by the reader, not by the compiler. If variable names are sufficient to guard against variable misues, you MAY rely on variable names to convey type information.

Where type intent information SHOULD be enforced by the compiler, use of explicit types is RECOMMENDED. The Go type system is a compile-time cost, not a runtime cost. Use types.

Examples:

type ID string                // (1)
type ID uint64                // (2)
type CookieID string
type UUID []byte
type Index uint
type Key string
type Value string
type Lookup map[Key]Value
  1. ID may have started out as a string

  2. ID could be easily changed to a uint64 and the consequences easily observed. NOTE: this wouldn’t compile due to the ID identifier being reused in the same package.

13. Return Parameters

Recommendation

  1. When deciding if a function or method should return an argument by value or pointer, returning a value SHOULD be your default position except in the following situations, in which case it is RECOMMENDED to return a pointer to a value:

    1. the API contract you want to establish with the caller is to force them to deal with errors by returning nil AND the construction of the zero-value is onerous or expensive (i.e. return "" for a string).

    2. ownership of the variable may change throughout the course of the variable’s life.

    3. the expense of copying the variable is measurable.

Rationale

Go uses pass-by-value semantics and employs variable escape analysis.

Embrace the pass-by-value nature of Go, be productive, and let the compiler do work for you.

Much of the above reading was shamelessly borrowed from a Stack Overflow article which is a good read on its own merits.

14. Interface Receivers

Recommendation

  1. When deciding if a receiver should be a value or a pointer, a pointer SHOULD be used by default except in the following situations, in which case it is RECOMMENDED to use a value:

    1. the value of the receiver is a simple underlying type (i.e. an int)

    2. invocation of the given interface method SHOULD result in a copy of the receiver.

Rationale

This is simple: use a pointer to a receiver in nearly all cases. Item 1b is very rare in practice.

type Foo struct {
  bar string
}

// Baz assigns "bur" to f.bar.  Without the pointer, this the instance of Foo
// would have been copied and the assignment would have been not visible to
// the caller (a nice source of frustration when first learning Go).
func (f *Foo) Baz() {
  f.bar = "bur"
}

In practice, use of non-pointer receivers is limited to the following example:

type MyEnum int

func (e MyEnum) String() string {
  switch e {
  case 0:
    return "zero"
  case 1:
    return "one"
  default:
    return "something not one or zero"
  }
}

var myEnum MyEnum = 0
fmt.Println("%s", myEnum)

Where the important takeaway is that in String(), it doesn’t matter if the value is copied.

15. const

Recommendation

  1. Use of const is RECOMMENDED.

  2. Create explicitly typed consts is RECOMMENDED.

  3. consts with type information SHOULD should be exported (both the type and the const values).

  4. Periodically using static analysis checks like goconst is RECOMMENDED but OPTIONAL.

Rationale

By creating a const, you give the Go tooling an identifier which you can search for referrers of the given const. See the referrers section of the Using Guru document (this document SHOULD be required reading).

16. Bitmasks

Recommendation

  1. Bitmasks SHOULD be created using const and iota.

  2. Bitmasks SHOULD be explicitly typed.

  3. The meaning of bits in a bitmask MAY change if it is documented in the interface that the meaning of individual bits may change.

  4. The meaning of bits MUST NOT change if the bitmask is exported and the position of individual bits is part of the contract.

  5. A new type, removal of the bitmask as a type, or other form of compile-time breakage MUST be introduced in order to communicate the change in behavior.

  6. Manual manipulation of bitmasks SHOULD NOT be performed without explicitly named bits.

Rationale

Go provides a convenient trick to automatically creating bitmasks:

type MyBitmask int

const (
      FlagA MyBitmask = 1 << iota  // (1)
      FlagB                        // (2)
      FlagC                        // (3)
      FlagD                        // (4)
)
  1. FlagA == 0x01

  2. FlagB == 0x02

  3. FlagC == 0x04

  4. FlagD == 0x08

Leverage this trick.

17. Documentation

Recommendation

  1. Projects MUST use godoc(1) to document their project.

Rationale

Read the Godoc: documenting Go code blog post.[6]

18. Context

Recommendation

  1. Projects MUST the context pattern for passing state along request-scoped state information (e.g. deadlines, cancelation signals, or request-specific information).

  2. Instances of context.Context SHOULD use the identifier name ctxt, however ctx MAY be used.

Rationale

Read the Go Concurrency Patterns: Context blog post.

ctxt is the most common identifier for context.Context in use. Unless there is a good naming conflict, ctxt is a good identifier to use that unambiguously communicates intent.

$ git branch
* (HEAD detached at go1.9rc2)
  master
$ git status -s
$ git grep ctxt | wc -l
    5149
$ git grep ctx | grep -v ctxt | wc -l
    1360

19. Signal Handling and context

Recommendation

  1. Signal handling SHOULD incorporate context where possible.

Rationale

context is a good way to handle signals from the OS. Go handles the sigprocmask(2) and sigaction(2) so that signals are delivered to a single OS thread.

link:examples/main/main.go[]
  1. Create a Background context.

  2. Save the newly created context.Context and its cancelation function.

  3. Create a buffered channel in order to prevent the OS thread from blocking.

  4. Wire up SIGINT to be delivered to sigCh.

  5. Push a defer handler onto the stack to shutdown the signal handler and cancel anything hanging off of the context.Context.

  6. Disable the delivery of new signals to sigCh.

  7. Cancel the context.Context instance. Cancelation operations are idempotent (which makes the cancelation at callout #7 safe).

  8. Spin off a goroutine to catch and handle signals delivered to sigCh.

  9. Drain the signal from sigCh.

  10. Switch on the signal type (not necessary for this simple example, but imagine a single signal hanler that is also capable of responding to SIGHUP).

  11. Handle the SIGINT

  12. Cancellation handler for powerNap()

Credit for this idiom and example:

20. Deadline Timers and Timeouts

Recommendation

  1. In-process timers and timeouts MUST use time.Duration.

  2. In-process timers and timeouts using the context package MUST use context.WithTimeout.

  3. Inter-process timeout enforcement MUST communicate using an absolute time reference using time.Time.

  4. Inter-process timeouts using the context package MUST use context.WithDeadline.

Rationale

21. gRPC

Recommendation

  1. gRPC SHOULD be preferred as the RPC framework for communicating between discrete Go processes locally or on the network.

  2. JSON MAY be used as the RPC framework when necessary to interoperate with non- gRPC clients.

Rationale

Read the Go Concurrency Patterns: Context blog post.

22. pprof and gops(1)

Recommendation

  1. Long-running daemons MUST have either a gops(1) or pprof endpoint available on a secure endpoint (a loopback interface, the default, is considered a secure endpoint).

  2. The gops(1) endpoint is RECOMMENDED and can be added to code like:

    link:examples/gops_test.go[]
    1. Import the agent

    2. Listen early on after configuration

  3. The pprof endpoint SHOULD use a standardized URL anchors.

  4. The pprof endpoint MAY be made available via alternate libraries other than the canonical net/http/pprof listener. See [agent] for a RECOMMENDED way of enabling the pprof interface. If [agent] CAN NOT be used, the following fragment is sub-optimal SHOULD NOT be used to create a local listener:

    link:examples/net_http_pprof_bad_test.go[]
    link:examples/net_http_pprof_bad_test.go[]
    1. The net/http package is only needed for the standalone listener used in callout 5.

    2. Initialize the net/http/pprof package.

    3. init() SHOULD NOT be used to spawn this listener in order to allow for proper configuration to take place and to allow the program to respond to errors in the event the pprof endpoint fails to initialie. The pprof endpoint SHOULD be initialized before this service enters its duty-cycle.

    4. Start a detached thread to handle pprof requests.

    5. Listen on a local endpoint and port

    6. "One second ought to be enough time for any service to fail."

    Warning

    This is a BAD example of how because: the code does not handle errors well; assumes the listener will start and fail within one second, and this introduces an arbitrary one second delay to program initialization. See the next example for a preferred way of initializng the pprof endpoint which addresses each of these concerns. The example snippet in net/http/pprof's documentation is convenient and uncomplicated but SHOULD NOT be used in production for the aforementioned reasons.

Rationale

TODO.

23. External Resources

  1. The Go Programming Language

  2. Less is exponentially more

  3. Go Proverbs

  4. Go wiki

  5. Awesome Go: A curated list of awesome Go frameworks, libraries and software.

Appendix A: Go 101 Resources

If you are completely new to Go, "I haven’t written a line of Go in my life before today," this section is a quick primmer to get you up and running in a hurry.

A.1. Five Minute Jump Start

Suppose you know nothing and just need to get started. The following steps SHOULD get you to a functioning "Hello World" in ~5min.

  1. Install a stable copy of Go. This can be done in a number of ways, either from the official Download Page or via your preferred package management system:

    1. pkgsrc: sudo pkgin -y install go

    2. MacpPrts: sudo port install go

    3. Homebrew: sudo brew install go

  2. Open a terminal and add the following environment variables to your shell (as long as your GOPATH is set and PATH includes $GOPATH/bin at the end of this step how you get this done doesn’t matter):

    $ echo 'export GOPATH=$HOME/go' >> ~/.profile
    $ echo 'export PATH=$GOPATH/bin:$PATH' >> ~/.profile
    $ export GOPATH=$HOME/go
    $ export PATH=$GOPATH/bin:$PATH
  3. Ensure git(1) has already been installed.

  4. Install gometalinter(1): go get -u github.com/alecthomas/gometalinter

  5. Install gometalinter(1)'s linters: gometalinter --install

  6. Install gops(1): go get -u github.com/google/gops

  7. Install expvarmon(1): go get -u github.com/divan/expvarmon

  8. Install go-torch(1): go get -u github.com/uber/go-torch

  9. Install FlameGraph: go get -d github.com/brendangregg/FlameGraph

  10. Add FlameGraph's scripts to your PATH: export PATH=$PATH:$GOPATH/github.com/brendangregg/FlameGraph

  11. Change into an existing sample program: cd $GOPATH/src/github.com/google/gops/examples/hello

  12. Edit main.go and make the following edits:

     package main
    
     import (
    +       "fmt"
            "log"
    -       "time"
    
            "github.com/google/gops/agent"
     )
    
    +func getFactorial(n int) int {
    +       if n < 0 {
    +               return 0
    +       }
    +
    +       if n <= 1 {
    +               return 1
    +       }
    +
    +       return n * getFactorial(n-1)
    +}
    +
     func main() {
            if err := agent.Listen(nil); err != nil {
                    log.Fatal(err)
            }
    -       time.Sleep(time.Hour)
    +
    +       fmt.Println("Hello World!  Please debug me.")
    +       for i := 1000000; i > 0; i-- {
    +               getFactorial(i)
    +       }
     }
  13. Build the program: go build

  14. Run the program:

    $ ./hello
    Hello World!  Please debug me.

    This program will run in the foreground for the next hour or until you interrupt the program (e.g. ctrl+c).

  15. The program is available to be inspected via gops(1):

    $ gops
    44326	gops	(/Users/seanc/go/bin/gops)
    44319*	hello	(/Users/seanc/go/src/github.com/google/gops/examples/hello/hello)
    $ gops stats 44319
    goroutines: 5
    OS threads: 7
    GOMAXPROCS: 8
    num CPU: 8
    $ gops memstats 44319
    alloc: 96.28KB (98592 bytes)
    total-alloc: 96.28KB (98592 bytes)
    sys: 67.72MB (71012352 bytes)
    lookups: 7
    mallocs: 354
    frees: 7
    heap-alloc: 96.28KB (98592 bytes)
    heap-sys: 32.69MB (34275328 bytes)
    heap-idle: 32.22MB (33783808 bytes)
    heap-in-use: 480.00KB (491520 bytes)
    heap-released: 0 bytes
    heap-objects: 347
    stack-in-use: 32.31MB (33882112 bytes)
    stack-sys: 32.31MB (33882112 bytes)
    next-gc: when heap-alloc >= 4.27MB (4473924 bytes)
    last-gc: -
    gc-pause: 0s
    num-gc: 0
    enable-gc: true
    debug-gc: false
    $ gops stack 44319
    goroutine 8 [running]:
    runtime/pprof.writeGoroutineStacks(0x12094e0, 0xc4200c8020, 0x0, 0x0)
    	/Users/seanc/go1.9/src/runtime/pprof/pprof.go:608 +0xa7
    runtime/pprof.writeGoroutine(0x12094e0, 0xc4200c8020, 0x2, 0x0, 0x0)
    	/Users/seanc/go1.9/src/runtime/pprof/pprof.go:597 +0x44
    runtime/pprof.(*Profile).WriteTo(0x1219060, 0x12094e0, 0xc4200c8020, 0x2, 0x0, 0x0)
    	/Users/seanc/go1.9/src/runtime/pprof/pprof.go:310 +0x3ab
    github.com/google/gops/agent.handle(0x12094e0, 0xc4200c8020, 0xc4200ca000, 0x1, 0x1, 0x0, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/agent/agent.go:171 +0x1a3
    github.com/google/gops/agent.listen()
    	/Users/seanc/go/src/github.com/google/gops/agent/agent.go:119 +0x2c5
    created by github.com/google/gops/agent.Listen
    	/Users/seanc/go/src/github.com/google/gops/agent/agent.go:100 +0x457
    
    goroutine 1 [runnable]:
    main.getFactorial(0xefb92, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:14 +0x82
    main.getFactorial(0xefb93, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefb94, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefb95, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefb96, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefb97, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefb98, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefb99, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefb9a, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefb9b, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefb9c, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefb9d, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefb9e, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefb9f, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefba0, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefba1, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefba2, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefba3, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefba4, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefba5, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefba6, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefba7, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefba8, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefba9, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbaa, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbab, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbac, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbad, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbae, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbaf, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbb0, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbb1, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbb2, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbb3, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbb4, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbb5, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbb6, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbb7, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbb8, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbb9, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbba, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbbb, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbbc, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbbd, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbbe, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbbf, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbc0, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbc1, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbc2, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbc3, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbc4, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbc5, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbc6, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbc7, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbc8, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbc9, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbca, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbcb, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbcc, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbcd, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbce, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbcf, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbd0, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbd1, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbd2, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbd3, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbd4, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbd5, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbd6, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbd7, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbd8, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbd9, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbda, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbdb, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbdc, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbdd, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbde, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbdf, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbe0, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbe1, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbe2, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbe3, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbe4, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbe5, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbe6, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbe7, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbe8, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbe9, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbea, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbeb, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbec, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbed, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbee, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbef, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbf0, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbf1, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbf2, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbf3, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbf4, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    main.getFactorial(0xefbf5, 0x0)
    	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
    ...additional frames elided...
    
    goroutine 5 [syscall]:
    os/signal.signal_recv(0x0)
    	/Users/seanc/go1.9/src/runtime/sigqueue.go:131 +0xa7
    os/signal.loop()
    	/Users/seanc/go1.9/src/os/signal/signal_unix.go:22 +0x22
    created by os/signal.init.0
    	/Users/seanc/go1.9/src/os/signal/signal_unix.go:28 +0x41
    
    goroutine 6 [select, locked to thread]:
    runtime.gopark(0x1166cb8, 0x0, 0x115e4e1, 0x6, 0x18, 0x1)
    	/Users/seanc/go1.9/src/runtime/proc.go:277 +0x12c
    runtime.selectgo(0xc420040f50, 0xc42007c180)
    	/Users/seanc/go1.9/src/runtime/select.go:395 +0x1138
    runtime.ensureSigM.func1()
    	/Users/seanc/go1.9/src/runtime/signal_unix.go:511 +0x1fe
    runtime.goexit()
    	/Users/seanc/go1.9/src/runtime/asm_amd64.s:2337 +0x1
    
    goroutine 7 [chan receive]:
    github.com/google/gops/agent.gracefulShutdown.func1(0xc420066060)
    	/Users/seanc/go/src/github.com/google/gops/agent/agent.go:132 +0x34
    created by github.com/google/gops/agent.gracefulShutdown
    	/Users/seanc/go/src/github.com/google/gops/agent/agent.go:130 +0xb5
    $ gops trace 44319
    Tracing now, will take 5 secs...
    Trace dump saved to: /var/folders/w0/tf58slqj3rs5mklbnvp5k7vh0000gp/T/trace902953220
    2017/08/03 14:17:23 Parsing trace...
    2017/08/03 14:17:23 Serializing trace...
    2017/08/03 14:17:23 Splitting trace...
    2017/08/03 14:17:23 Opening browser
    # gops will block in the foreground - push ctrl+c when done
    # This program is uneventful so there's nothing to see
    $ gops pprof-heap 44319
    Profiling dump saved to: /var/folders/w0/tf58slqj3rs5mklbnvp5k7vh0000gp/T/profile786664192
    Binary file saved to: /var/folders/w0/tf58slqj3rs5mklbnvp5k7vh0000gp/T/binary663659615
    File: binary663659615
    Type: inuse_space
    Time: Aug 3, 2017 at 2:19pm (PDT)
    Entering interactive mode (type "help" for commands, "o" for options)
    (pprof) help
      Commands:
        callgrind        Outputs a graph in callgrind format
        comments         Output all profile comments
        disasm           Output assembly listings annotated with samples
        dot              Outputs a graph in DOT format
        eog              Visualize graph through eog
        evince           Visualize graph through evince
        gif              Outputs a graph image in GIF format
        gv               Visualize graph through gv
        kcachegrind      Visualize report in KCachegrind
        list             Output annotated source for functions matching regexp
        pdf              Outputs a graph in PDF format
        peek             Output callers/callees of functions matching regexp
        png              Outputs a graph image in PNG format
        proto            Outputs the profile in compressed protobuf format
        ps               Outputs a graph in PS format
        raw              Outputs a text representation of the raw profile
        svg              Outputs a graph in SVG format
        tags             Outputs all tags in the profile
        text             Outputs top entries in text form
        top              Outputs top entries in text form
        topproto         Outputs top entries in compressed protobuf format
        traces           Outputs all profile samples in text form
        tree             Outputs a text rendering of call graph
        web              Visualize graph through web browser
        weblist          Display annotated source in a web browser
        o/options        List options and their current values
        quit/exit/^D     Exit pprof
    
      Options:
        call_tree        Create a context-sensitive call tree
        compact_labels   Show minimal headers
        divide_by        Ratio to divide all samples before visualization
        drop_negative    Ignore negative differences
        edgefraction     Hide edges below <f>*total
        focus            Restricts to samples going through a node matching regexp
        hide             Skips nodes matching regexp
        ignore           Skips paths going through any nodes matching regexp
        mean             Average sample value over first value (count)
        nodecount        Max number of nodes to show
        nodefraction     Hide nodes below <f>*total
        output           Output filename for file-based outputs
        positive_percentages Ignore negative samples when computing percentages
        prune_from       Drops any functions below the matched frame.
        relative_percentages Show percentages relative to focused subgraph
        sample_index     Sample value to report (0-based index or name)
        show             Only show nodes matching regexp
        source_path      Search path for source files
        tagfocus         Restrict to samples with tags in range or matched by regexp
        taghide          Skip tags matching this regexp
        tagignore        Discard samples with tags in range or matched by regexp
        tagshow          Only consider tags matching this regexp
        trim             Honor nodefraction/edgefraction/nodecount defaults
        unit             Measurement units to display
    
      Option groups (only set one per group):
        cumulative
          cum              Sort entries based on cumulative weight
          flat             Sort entries based on own weight granularity
          addresses        Aggregate at the function level.
          addressnoinlines Aggregate at the function level, including functions' addresses in the output.
          files            Aggregate at the file level.
          functionnameonly Aggregate at the function level.
          functions        Aggregate at the function level.
          lines            Aggregate at the source code line level.
          noinlines        Aggregate at the function level.
    
      type "help <cmd|option>" for more information
    (pprof) exit
    $ go-torch -b /var/folders/w0/tf58slqj3rs5mklbnvp5k7vh0000gp/T/profile786664192
    INFO[18:44:38] Run pprof command: go tool pprof -raw /var/folders/w0/tf58slqj3rs5mklbnvp5k7vh0000gp/T/profile786664192
    INFO[18:44:38] Writing svg to torch.svg
    $ open -a /Applications/FirefoxNightly.app torch.svg
    $ gops pprof-cpu 44319
    Profiling CPU now, will take 30 secs...
    Profiling dump saved to: /var/folders/w0/tf58slqj3rs5mklbnvp5k7vh0000gp/T/profile127087150
    Binary file saved to: /var/folders/w0/tf58slqj3rs5mklbnvp5k7vh0000gp/T/binary771063221
    File: binary771063221
    Type: cpu
    Time: Aug 3, 2017 at 6:46pm (PDT)
    Duration: 30.16s, Total samples = 26.21s (86.90%)
    Entering interactive mode (type "help" for commands, "o" for options)
    (pprof) exit
    $ go-torch -b /var/folders/w0/tf58slqj3rs5mklbnvp5k7vh0000gp/T/profile127087150
    INFO[18:47:35] Run pprof command: go tool pprof -raw /var/folders/w0/tf58slqj3rs5mklbnvp5k7vh0000gp/T/profile127087150
    INFO[18:47:36] Writing svg to torch.svg
    $ open -a /Applications/FirefoxNightly.app torch.svg
    $ cat $HOME/.config/gops/${PID_OF_HELLO_PROCESS} ; echo
    62023
Scheduler Latency
Figure 1. Output from gops trace and the scheduler latency profile

It’s obviously not required to step through and enumerate all of the capabilities of gops(1) and the pprof endpoints when writing a "Hello World" app but it shows off some of the built-in capabilities. Additional information is covered in the section on pprof.

A.2. Books

The following is a list of books that people have found helpful for getting started. One of the problems with learning by dead tree is the disparity between inches of shelf-space occupied to a given subject versus the mental real-estate a given subject occupies. Because a language feature is address - sometimes at length - doesn’t mean it is a primary concern or something you need to become a subject-matter expert on.

The following are "approved" resources ("approved" in this context only means "this wasn’t awful and will point you in the right direction" and doesn’t constitute as a explicit endorsement #JoyentEmployee). There may be other good resources that exist, but they haven’t been reviewed (pull requests welcome).

  • The Go Programming citenp:[go-lang(27-282), go-lang(353-366)]

  • Programming in Go citenp:[prog-in-go(1-250), prog-in-go(265-424)]

  • An Introduction to Programming in Go citenp:[intro-to-prog-go] (now available as a PDF as it is now longer in print).

A.2.2. Extra Resources

A.3. make(1)

make(1) comes in many shapes and forms. GNU make(1) is the predominant make(1) implementation. GNU make(1), however, has its own dialect and its dialect is not universally supported. If you use GNU make(1)-specific extensions, you MUST spell the Makefile like GNUmakefile. The same rule applies to BSD make(1) and Makefile.[7] It is unfortunate that there is poor interopoerability between the various make(1) implementations and it’s equally unfortunate that the prevailing attitudes is to be conformist (versus taking a zero-cost principled stand for correctness).

$ cat GNUmakefile
test:
    echo $(shell echo bar)
$ bsdmake -f GNUmakefile
echo

$ gnumake -f GNUmakefile
echo bar
bar

In the end, this fragmentation has led many to adopt make(1) as the universal human UX and system interoperability layer to front tools like bezel(1) or goreleaser(1).

Note

make(1) is like Latin. Many western languages decend from make(1), however no one speaks Latin anymore. Instead people speak Spanish, Italian, French, English, etc. If you don’t speak French and call it Latin, I won’t speak Spanish and call it Latin. Incorrectness has consequences. In software it most oftens in deferred cost.

Glossary

CI

Continuous Integration

Bibliography

bibliography::[]


1. Cleaning out $GOPATH/pkg is an ounce of prevention whose tiny cost is worth more than the pound of debugging. Post Go 1.5 I’ve heard a handful of anecdotal reports of this fixing some extremely unlikely and inexplicable panics that gets cleared up after $GOPATH/pkg has been preened. It’s unclear if this is true, rare, a myth, or a scapegopher for something else.
2. Tools similar to bazel could influence this recommendation in the future however there are no plans to augment the workflow presented by the go(1) tool.
3. Monorepos can be justified by either their productivity gains, by Parkinson’s law, or by blurring blurring the natural organizational lines stemming from Conway’s law by embracing the egalitarian nature of software.
6. It would be nice, however, if godoc(1) supported a richer markup like asciidoc.
7. Irritatingly enough, .MAKE.MAKEFILE_PREFERENCE does not default to using BSDmakefile in its preference list by default.
You can’t perform that action at this time.