Skip to content
Permalink
Browse files

Episode 28 - running Athens for Go Modules (#178)

* Episode 28 - running Athens for Go Modules

* Adding more to the README

* reordering bullets

* adding webpage

* adding YouTube ID
  • Loading branch information...
arschles committed Aug 9, 2019
1 parent 28733d5 commit 8296b2c92e7a98c7d7cb40bc8add68806deb0750
@@ -0,0 +1,4 @@
tmp
public/assets
hugo
episode28
@@ -0,0 +1,93 @@
# Using Athens to Serve Your Go Modules

Go in 5 Minutes, episode 28.

In [episode 27](https://www.goin5minutes.com/screencast/episode_27_intro_to_modules/), we talked about how to use [Go Modules](https://github.com/golang/go/wiki/Modules) to manage dependencies, but there's more to the story!

Go modules come with module servers that you can download your dependencies from too! The servers themselves are pretty cool, but here's why you should consider using one:

- Downloading dependencies will almost always be faster (sometimes up to 4x faster)
- Think about how much faster CI runs can be ;)
- You can avoid broken builds when someone deletes a commit, tag or repository

I'm going to explain how module servers work, introduce Athens, and show how to use it in action.

Check out the screencast for more!

# How To Run Athens

We're going to run an Athens and build a little server with it as our module proxy. We'll actually do _two_ builds here. The first will be to run Athens when connected to the internet, and the second will be to do it when disconnected from the internet - to show how well Athens works in an isolated environment (like inside a firewall).

These instructions are for Linux/Mac OS X systems.

## Build #1: Build With Athens and an Upstream VCS

Athens maintains its own database of modules. When you do a `go get` and request a module from Athens, it checks its database and sends the module back to you if it's there. If not, then Athens does this:

1. Fetch the module from version control
1. Store it in the database
1. Send it back to you

Since we're starting Athens with nothing in its storage, every dependency we request in this build will make it download modules from version control.

>By the way, you can also configure Athens to download from module mirrors like [proxy.golang.org](https://proxy.golang.org) or [gocenter.io](https://gocenter.io), instead of version control hosts!
### Run The Server!

We try hard to make it easy to run your own Athens. See [here](https://docs.gomods.io/install) for instructions for running the server a few different ways. Today, we're going to use [Docker](https://www.docker.com/) to run ours.

First, run this to start Athens up:

```console
$ docker run -p 3000:3000 -e GO_ENV=development -e ATHENS_GO_GET_WORKERS=5 gomods/athens:v0.5.0
```

And then to set your `GOPROXY` environment variable to tell modules to use the local server:

```console
$ export GOPROXY=http://localhost:3000
```

Also, the Go tool keeps a read-only on-disk cache of every module version you've downloaded for any build. To make it read-only, it stores each file in the cache with `-r--r--r--` permissions. Since that's the case, you need to use `sudo` to clear the cache.

```console
$ sudo rm -rf $(go env GOPATH)/pkg/mod
```

And then build and run the server!

```console
$ go run .
```

## Second Way: Use Your Athens While Offline :scream:

Did I mention that Athens stores your dependencies in storage? Well, it actually stores your dependencies in storage _forever_! That means that you can build your code without access to the internet. And it's faster. Let's do a build and see.

First, make sure not to shut down the Athens server from last time - its storage is inside the Docker container!

Next, clear out your cache again:

```console
$ sudo rm -rf $(go env GOPATH)/pkg/mod
```

And then, **shut down your internet connection** :see_no_evil:.

And finally, do the build & run again!

```console
$ go run .
```

And you're done!

# Show Notes

- [The module download protocol](https://docs.gomods.io/intro/protocol/)
- [The Athens Project](https://docs.gomods.io)
- [Joining the Gophers Slack](https://invite.slack.golangbridge.org/)
- Come say hi in the `#athens` channel!
- [List of all available module proxies](https://github.com/golang/go/wiki/Modules#are-there-always-on-module-repositories-and-enterprise-proxies)
- [The Go team's module proxy](https://proxy.golang.org)
- [The JFrog public module proxy](https://gocenter.io)
@@ -0,0 +1,5 @@
module github.com/arschles/go-in-5-minutes/episode27

go 1.12

require github.com/gin-gonic/gin v1.4.0
@@ -0,0 +1,28 @@
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/gin-contrib/sse v0.0.0-20190301062529-5545eab6dad3 h1:t8FVkw33L+wilf2QiWkw0UV77qRpcH/JHPKGpKa2E8g=
github.com/gin-contrib/sse v0.0.0-20190301062529-5545eab6dad3/go.mod h1:VJ0WA2NBN22VlZ2dKZQPAPnyWw5XTlK1KymzLKsr59s=
github.com/gin-gonic/gin v1.4.0 h1:3tMoCCfM7ppqsR0ptz/wi1impNpT7/9wQtMZ8lr1mCQ=
github.com/gin-gonic/gin v1.4.0/go.mod h1:OW2EZn3DO8Ln9oIKOvM++LBO+5UPHJJDH72/q/3rZdM=
github.com/golang/protobuf v1.3.1 h1:YF8+flBXS5eO826T4nzqPrxfhQThhXl0YzfuUPu4SBg=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=
github.com/mattn/go-isatty v0.0.7 h1:UvyT9uN+3r7yLEYSlJsbQGdsaB/a0DlgWP3pql6iwOc=
github.com/mattn/go-isatty v0.0.7/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/ugorji/go v1.1.4 h1:j4s+tAvLfL3bZyefP2SEWmhBzmuIlH/eqNuPdFPgngw=
github.com/ugorji/go v1.1.4/go.mod h1:uQMGLiO92mf5W77hV/PUCpI3pbzQx3CRekS0kk+RGrc=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/go-playground/assert.v1 v1.2.1/go.mod h1:9RXL0bg/zibRAgZUYszZSwO/z8Y/a8bDuhia5mkpMnE=
gopkg.in/go-playground/validator.v8 v8.18.2 h1:lFB4DoMU6B626w8ny76MV7VX6W2VHct2GVOI3xgiMrQ=
gopkg.in/go-playground/validator.v8 v8.18.2/go.mod h1:RX2a/7Ha8BgOhfk7j780h4/u/RRjR0eouCJSH80/M2Y=
gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,31 @@
package main

import (
"fmt"
"net/http"

"github.com/gin-gonic/gin"
)

func kittyHandler(c *gin.Context) {
cats := []string{
"/img/kitties/1.jpg",
"/img/kitties/2.jpg",
"/img/kitties/3.jpg",
"/img/kitties/4.jpg",
"/img/kitties/5.jpg",
"/img/kitties/6.jpg",
}
htmlStr := fmt.Sprintf(`<html>
<head></head>
<body>
<center>
<img src="%s" width="1000"/>
</center>
</body>
</html>`, randStr(cats))
c.Writer.Header().Set("Content-Type", "text/html")
c.Writer.WriteHeader(http.StatusOK)
c.Writer.WriteHeader(http.StatusOK)
c.Writer.Write([]byte(htmlStr))
}
@@ -0,0 +1,19 @@
package main

import (
"math/rand"
"time"

"github.com/gin-gonic/gin"
)

func main() {
rand.Seed(time.Now().UnixNano())
router := gin.Default()

router.Static("/img", "./img")
router.GET("/", mainHandler)
router.GET("/kitty", kittyHandler)
router.GET("/pup", pupHandler)
router.Run(":8080")
}
@@ -0,0 +1,24 @@
package main

import (
"net/http"

"github.com/gin-gonic/gin"
)

func mainHandler(c *gin.Context) {
htmlStr := `<html>
<head></head>
<body>
<center>
<h1>Hey Gophers!</h1>
<h3>What better way to demo Go Modules than with cat and dog pictures?</h3>
<p><a href="/kitty">Cats</a></p>
<p><a href="/pup">Dogs</a></p>
</center>
</body>
</html>`
c.Writer.Header().Set("Content-Type", "text/html")
c.Writer.WriteHeader(http.StatusOK)
c.Writer.Write([]byte(htmlStr))
}
@@ -0,0 +1,33 @@
package main

import (
"fmt"
"net/http"

"github.com/gin-gonic/gin"
)

func pupHandler(c *gin.Context) {
dogs := []string{
"/img/pups/1.jpg",
"/img/pups/2.jpg",
"/img/pups/3.png",
"/img/pups/3.png",
"/img/pups/4.png",
"/img/pups/5.png",
"/img/pups/6.jpg",
"/img/pups/7.jpg",
}

htmlStr := fmt.Sprintf(`<html>
<head></head>
<body>
<center>
<img src="%s" width="1000"/>
</center>
</body>
</html>`, randStr(dogs))
c.Writer.Header().Set("Content-Type", "text/html")
c.Writer.WriteHeader(http.StatusOK)
c.Writer.Write([]byte(htmlStr))
}
@@ -0,0 +1,15 @@
package main

import (
"math/rand"
"time"
)

func init() {
rand.Seed(time.Now().UnixNano())
}

func randStr(slc []string) string {
idx := rand.Intn(len(slc))
return slc[idx]
}
@@ -0,0 +1,5 @@
#!/bin/sh

set -eou pipefail

docker run -p 3000:3000 -e GO_ENV=development -e ATHENS_GO_GET_WORKERS=5 gomods/athens:v0.5.0
@@ -0,0 +1,30 @@
+++
type = "screencast"
title = "Using Modules with the Athens Project!"
teaser = "Module API servers are a huge boost for Go dependencies. Check out why!"
author = "Aaron Schlesinger"
date = "2019-08-09T13:10:24-07:00"
+++

_Episode 28_

In [episode 27](https://www.goin5minutes.com/screencast/episode_27_intro_to_modules/), we talked about how to use [Go Modules](https://github.com/golang/go/wiki/Modules) to manage dependencies, but there's more to the story!

Check out how module servers and Athens fit into the dependencies ecosystem.

<!--more-->

Go modules come with module servers that you can download your dependencies from too! The servers themselves are pretty cool, but here's why you should consider using one:

- Downloading dependencies will almost always be faster (sometimes up to 4x faster)
- Think about how much faster CI runs can be ;)
- You can avoid broken builds when someone deletes a commit, tag or repository

I'm going to explain how module servers work, introduce Athens, and show how to use it in action.

Keep on rockin', Gophers!

<iframe width="560" height="315" src="https://www.youtube.com/embed/P3P9NINDW1k" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>

Check out the example code [on Github](https://github.com/arschles/go-in-5-minutes/tree/master/episode28).

0 comments on commit 8296b2c

Please sign in to comment.
You can’t perform that action at this time.