Herokuish builds, deployment rework

[bobheadxi]

🎟️ Ticket(s): Closes #67

There's no overlap between this PR and the other one

---

👷 Changes

> Reworked Project Package

Reworked how deployments are managed using a new Deployment class, complete with mutex locks and an interface that can be mocked out to test the endpoints, as well as a wide range of minor tweaks and improvements for clarity and maintainability (I think) - this includes using option structs, inspired by the Docker client, to configure commands.

logs, err := deployment.Logs(project.LogOptions{
	Container: upReq.Container,
	Stream:    upReq.Stream,
}, cli)

Hoping to rework the client package in a similar manner (#140)

The Deployment class holds a whole bunch of parameters and settings pertaining to the state of the deployment.

type Deployment struct {
	project   string
	branch    string
	buildType string

	repo *git.Repository
	auth ssh.AuthMethod
	mux  sync.Mutex
}

The daemon maintains one global Deployer at a time that defines the API the daemon uses to control the project deployment.

type Deployer interface {
	Deploy(DeployOptions, *docker.Client, io.Writer) error
	Down(*docker.Client, io.Writer) error
	Destroy(*docker.Client, io.Writer) error

	Logs(LogOptions, *docker.Client) (io.ReadCloser, error)
	GetStatus(*docker.Client) (*DeploymentStatus, error)

	SetConfig(DeploymentConfig)
	GetBranch() string
	CompareRemotes(string) error
}

> Herokuish and Heroku Buildpacks

Daemon startup now downloads two things: docker-compose and herokuish. The latter is from the herokuish project which aims to provide a simulated Heroku build environment, which means any Heroku project should theoretically be able to be turned into a Heroku slug and deployed as a container. Here I try to deploy my Go project:

bobbook:calories robertlin$ inertia local up --stream
No deployment detected
Setting up project...
Cloning branch dev from git@github.com:bobheadxi/calories.git...
Shutting down active containers...
Setting up herokuish...
Building project...
-----> Go app detected89414000Z
-----> Fetching jq... done7700Z
-----> Fetching tq... done3500Z
-----> Installing go1.10999900Z
-----> Fetching go1.10.linux-amd64.tar.gz... done
-----> Fetching dep... done800Z
-----> Fetching any unsaved dependencies (dep ensure)
-----> Running: go install -v -tags heroku ./...
       github.com/bobheadxi/calories/vendor/github.com/lib/pq/oid
       github.com/bobheadxi/calories/config
       github.com/bobheadxi/calories/vendor/github.com/lib/pq
       github.com/bobheadxi/calories/facebook
       github.com/bobheadxi/calories/server
       github.com/bobheadxi/calories/bot
       github.com/bobheadxi/calories
-----> Discovering process types
       Procfile declares types -> web
Build exited with status 0
Saving build...
Starting up project...
�2018-04-05T06:47:14.726467900Z 2018/04/05 06:47:14 The following variables must be set in your deployment environment: $PORT $DATABASE_URL $FB_TOKEN $PAGE_ID
# that's pretty much a success lol, for my example at least

The Herokuish build is a two-step process:

1. Create and run a container, based off of the Herokuish image, that will turn the project into a "slug" using one of the available "buildpacks". Each buildpack is essentially a complex set of scripts designed to set up a project of a particular language, and the herokuish Docker image comes with the most popular ones preinstalled. Buildpack detection is done automatically and the process should be identical to that used by Heroku itself, so Heroku's documentation can be used for configuration. Programmatically, this step looks like this:

resp, _ := cli.ContainerCreate(
	ctx, &container.Config{
		Image: HerokuishVersion,
		Cmd:   []string{"/build"},
	},
	&container.HostConfig{
		Binds: []string{
			// "/tmp/app" is the directory herokuish looks
			// for during a build, so mount project there
			os.Getenv("HOME") + "/project:/tmp/app",
		},
	}, nil, "build",
)
cli.ContainerStart(ctx, resp.ID, types.ContainerStartOptions{})

2. Once the slug has been created, we need to commit the container and then execute that container again with a new entrypoint - the "start" command. This should get the project up and running in a separate container:

cli.ContainerCommit(ctx, resp.ID, types.ContainerCommitOptions{
    Reference: "inertia-build",
})
resp, _ = cli.ContainerCreate(ctx, &container.Config{
    Image: "inertia-build:latest",
    Cmd:   []string{"/start", "web"},
}, nil, nil, d.project)
cli.ContainerStart(ctx, resp.ID, types.ContainerStartOptions{})

These snippets have been trimmed of err checks, etc for clarity.

> Caveats

• currently, only one process is started (web), which is one of many that Procfile supports. I think Procfile also supports docker-compose-style multi-process startups, but I think it might be best to leave that for another PR
• Herokuish container creation and commit is awfully slow on my laptop, maybe it's just me
• compiled Herokuish image sizes can be massive:

inertia-build          latest              e19fa61ee5a1        12 minutes ago      1.29GB
ubclaunchpad/inertia   test                ec12b33c0780        21 minutes ago      22.5MB

• the Herokuish image itself is quite large:

gliderlabs/herokuish   v0.4.0              ff5bdc126acc        3 weeks ago         893MB
docker/compose         1.19.0              e06b58ce9de2        7 weeks ago         107MB

> Resources

Wow this PR was a pain. Some interesting resources I spent way too much time digging through:

• herokuish documentation for how the CLI and Docker image works
• herokuish source code for more specifics, especially in regards to how /start works
• some herokuish issues for other people asking similar questions about how to make Docker images using herokuish
• dokku source code, most notably this PR, where the dokku team first started using Herokuish. On a side note, dokku's vision is quite similar to Inertia's, but we are simpler 😎
• Docker CLI source code for how the FUUUUUU**********KK docker run works (for some reason this took me forever to get write, not as intuitive via the Golang client as it sounds and looks 😢 - perhaps a hint that we need a faster and more iterative testing process for deployment code)
• this guy's herokuish workflow - someone's script for building and distributing herokuish-compiled Docker images
• heroku documentation for how Procfile works and how to set up projects for Heroku

🔦 Testing Instructions

In a Heroku-configured project:

inertia init
inertia remote add local
inertia local init
inertia local up --stream

[PiggySpeed]

Wow this is actually very impressive 😮
You're a beast @bobheadxi

[PiggySpeed]

I'm not sure whether it's necessary at this time, but I wonder whether it's a good idea to turn our build types into the go-equivalent of an enum?

[bobheadxi]

enums in Go seem interesting but I think this would be a good idea

[PiggySpeed]

Was looking at the same article as well 😝

If we go with enums, would we put it in a global constants file; lets say... under common?

[bobheadxi]

Yeah, common would be the right place, since the setting is determined clientside 👍

[bobheadxi]

Going to leave out the enum for now since it's tricky to handle alongside TOML conversion.