Permalink
Browse files

Rehauled build system and integration testing

- USE_CONTAINER allow to seamlessly run targets inside or outside containers
- all build calls have been harmonized, honoring the same env variables
- contributing doc has been streamlined according to that
- kill the distinction between remote and local docker builds
- got rid of some of the byzantine calls in various asorted scripts
- support for static build, debug builds, verbose

Signed-off-by: Olivier Gambier <olivier@docker.com>
  • Loading branch information...
Olivier Gambier authored and dmp42 committed Sep 11, 2015
1 parent 0aca429 commit 5fbfbe1ff7f6383bc8b9367d93028c2ce1b1cab1
View
@@ -1,2 +1,4 @@
docker-machine*
.git
*.log
bin
cover
View
@@ -2,3 +2,5 @@ docker-machine*
*.log
*.iml
.idea/
bin
cover
View
@@ -1,15 +1,13 @@
language: go
sudo: false
go:
- 1.4.2
- 1.5
install:
- export GOPATH=${TRAVIS_BUILD_DIR}/Godeps/_workspace:$GOPATH
- export PATH=${TRAVIS_BUILD_DIR}/Godeps/_workspace/bin:$PATH
- go get -t -v ./...
- go get golang.org/x/tools/cmd/cover
- go get github.com/mattn/goveralls
script:
- script/validate-dco
- script/validate-gofmt
- go test -v -short ./...
- script/generate-coverage report
- make dco
- make fmt
# - make lint
# - make vet
- make test-short
- make test-long
- make coverage-send
View
@@ -2,6 +2,7 @@
[![GoDoc](https://godoc.org/github.com/docker/machine?status.png)](https://godoc.org/github.com/docker/machine)
[![Build Status](https://travis-ci.org/docker/machine.svg?branch=master)](https://travis-ci.org/docker/machine)
[![Coverage Status](https://coveralls.io/repos/docker/machine/badge.svg?branch=upstream-master&service=github)](https://coveralls.io/github/docker/machine?branch=upstream-master)
Want to hack on Machine? Awesome! Here are instructions to get you
started.
@@ -13,108 +14,139 @@ Docker does things, you'll feel right at home.
Otherwise, please read [Docker's contributions
guidelines](https://github.com/docker/docker/blob/master/CONTRIBUTING.md).
# Building
# Building using docker
The requirements to build Machine are:
1. A running instance of Docker
1. A running instance of Docker (or alternatively a golang 1.5 development environment)
2. The `bash` shell
3. [Make](https://www.gnu.org/software/make/)
To build, run:
Call `export USE_CONTAINER=true` to instruct the build system to use containers to build.
If you want to build natively using golang instead, don't set this variable.
## Building
To build the docker-machine binary, simply run:
$ make
From the Machine repository's root. Machine will run the build inside of a
Docker container and the compiled binaries will appear in the project directory
on the host.
From the Machine repository's root. You will now find a `docker-machine`
binary at the root of the project.
By default, Machine will run a build which cross-compiles binaries for a variety
of architectures and operating systems. If you know that you are only compiling
for a particular architecture and/or operating system, you can speed up the
compile by setting the standard `GOOS` and/or `GOARCH` environment variables to
deviate from the defaults.
You may call:
For instance, if you only want to compile for use on OS X with the x86_64 arch,
run:
$ make clean
$ GOOS=darwin GOARCH=amd64 make
to clean-up build results.
If you have any questions we're in #docker-machine on Freenode.
## Tests and validation
## Remote Build
To run basic validation (dco, fmt), and the project unit tests, call:
You can use Machine to build Machine:
$ make test
$ GOOS=darwin GOARCH=amd64 make remote
If you want more indepth validation (vet, lint), and all tests with race detection, call:
The difference is this works even if you don't have local docker daemon, but
instead a docker host that you [set up using Machine](https://github.com/docker/machine).
$ make validate
## Unit Tests
If you make a pull request, it is highly encouraged that you submit tests for
the code that you have added or modified in the same pull request.
To run the unit tests for the whole project, using the following script:
## Code Coverage
$ script/test
To generate an html code coverage report of the Machine codebase, run:
This will run the unit tests inside of a container, so you don't have to worry
about configuring your environment properly before doing so.
make coverage-serve
To run the unit tests for only a specific subdirectory of the project, you can
pass an argument to that script to specify which directory, e.g.:
And navigate to http://localhost:8000 (hit `CTRL+C` to stop the server).
$ script/test ./drivers/amazonec2
Alternatively, if you are building natively, you can simply run:
If you make a pull request, it is highly encouraged that you submit tests for
the code that you have added or modified in the same pull request.
make coverage-html
## Code Coverage
This will generate and open the report file:
Machine includes a script to check for missing `*_test.go` files and to generate
an [HTML-based representation of which code is covered by tests](http://blog.golang.org/cover#TOC_5.).
![](/docs/img/coverage.png)
To run the code coverage script, execute:
## List of all targets
```console
$ ./script/coverage serve
```
### High-level targets
You will see the results of the code coverage check as they come in.
make clean
make build
make test
make validate
This will also generate the code coverage website and serve it from a container
on port 8000. By default, `/` will show you the source files from the base
directory, and you can navigate to the coverage for any particular subdirectory
of the Docker Machine repo's root by going to that path. For instance, to see
the coverage for the VirtualBox driver's package, browse to `/drivers/virtualbox`.
### Build targets
![](/docs/img/coverage.png)
Build a single, native machine binary:
make build-simple
Build for all supported oses and architectures (binaries will be in the `bin` project subfolder):
You can hit `CTRL+C` to stop the server.
make build-x
Build for a specific list of oses and architectures:
TARGET_OS=linux TARGET_ARCH="amd64 arm" make build-x
You can further control build options through the following environment variables:
DEBUG=true # enable debug build
STATIC=true # build static (note: when cross-compiling, the build is always static)
VERBOSE=true # verbose output
PARALLEL=X # lets you control build parallelism when cross-compiling multiple builds
PREFIX=folder
Scrub build results:
make build-clean
### Coverage targets
make coverage-html
make coverage-serve
make coverage-send
make coverage-generate
make coverage-clean
### Tests targets
make test-short
make test-long
make test-integration
### Validation targets
make fmt
make vet
make lint
make dco
## Integration Tests
### Setup
We utilize [BATS](https://github.com/sstephenson/bats) for integration testing.
This runs tests against the generated binary. To use, first make sure to
[install BATS](https://github.com/sstephenson/bats). Then run `./script/build`
to generate the binary for your system.
We use [BATS](https://github.com/sstephenson/bats) for integration testing, so,
first make sure to [install it](https://github.com/sstephenson/bats#installing-bats-from-source).
### Basic Usage
Once you have the binary, the integration tests can be invoked using the
`test/integration/run-bats.sh` wrapper script.
Integration tests can be invoked calling `make test-integration`.
Using this wrapper script, you can invoke a test or subset of tests for a
particular driver. To set the driver, use the `DRIVER` environment variable.
:warn: you cannot run integration test inside a container for now.
Be sure to unset the `USE_CONTAINER` env variable if you set it earlier, or alternatively
call directly `./test/integration/run-bats.sh` instead of `make test-integration`.
The following examples are all shown relative to the project's root directory,
but you should be able to invoke them from any directory without issue.
You can invoke a test or subset of tests for a particular driver.
To set the driver, use the `DRIVER` environment variable.
To invoke just one test:
```console
$ DRIVER=virtualbox ./test/integration/run-bats.sh test/integration/core/core-commands.bats
$ DRIVER=virtualbox make test-integration test/integration/core/core-commands.bats
✓ virtualbox: machine should not exist
✓ virtualbox: create
✓ virtualbox: ls
@@ -141,23 +173,17 @@ Successfully removed bats-virtualbox-test
To invoke a shared test with a different driver:
```console
$ DRIVER=digitalocean ./test/integration/run-bats.sh test/integration/core/core-commands.bats
$ DRIVER=digitalocean make test-integration test/integration/core/core-commands.bats
...
```
To invoke a directory of tests recursively:
```console
$ DRIVER=virtualbox ./test/integration/run-bats.sh test/integration/core/
$ DRIVER=virtualbox make test-integration test/integration/core/
...
```
If you want to invoke a group of tests across two or more different drivers at
once (e.g. every test in the `drivers` directory), at the time of writing there
is no first-class support to do so - you will have to write your own wrapper
scripts, bash loops, etc. However, in the future, this may gain first-class
support as usage patterns become more clear.
### Extra Create Arguments
In some cases, for instance to test the creation of a specific base OS (e.g.
@@ -175,7 +201,7 @@ $ DRIVER=amazonec2 \
AWS_ACCESS_KEY_ID=zzzzzzzzzzzzzzzz \
AWS_AMI=ami-12663b7a \
AWS_SSH_USER=ec2-user \
./test/integration/run-bats.sh test/integration/core
make test-integration test/integration/core
```
in order to run the core tests on Red Hat Enterprise Linux on Amazon.
View
@@ -1,15 +1,11 @@
FROM golang:1.5
# TODO: Vendor these `go get` commands using Godep.
RUN go get github.com/mitchellh/gox
RUN go get github.com/aktau/github-release
RUN go get github.com/tools/godep
RUN go get golang.org/x/tools/cmd/cover
ENV GOPATH /go/src/github.com/docker/machine/Godeps/_workspace:/go
ENV MACHINE_BINARY /go/src/github.com/docker/machine/docker-machine
ENV USER root
RUN go get github.com/mitchellh/gox \
github.com/golang/lint/golint \
github.com/mattn/goveralls \
golang.org/x/tools/cover \
github.com/aktau/github-release
WORKDIR /go/src/github.com/docker/machine
ADD . /go/src/github.com/docker/machine
ADD . /go/src/github.com/docker/machine
View
@@ -1,26 +1,31 @@
.PHONY: test validate-dco validate-gofmt
default: build
remote: build-remote
test:
script/test
validate-dco:
script/validate-dco
validate-gofmt:
script/validate-gofmt
validate: validate-dco validate-gofmt test
build: clean
script/build
build-remote: clean
script/build-remote
clean:
rm -f docker-machine_*
rm -rf Godeps/_workspace/pkg
# Plain make targets if not requested inside a container
ifeq ($(USE_CONTAINER),)
include Makefile.inc
include mk/main.mk
else
# Otherwise, with docker, swallow all targets and forward into a container
DOCKER_IMAGE_NAME := "docker-machine-build"
DOCKER_CONTAINER_NAME := "docker-machine-build-container"
%:
@docker build -t $(DOCKER_IMAGE_NAME) .
@test -z '$(shell docker ps -a | grep $(DOCKER_CONTAINER_NAME))' || docker rm -f $(DOCKER_CONTAINER_NAME)
@docker run --name $(DOCKER_CONTAINER_NAME) \
-e DEBUG \
-e STATIC \
-e VERBOSE \
-e BUILDTAGS \
-e PARALLEL \
-e COVERAGE_DIR \
-e TARGET_OS \
-e TARGET_ARCH \
-e PREFIX \
$(DOCKER_IMAGE_NAME) \
make $@
@test ! -d bin || rm -Rf bin
@test -z "$(findstring build,$@)" || docker cp $(DOCKER_CONTAINER_NAME):/go/src/github.com/docker/machine/bin bin
endif
View
@@ -0,0 +1,39 @@
# Project name, used to name the binaries
PKG_NAME := docker-machine
GH_USER ?= docker
GH_REPO ?= machine
# If true, disable optimizations and does NOT strip the binary
DEBUG ?=
# If true, "build" will produce a static binary (cross compile always produce static build regardless)
STATIC ?=
# If true, turn on verbose output for build
VERBOSE ?=
# Build tags
BUILDTAGS ?=
# Adjust parallelism for gox
PARALLEL ?= -1
# Coverage default directory
COVERAGE_DIR ?= cover
# Whether to perform targets inside a docker container, or natively on the host
USE_CONTAINER ?=
# List of cross compilation targets
ifeq ($(TARGET_OS),)
TARGET_OS := darwin freebsd linux windows
endif
ifeq ($(TARGET_ARCH),)
TARGET_ARCH := amd64 arm 386
endif
# Output prefix, defaults to local directory if not specified
ifeq ($(PREFIX),)
PREFIX := $(shell pwd)
endif
default: build
clean: coverage-clean build-clean
build: build-simple
test: dco fmt test-short
validate: dco fmt vet lint test-short test-long
Oops, something went wrong.

2 comments on commit 5fbfbe1

@kacole2

This comment has been minimized.

Show comment
Hide comment
@kacole2

kacole2 Sep 23, 2015

Contributor

is there a reason the make remote was left out? I use that as my primary build since i'm developing on my mac and don't want to garbage it up with packages.
TIA

Contributor

kacole2 replied Sep 23, 2015

is there a reason the make remote was left out? I use that as my primary build since i'm developing on my mac and don't want to garbage it up with packages.
TIA

@dmp42

This comment has been minimized.

Show comment
Hide comment
@dmp42

dmp42 Sep 24, 2015

Member

There is no longer a difference between building remote or local.
You should be able to just USE_CONTAINER=1 make build and it will build against your docker engine, no matter "where" it is.

Member

dmp42 replied Sep 24, 2015

There is no longer a difference between building remote or local.
You should be able to just USE_CONTAINER=1 make build and it will build against your docker engine, no matter "where" it is.

Please sign in to comment.