Use Make to build binaries

[riobard]

Run make to build for your current system, and run make all to build for Darwin, Linux, and Windows. You can also make darwin, make windows and make linux individually.

[gmlewis]

Hmmm... adding a Makefile to a Go project? This doesn't seem very portable or Go-like.
Why do we need this?

[tianon]

I am definitely also confused, and that build.sh script is used by the Dockerfile build method, so it can't be removed. ;)

[tianon]

Also, why not just do something like:

default:
    ./build.sh

all: darwin linux windows

%:
    GOOS=$@ ./build.sh

[riobard]

Oops…

I wrote the Makefile to simplify releasing binaries. For each release we need to build three binaries according to a specific naming convention (see https://github.com/boot2docker/boot2docker-cli/releases). go build does not solve that, and build.sh does not take additional command line arguments.

I'll fix the problems you mentioned now.

[tianon]

Easy, just add "$@" to the end of the go build line in build.sh :)

[riobard]

@tianon I also need the version number for naming… :|

[tianon]

Indeed, still simple (but more importantly, still extremely DRY):

diff --git a/Makefile b/Makefile
new file mode 100644
index 0000000..433db0d
--- /dev/null
+++ b/Makefile
@@ -0,0 +1,11 @@
+default:
+   ./build.sh
+
+all: darwin linux windows
+   @true # stop "all" from matching "%" later
+
+GOARCH := amd64
+VERSION = $(shell cat VERSION)
+
+%:
+   GOOS=$@ GOARCH=$(GOARCH) ./build.sh -o boot2docker-cli-$(VERSION)-$@-$(GOARCH)$(if $(filter windows, $@),.exe)
diff --git a/build.sh b/build.sh
index 10ab07b..6b05be2 100755
--- a/build.sh
+++ b/build.sh
@@ -5,4 +5,4 @@ version="$(cat VERSION)"
 gitSha="$(git rev-parse --short HEAD)"

 set -x
-exec go build -ldflags "-X main.Version $version -X main.GitSHA $gitSha"
+exec go build -ldflags "-X main.Version $version -X main.GitSHA $gitSha" "$@"

# make all
GOOS=darwin GOARCH=amd64 ./build.sh -o boot2docker-cli-v0.8.0-darwin-amd64
+ exec go build -ldflags -X main.Version v0.8.0 -X main.GitSHA 408532a -o boot2docker-cli-v0.8.0-darwin-amd64
GOOS=linux GOARCH=amd64 ./build.sh -o boot2docker-cli-v0.8.0-linux-amd64
+ exec go build -ldflags -X main.Version v0.8.0 -X main.GitSHA 408532a -o boot2docker-cli-v0.8.0-linux-amd64
GOOS=windows GOARCH=amd64 ./build.sh -o boot2docker-cli-v0.8.0-windows-amd64.exe
+ exec go build -ldflags -X main.Version v0.8.0 -X main.GitSHA 408532a -o boot2docker-cli-v0.8.0-windows-amd64.exe

[riobard]

BLACK MAGIC! @tianon you're the best!

Never know you could do conditionals in Makefile like that… Lesson learned! :D

[tianon]

[tianon]

:)

[riobard]

I'm gonna go back and re-learn Make!

[riobard]

@tianon I noticed quite a few Docker builds tend to prefer shell script over Makefiles, but I'm wondering why?

[tianon]

I would guess that it's because Bash is so much easier to wrap your head around than GNU Make (since Bash is an actual programming/scripting language, where GNU Make is a lot of strange incantations). Personally, I'm a fan of both, but I love using them together especially for cool stuff like this. :)

Needing a Makefile is also sometimes a sign that your build/project is getting too complicated, so there's some natural aversion there. :)

[tianon]

Oh, and of course, since the crux of this is basically exactly what I created, LGTM now, but I think it ought to have at least two of @SvenDowideit @gmlewis and/or @steeve review it also, since I'm "too close to the fire" now, so to speak. :)

[riobard]

Thanks for the explanation! That confirmed my doubt. I actually wrote a Bash script that handles an entire Docker build (by pumping commands to Docker via stdin instead of storing them in a Dockerfile, just to have some flexibility :). Makefile still feels a bit foreign IMHO… Maybe I should extend build.sh instead of relying on make?

[SvenDowideit]

@tianon @riobard I'm confused.

I use the a Docker container to build:

docker build -t boot2docker-golang .
docker rm boot2docker-buildcli
docker run -t -i -e GOOS=windows -e GOARCH=amd64 --name boot2docker-buildcli boot2docker-golang
docker cp boot2docker-buildcli:/go/src/github.com/boot2docker/boot2docker-cli/boot2docker-cli.exe .

I'm assuming this does not currently work for OSX, but I would be much more comfortable with the Linux and Windows builds being done in a Container - so we have consistent builds for everyone.

(so no, I do not like.)

[tianon]

@SvenDowideit of course - note my example output, which occurred in a container ;)

This makes it easy to "make all" in just one container instead of running three separate containers to perform that same task.

[riobard]

@SvenDowideit Yep, basically it's to make the job easier and to enforce the naming convention for the binary releases. Otherwise you'll have to repeat the Docker build for different OS and also remember to rename the binaries copied out of the container correctly each time.

[gmlewis]

LGTM

[steeve]

LGTM fosho'! (From the airport)—
Steeve Morin

On Wed, Apr 9, 2014 at 4:58 AM, Sven Dowideit notifications@github.com
wrote:

@tianon @riobard I'm confused.
I use the a Docker container to build:

docker build -t boot2docker-golang .
docker rm boot2docker-buildcli
docker run -t -i -e GOOS=windows -e GOARCH=amd64 --name boot2docker-buildcli boot2docker-golang
docker cp boot2docker-buildcli:/go/src/github.com/boot2docker/boot2docker-cli/boot2docker-cli.exe .

I'm assuming this does not currently work for OSX, but I would be much more comfortable with the Linux and Windows builds being done in a Container - so we have consistent builds for everyone.

Reply to this email directly or view it on GitHub:
#88 (comment)

[SvenDowideit]

So there's a build process change - can you please update the README.md to match?

[riobard]

@SvenDowideit I didn't intend to replace the existing Docker build with the Make approach here. This is strictly a convenience tool for us to release binaries, i.e. mostly likely it'll be used just by you and me, until we figure out how to do CI later.

Or do you want others to use it too? In the case, I can definitely update the README. We'll have to mention the caveat of building for OS X from Docker anyway due to the TLS issue.

Please let me know which you'd prefer.

[SvenDowideit]

@riobard for repeat ability, it would be better if we all only build one way (tho the OSX case is a problem until we fix it). I don't like developing with a build system that is not identical to the release one - it introduces unknowns that we just don't need.

[riobard]

You're still confused. All the Makefile does is equivalent to

1. repeat the Docker build three times for supported OS, and
2. rename the built binaries correctly.

There is no different builds.

[SvenDowideit]

excellent. me being confused is good.

so - can we please update the README.md to use the makefile? that way only experts will do the typing that might lead to errors :)

[riobard]

Well, now the build process is completely streamlined and simplified.

@SvenDowideit For a Docker build you can now just run make dockerbuild and all the details will be taken care of for you :D

@tianon You might want to take a second look at the Makefile and please let me know if you have any suggestions for improvement! :)

[riobard]

README is also updated to reflect the changes.

[riobard]

If you are unclear about the new build process, please read the new README https://github.com/riobard/boot2docker-cli/tree/Makefile

[SvenDowideit]

much awesome. thankyou :) I'll test tomorrow when I stop getting kicked by heartblead related leftovers of a past life

[riobard]

No problem! :) I tested it on OS X and it works fine. I don't think the build process works on Windows though (no shell/make).

This Heartbleed thing is so f*cked up…

[tianon]

LGTM

[steeve]

Sorry guys I'm locked out of my gh account for tech reasons (in the us with a different sim, thus 2 factor is out).

But that lgtm
—
Steeve Morin

On Mon, Apr 14, 2014 at 9:44 AM, Tianon Gravi notifications@github.com
wrote:

LGTM

Reply to this email directly or view it on GitHub:
#88 (comment)

[riobard]

@SvenDowideit Have you run into any issues with this PR so far?

[SvenDowideit]

@riobard just tried - its missing something:

boot2docker-cli(pr/88) $ make all
GOOS=darwin GOARCH=amd64 go build -ldflags "-X main.Version v0.8.0 -X main.GitSHA a29c5fb" -o boot2docker-cli-v0.8.0-darwin-amd64
cmds.go:12:2: cannot find package "github.com/boot2docker/boot2docker-cli/virtualbox" in any of:
        /usr/lib/go/src/pkg/github.com/boot2docker/boot2docker-cli/virtualbox (from $GOROOT)
        /home/sven/usr/lib/go/src/github.com/boot2docker/boot2docker-cli/virtualbox (from $GOPATH)
config.go:14:2: cannot find package "github.com/ogier/pflag" in any of:
        /usr/lib/go/src/pkg/github.com/ogier/pflag (from $GOROOT)
        /home/sven/usr/lib/go/src/github.com/ogier/pflag (from $GOPATH)
make: *** [darwin] Error 1

yup - I expect to be able to

git clone
cd b2d-cli
make all

[riobard]

git clone isn't the right way to get the source and dependencies. go get github.com/boot2docker/boot2docker-cli is the way to go (pun intended :). https://github.com/riobard/boot2docker-cli/tree/Makefile#install-from-source

[tianon]

We could fix that with a go get -u -d in the Makefile, but this point is certainly a debatable one with any Go project.

[riobard]

@tianon I think the major problem is we have to put the source into the right place in $GOPATH. git clone directly won't cut.

[riobard]

Of course another possibility is we manage per-project $GOPATH. I did that before for another project, but the consensus seems to be that it isn't idiomatic Go.

[SvenDowideit]

seriously, if go idiom is different from well established idioms, then its only going to slow down casual contribution and should be ignored.

this is a github project - so its expected that one will get the code using git and then make.

I'm somewhat sure that typing go get github.com/boot2docker/boot2docker-cli is not going to get the code from my github fork, then allow me to make changes and push them back to be PR'd and merged. that is the main use case the Makefile is for.

[riobard]

Well, every language has its idioms and well-established best practices. A Node project on GitHub expects different procedures from a Rails project. There're no standard "GitHub projects" and although we take git for granted (which isn't the case for Go anyway because many Go dependences are on Mercurial), make certainly isn't the norm. Even C/C++ projects which do use make require you do the configuration differently.

At this point, putting everything correctly under $GOPATH is the recommended way for Go projects, and you are expected to use go get to handle that. Most Go projects do that. I didn't like that approach in the beginning, and I still question some of the rationale behind it now, but fighting against it was pretty painful. I tried before, and the result was nasty. Now I just give up and follow suit.

As for the best way to handle forks and PRs on GitHub, @tianon once told me his setup and I found it really nice. I've been using his approach for all my GitHub projects ever since. I wrote a short comment explaining how here #70 (comment) You can give a try and see if you like it.

BTW, it's not the Makefile's fault that you have this problem. The problem exists for the old build.sh script too. This PR is basically make the rest of the routine build job easier and more consistent.

[riobard]

PS: make dockerbuild doesn't have this problem as it copies the project into the Docker container in the correct place and build from there.

[tianon]

Maybe the dockerbuild should be our default? It'll be the most consistent, for sure.

[riobard]

Docker build is gonna take REALLY long time initially. I'm not sure it's a
good idea to make it the default as local build is much faster to iterate
during development. I actually use make install as my default :)

For deployment we should use docker build for sure (except for OS X).

On Thursday, April 17, 2014, Tianon Gravi notifications@github.com wrote:

Maybe the dockerbuild should be our default? It'll be the most consistent,
for sure.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/88#issuecomment-40681038
.

[SvenDowideit]

we want the default builds to always use the container - only rare and odd people would realise they can build outside - and once we fix the bugs in cross building, then building outside the container creates an unsupported

Docker is about doing things in containers for consistency. GitHub is to simplify contribution.

The other options are there for experts - so don't need to be defaults. make all should work irrespective of how you got the code - tgz, github clone, or whatever.

[SvenDowideit]

If initial container build is a problem, we can just make it a trusted build and docker pull it. its certainly not the fastest :)

[SvenDowideit]

now that its done - yes, swap it round so make and make all and make install all use the container - and make the-risky-way or something similarly discouraging to the uninitiated does it with whatever random stuff you have installed on your box.

go-zi-ens can still do their personal, re-invention of make by typing go get github.com/boot2docker/boot2docker-cli - pretty much every immature new language builds their own make - and then 20 years later works out that its got problems that make fixed before they were born (and seriously, those guys know this too)

[riobard]

Now make defaults to Docker build. make install is gone (unless we have an idea where to install to?) and replaced by make goinstall for native Go build.

[tianon]

Seems to work OK here; LGTM.