build.sh does not handle dependencies

[sjmudd]

Installation referenced https://github.com/datacharmer/dbdeployer/ basically gets you to pull down a pre-built set of binaries. That's fine but I like to build my go binaries directly from source. Other projects I use such as orchestrator or vitess basically provide a command line tool which pulls in any required dependencies to enable the build to take place yet this does not happen with dbdeployer.

This leads to something like:

[myuser@myhost ~/src/dbdeployer/src/github.com/datacharmer/dbdeployer]$ ./build.sh OSX 1.1.1
+ env GOOS=darwin GOARCH=386 go build -o dbdeployer-1.1.1.osx .
main.go:19:2: cannot find package "github.com/datacharmer/dbdeployer/abbreviations" in any of:
	/usr/local/Cellar/go/1.10/libexec/src/github.com/datacharmer/dbdeployer/abbreviations (from $GOROOT)
	/Users/smudd/src/orchestrator/src/github.com/datacharmer/dbdeployer/abbreviations (from $GOPATH)
main.go:20:2: cannot find package "github.com/datacharmer/dbdeployer/cmd" in any of:
	/usr/local/Cellar/go/1.10/libexec/src/github.com/datacharmer/dbdeployer/cmd (from $GOROOT)
	/Users/smudd/src/orchestrator/src/github.com/datacharmer/dbdeployer/cmd (from $GOPATH)
tar: dbdeployer-1.1.1.osx: Cannot stat: No such file or directory
tar: Error exit delayed from previous errors.

One of the issues with go is tracking/handling of dependencies. It looks like this may change soon but given the dependencies you are using are quite small putting them under vendor/ and in your tree means there is no external dependency to your packages.

Failing that looking for the missing dependencies and pulling them down can work but then it's not clear if you used the same version of these dependencies that I may be using. Orchestrator has broken because of this and thus keeps its dependencies under vendor/ and these can be tracked in various ways. Vitess has an initial bootstrap.sh script (as this is more complex and includes other things like zookeeper etc) and pulls down the specific versions that are needed.

I see that:

$ go get github.com/datacharmer/dbdeployer/abbreviations
$ go get github.com/spf13/cobra

is all that is required to allow dbdeployer to build but do worry that if either of these libraries change unexpectedly you may see breakage.

So commenting on the best way to build from source would be good as it adds a bit more information in addition to the offered binaries.

[datacharmer]

The process should be really simple, and I will add this to the documentation:

~$ echo $GOPATH
 /home/gmax/go
 ~$ mkdir go
 ~$ cd go

Here I have an empty go directory. I need just one command to get the package and its dependencies:

~/go$ go get github.com/datacharmer/dbdeployer

~/go$ ls -l src/
total 4
drwxrwxr-x 4 gmax gmax 4096 Apr 11 20:45 github.com
~/go$ cd src/github.com/datacharmer/dbdeployer/

~/go/src/github.com/datacharmer/dbdeployer$ ls
abbreviations  Changelog  CNAME   concurrent   defaults  LICENSE  mkreadme  README.md  test
build.sh       cmd        common  _config.yml  docs      main.go  pflag     sandbox    unpack
~/go/src/github.com/datacharmer/dbdeployer$ ./build.sh linux 1.1.1
+ env GOOS=linux GOARCH=386 go build -o dbdeployer-1.1.1.linux .
-rwxrwxr-x 1 gmax gmax 4.4M Apr 11 20:46 dbdeployer-1.1.1.linux
-rw-rw-r-- 1 gmax gmax 1.7M Apr 11 20:46 dbdeployer-1.1.1.linux.tar.gz

[datacharmer]

Also, if you want to compile on your own, please read this notice about pflags.
I will consider your suggestion about vendor, but I want to avoid distributing code from others into my repository. I have several tests that check the behavior of the code that depends on external packages and I will deal with breakages when I see one.

[datacharmer]

I have updated build.sh to check dependencies and give advice on how to build.
Thanks for raising awareness about this issue.
The solution may not be final. Please iterate if you get more problems.

[sjmudd]

Hi Giuseppe.

Thanks for your comments and the update. golang version dependency management is planned to get better so maybe this will soon be a non-issue. Having had some bugs due to the dependent libraries changing I still like the idea of recording that somewhere if possible.

[datacharmer]

I have changed my mind about using a vendor folder, and I am experimenting with it using govendor.
The pflags problem would be avoided completely, and it would possibly allow me to enhance usability by playing with cobra defaults.

[datacharmer]

Fixed (better) in version 1.8.3