Skip to content

buildGoModule: building Go binaries in two phases #57080

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Mar 14, 2019

Conversation

kalbasit
Copy link
Member

@kalbasit kalbasit commented Mar 8, 2019

Motivation for this change

This change provides a new builder called buildGoModule for building Go packages with go modules support in mind. It does build through two phases:

  1. The first phase consists of building the go-modules derivation: go mod download will be executed within the source directory to download all of the dependencies, and store them at $GOPATH/pkg/mod. After removing the impure directory $GOPATH/pkg/mod/cache/vcs, we persist $GOPATH/pkg/mod as the $out of this derivation.
  2. The second phase consists of building the actual Go module, and this is done by setting GOPROXY to point to the download directory within ${go-modules}/cache/download which instructs Go to skip any network call, and simply use the already-downloaded dependencies.

The benefit of such infrastructure is that we no longer need tools such as dep2nix or vgo2nix to parse the go.mod and go.sum in order to generate Nix dependency file.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date. Rendered.
  • Fits CONTRIBUTING.md.
  • Converted packages
    • Move over hugo.
    • Move over pet.
    • Move over jx.
    • Move over gotools.
    • Move over terminal-parrot.
    • Move over todoist.
    • Move over hetzner-kube.
  • Create new packages
    • Create mod.

cc @ehmry @lethalman @fpletz @Mic92 @nlewo

nix run -f https://github.com/kalbasit/nur-packages/archive/master.tar.gz nix-verify -c nix-verify -I nixpkgs=`pwd` --keep-going \
	-A jx \
	-A pet \
	-A hugo \
    -A gotools \
    -A terminal-parrot \
    -A todoist \
    -A hetzner-kube \
    -A mod

@kalbasit kalbasit added the 2.status: work-in-progress This PR isn't done label Mar 8, 2019
@kalbasit kalbasit force-pushed the nixpkgs_create-build-go-module branch from 53d9cdf to 53829fb Compare March 8, 2019 19:48
@kalbasit

This comment has been minimized.

@kalbasit kalbasit force-pushed the nixpkgs_create-build-go-module branch from 53829fb to 1f21995 Compare March 8, 2019 20:01
@GrahamcOfBorg GrahamcOfBorg added 11.by: package-maintainer This PR was created by the maintainer of the package it changes 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Mar 8, 2019
@kalbasit kalbasit force-pushed the nixpkgs_create-build-go-module branch 2 times, most recently from 7fc305e to b3437e2 Compare March 8, 2019 23:15
@kalbasit
Copy link
Member Author

kalbasit commented Mar 8, 2019

@GrahamcOfBorg build jx pet hugo gotools

@kalbasit kalbasit force-pushed the nixpkgs_create-build-go-module branch from b3437e2 to a1d010c Compare March 8, 2019 23:29
@GrahamcOfBorg GrahamcOfBorg added the 8.has: documentation This PR adds or changes documentation label Mar 9, 2019
@kalbasit kalbasit force-pushed the nixpkgs_create-build-go-module branch from 556fe3f to 6f89cdd Compare March 9, 2019 05:48
@kalbasit kalbasit removed the 2.status: work-in-progress This PR isn't done label Mar 9, 2019
@kalbasit kalbasit marked this pull request as ready for review March 9, 2019 06:04
@kalbasit kalbasit requested review from Mic92, fpletz and nlewo March 9, 2019 06:05
@kalbasit
Copy link
Member Author

kalbasit commented Mar 9, 2019

@GrahamcOfBorg build jx pet hugo gotools

@kalbasit kalbasit force-pushed the nixpkgs_create-build-go-module branch from 48efa69 to c717237 Compare March 14, 2019 05:42
@sondr3 sondr3 mentioned this pull request Mar 14, 2019
10 tasks
@kalbasit kalbasit force-pushed the nixpkgs_create-build-go-module branch from c717237 to 627b088 Compare March 14, 2019 17:30
@kalbasit kalbasit force-pushed the nixpkgs_create-build-go-module branch from af208c4 to 8f8e71c Compare March 14, 2019 17:34
@kalbasit kalbasit changed the title [DO NOT MERGE] buildGoModule: building Go binaries in two phases buildGoModule: building Go binaries in two phases Mar 14, 2019
@kalbasit
Copy link
Member Author

kalbasit commented Mar 14, 2019

@Mic92 I've addressed your comments. I have also conducted some tests to verify with certainty that the modules downloaded are not affected by changes to the upstream repository, such as with the release of a new verison. This PR is now clear to be merged as well.

PTAL.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: golang 8.has: documentation This PR adds or changes documentation 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100 11.by: package-maintainer This PR was created by the maintainer of the package it changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants