Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

buildGoModule: building Go binaries in two phases #57080

Merged
merged 13 commits into from Mar 14, 2019

Conversation

@kalbasit
Copy link
Member

@kalbasit kalbasit commented Mar 8, 2019

Motivation for this change

This change provides a new builder called buildGoModule for building Go packages with go modules support in mind. It does build through two phases:

  1. The first phase consists of building the go-modules derivation: go mod download will be executed within the source directory to download all of the dependencies, and store them at $GOPATH/pkg/mod. After removing the impure directory $GOPATH/pkg/mod/cache/vcs, we persist $GOPATH/pkg/mod as the $out of this derivation.
  2. The second phase consists of building the actual Go module, and this is done by setting GOPROXY to point to the download directory within ${go-modules}/cache/download which instructs Go to skip any network call, and simply use the already-downloaded dependencies.

The benefit of such infrastructure is that we no longer need tools such as dep2nix or vgo2nix to parse the go.mod and go.sum in order to generate Nix dependency file.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date. Rendered.
  • Fits CONTRIBUTING.md.
  • Converted packages
    • Move over hugo.
    • Move over pet.
    • Move over jx.
    • Move over gotools.
    • Move over terminal-parrot.
    • Move over todoist.
    • Move over hetzner-kube.
  • Create new packages
    • Create mod.

cc @ehmry @lethalman @fpletz @Mic92 @nlewo

nix run -f https://github.com/kalbasit/nur-packages/archive/master.tar.gz nix-verify -c nix-verify -I nixpkgs=`pwd` --keep-going \
	-A jx \
	-A pet \
	-A hugo \
    -A gotools \
    -A terminal-parrot \
    -A todoist \
    -A hetzner-kube \
    -A mod
@kalbasit kalbasit force-pushed the kalbasit:nixpkgs_create-build-go-module branch from 53d9cdf to 53829fb Mar 8, 2019
@kalbasit

This comment has been hidden.

@kalbasit kalbasit force-pushed the kalbasit:nixpkgs_create-build-go-module branch from 53829fb to 1f21995 Mar 8, 2019
@kalbasit kalbasit force-pushed the kalbasit:nixpkgs_create-build-go-module branch 2 times, most recently from 7fc305e to b3437e2 Mar 8, 2019
@kalbasit
Copy link
Member Author

@kalbasit kalbasit commented Mar 8, 2019

@GrahamcOfBorg build jx pet hugo gotools

@kalbasit kalbasit force-pushed the kalbasit:nixpkgs_create-build-go-module branch from b3437e2 to a1d010c Mar 8, 2019
@kalbasit kalbasit force-pushed the kalbasit:nixpkgs_create-build-go-module branch from 556fe3f to 6f89cdd Mar 9, 2019
@kalbasit kalbasit marked this pull request as ready for review Mar 9, 2019
@kalbasit kalbasit requested review from Mic92, fpletz and nlewo Mar 9, 2019
@kalbasit
Copy link
Member Author

@kalbasit kalbasit commented Mar 9, 2019

@GrahamcOfBorg build jx pet hugo gotools

@sondr3 sondr3 mentioned this pull request Mar 14, 2019
6 of 10 tasks complete
@kalbasit kalbasit force-pushed the kalbasit:nixpkgs_create-build-go-module branch from c717237 to 627b088 Mar 14, 2019
@kalbasit kalbasit force-pushed the kalbasit:nixpkgs_create-build-go-module branch from af208c4 to 8f8e71c Mar 14, 2019
@kalbasit kalbasit changed the title [DO NOT MERGE] buildGoModule: building Go binaries in two phases buildGoModule: building Go binaries in two phases Mar 14, 2019
@kalbasit
Copy link
Member Author

@kalbasit kalbasit commented Mar 14, 2019

@Mic92 I've addressed your comments. I have also conducted some tests to verify with certainty that the modules downloaded are not affected by changes to the upstream repository, such as with the release of a new verison. This PR is now clear to be merged as well.

PTAL.

@Mic92
Mic92 approved these changes Mar 14, 2019
@kalbasit kalbasit merged commit a0e4f44 into NixOS:master Mar 14, 2019
13 checks passed
13 checks passed
@ofborg
gotools, hetzner-kube, hugo, jx, mod, pet, terminal-parrot, todoist on aarch64-linux Success
Details
@ofborg
gotools, hetzner-kube, hugo, jx, mod, pet, terminal-parrot, todoist on x86_64-darwin Success
Details
@ofborg
gotools, hetzner-kube, hugo, jx, mod, pet, terminal-parrot, todoist on x86_64-linux Success
Details
@GrahamcOfBorg
grahamcofborg-eval ^.^!
Details
@GrahamcOfBorg
grahamcofborg-eval-check-maintainers matching changed paths to changed attrs...
Details
@GrahamcOfBorg
grahamcofborg-eval-check-meta config.nix: checkMeta = true
Details
@GrahamcOfBorg
grahamcofborg-eval-nixos-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A manual
Details
@GrahamcOfBorg
grahamcofborg-eval-nixos-options nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A options
Details
@GrahamcOfBorg
grahamcofborg-eval-nixpkgs-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A manual
Details
@GrahamcOfBorg
grahamcofborg-eval-nixpkgs-tarball nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A tarball
Details
@GrahamcOfBorg
grahamcofborg-eval-nixpkgs-unstable-jobset nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A unstable
Details
@GrahamcOfBorg
grahamcofborg-eval-package-list nix-env -qa --json --file .
Details
@GrahamcOfBorg
grahamcofborg-eval-package-list-no-aliases nix-env -qa --json --file . --arg config { allowAliases = false; }
Details
@kalbasit kalbasit deleted the kalbasit:nixpkgs_create-build-go-module branch Mar 14, 2019
@elasticdog elasticdog mentioned this pull request Mar 22, 2019
10 of 10 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants