New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: distribute linux/arm64 binaries for Go releases #19082

Closed
benshi001 opened this Issue Feb 14, 2017 · 38 comments

Comments

Projects
None yet
@benshi001
Member

benshi001 commented Feb 14, 2017

Current go 1.8 only contain an armv7l-linux package.

However linaro is also a popular 32/64-bit arm-linux platform, an aarch64 package in future go releases might be useful.

http://www.linaro.org/

@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney Feb 14, 2017

Contributor
Contributor

davecheney commented Feb 14, 2017

@benshi001

This comment has been minimized.

Show comment
Hide comment
@benshi001

benshi001 Feb 14, 2017

Member
Member

benshi001 commented Feb 14, 2017

@quentinmit quentinmit added this to the Go1.9Maybe milestone Feb 27, 2017

@vielmetti

This comment has been minimized.

Show comment
Hide comment
@vielmetti

vielmetti Mar 24, 2017

An officially supported 64-bit ARM version of Go would help a whole bunch of downstream projects that depend on Go, including especially Docker, and then anyone building multi-architecture projects that depend on Go in Docker containers. As it stands now the container process would have to bootstrap Go from source which adds complexity at the minimum and bloat.

There's a whole set of tooling and packaging for other platforms that depends on plopping in a supported binary as part of an automated build system.

vielmetti commented Mar 24, 2017

An officially supported 64-bit ARM version of Go would help a whole bunch of downstream projects that depend on Go, including especially Docker, and then anyone building multi-architecture projects that depend on Go in Docker containers. As it stands now the container process would have to bootstrap Go from source which adds complexity at the minimum and bloat.

There's a whole set of tooling and packaging for other platforms that depends on plopping in a supported binary as part of an automated build system.

@vielmetti

This comment has been minimized.

Show comment
Hide comment
@vielmetti

vielmetti Mar 24, 2017

To give you an idea of some of the hoops people are going through to bootstrap Go right now, I give you https://github.com/hypriot/golang-armbuilds - this should all be unnecessary when there's an official build.

vielmetti commented Mar 24, 2017

To give you an idea of some of the hoops people are going through to bootstrap Go right now, I give you https://github.com/hypriot/golang-armbuilds - this should all be unnecessary when there's an official build.

@zsmith928

This comment has been minimized.

Show comment
Hide comment
@zsmith928

zsmith928 Mar 29, 2017

agreed, Go is a popular multi-arch capable supporting tool for many projects. Including a binary would be very convenient and also support more testing and CI that should be using an official build.

zsmith928 commented Mar 29, 2017

agreed, Go is a popular multi-arch capable supporting tool for many projects. Including a binary would be very convenient and also support more testing and CI that should be using an official build.

@bradfitz bradfitz changed the title from Include an ARM64 package in future go releases to proposal: include an ARM64 package in future go releases Apr 12, 2017

@gopherbot gopherbot added the Proposal label Apr 12, 2017

@bradfitz bradfitz modified the milestones: Proposal, Go1.9Maybe Apr 12, 2017

@rsc rsc changed the title from proposal: include an ARM64 package in future go releases to proposal: build: distribute linux/arm64 binaries for Go releaess Apr 17, 2017

@rsc rsc changed the title from proposal: build: distribute linux/arm64 binaries for Go releaess to proposal: build: distribute linux/arm64 binaries for Go releases Apr 17, 2017

@benshi001

This comment has been minimized.

Show comment
Hide comment
@benshi001

benshi001 Apr 26, 2017

Member

The linux/arm64 can be attributed to "Other Ports". Something like
go1.9.linux-arm64.tar.gz

There are
go1.8.1.linux-ppc64le.tar.gz Archive Linux ppc64le 73MB b7b47572a2676449716865a66901090c057f6f1d8dfb1e19528fcd0372e5ce74
go1.8.1.linux-s390x.tar.gz Archive Linux s390x 72MB 0a59f4034a27fc51431989da520fd244d5261f364888134cab737e5bc2158cb2
in "Other Ports"

Member

benshi001 commented Apr 26, 2017

The linux/arm64 can be attributed to "Other Ports". Something like
go1.9.linux-arm64.tar.gz

There are
go1.8.1.linux-ppc64le.tar.gz Archive Linux ppc64le 73MB b7b47572a2676449716865a66901090c057f6f1d8dfb1e19528fcd0372e5ce74
go1.8.1.linux-s390x.tar.gz Archive Linux s390x 72MB 0a59f4034a27fc51431989da520fd244d5261f364888134cab737e5bc2158cb2
in "Other Ports"

@andrewhsu

This comment has been minimized.

Show comment
Hide comment
@andrewhsu

andrewhsu May 3, 2017

Having official google builds of golang for arm64 would be fantastically useful for users that just want to simply compile stuff on arm64 without having to do the golang bootstrap procedure.

andrewhsu commented May 3, 2017

Having official google builds of golang for arm64 would be fantastically useful for users that just want to simply compile stuff on arm64 without having to do the golang bootstrap procedure.

@bradfitz

This comment has been minimized.

Show comment
Hide comment
@bradfitz

bradfitz May 3, 2017

Member

Per https://golang.org/wiki/NoMeToo, please vote with the emoji thumbs up at the top.

Member

bradfitz commented May 3, 2017

Per https://golang.org/wiki/NoMeToo, please vote with the emoji thumbs up at the top.

@tianon

This comment has been minimized.

Show comment
Hide comment
@tianon

tianon May 3, 2017

As a datapoint against the target system build times, there are official binary releases for s390x and ppc64le, both of which are mainframe platforms and thus usually can also build Go fairly fast (but having official binary releases is still really useful).

tianon commented May 3, 2017

As a datapoint against the target system build times, there are official binary releases for s390x and ppc64le, both of which are mainframe platforms and thus usually can also build Go fairly fast (but having official binary releases is still really useful).

@glevand

This comment has been minimized.

Show comment
Hide comment
@glevand

glevand May 4, 2017

Just FYI, you can use gimme: https://github.com/travis-ci/gimme. It will build from source if it can't find a binary release.

glevand commented May 4, 2017

Just FYI, you can use gimme: https://github.com/travis-ci/gimme. It will build from source if it can't find a binary release.

@alexellis

This comment has been minimized.

Show comment
Hide comment
@alexellis

alexellis May 7, 2017

An official build of Golang for ARMv8 will save a lot of hassle for Docker images and build pipelines. Automation is everything and this lack of support makes working with 64-bit ARM harder.

RE: the argument of only taking 10 mins to build Go - I haven't tried this on one of the lower-powered SoCs, but certainly for CI - 10 min is a huge penalty over the time it takes to DL/un-tar.

alexellis commented May 7, 2017

An official build of Golang for ARMv8 will save a lot of hassle for Docker images and build pipelines. Automation is everything and this lack of support makes working with 64-bit ARM harder.

RE: the argument of only taking 10 mins to build Go - I haven't tried this on one of the lower-powered SoCs, but certainly for CI - 10 min is a huge penalty over the time it takes to DL/un-tar.

@zsmith928

This comment has been minimized.

Show comment
Hide comment
@zsmith928

zsmith928 May 7, 2017

@davecheney just as a point of clarification, Linaro isn't a Linux distribution. Rather the organization simply works on upstreaming open source arm software eg kernel, etc.

However, I think that Linaro and others in the arm ecosystem (eg huawei, cavium, Qualcomm, etc) would be willing to help maintain a native aarch64 build of GoLang.

Packet and ARM are already donating CI infrastructure for both 32 and 64 bit arm builds. I suspect Scaleway would also be supportive.

In the end, Packet and scale way each have thousands of customers using our armv8 infrastructures. A large percentage of these users have expressed a desire to have a pre built official binary. The main use case seems to be for using golang dependent software, eg Docker, k8s, cockroach etc

zsmith928 commented May 7, 2017

@davecheney just as a point of clarification, Linaro isn't a Linux distribution. Rather the organization simply works on upstreaming open source arm software eg kernel, etc.

However, I think that Linaro and others in the arm ecosystem (eg huawei, cavium, Qualcomm, etc) would be willing to help maintain a native aarch64 build of GoLang.

Packet and ARM are already donating CI infrastructure for both 32 and 64 bit arm builds. I suspect Scaleway would also be supportive.

In the end, Packet and scale way each have thousands of customers using our armv8 infrastructures. A large percentage of these users have expressed a desire to have a pre built official binary. The main use case seems to be for using golang dependent software, eg Docker, k8s, cockroach etc

glevand added a commit to glevand/travis-ci--gimme that referenced this issue May 8, 2017

gimme: Add url for arm64 bootstrap binary
To allow gimme to run on arm64 machines add a URL to a hosted arm64 binary
release.  This is hopefully a temporary work-around until official arm64
go binaries are released.

Note that arm64 support was added to go1.5, so gimme cannot bootstrap from the
1.4 sources when run on an arm64 machine.

See: golang/go#19082 (proposal: build: distribute
linux/arm64 binaries for Go releases).

Signed-off-by: Geoff Levand <geoff@infradead.org>

glevand added a commit to glevand/travis-ci--gimme that referenced this issue May 8, 2017

gimme: Add url for arm64 bootstrap binary
To allow gimme to run on arm64 machines add a URL to a hosted arm64 binary
release.  This is hopefully a temporary work-around until official arm64
go binaries are released.

Note that arm64 support was added to go1.5, so gimme cannot bootstrap from the
1.4 sources when run on an arm64 machine.

See: golang/go#19082 (proposal: build: distribute
linux/arm64 binaries for Go releases).

Signed-off-by: Geoff Levand <geoff@infradead.org>

glevand added a commit to glevand/travis-ci--gimme that referenced this issue May 8, 2017

gimme: Add url for arm64 bootstrap binary
To allow gimme to run on arm64 machines add a URL to a hosted arm64 binary
release.  This is hopefully a temporary work-around until official arm64
go binaries are released.

Note that arm64 support was added to go1.5, so gimme cannot bootstrap from the
1.4 sources when run on an arm64 machine.

See: golang/go#19082 (proposal: build: distribute
linux/arm64 binaries for Go releases).

Signed-off-by: Geoff Levand <geoff@infradead.org>

glevand added a commit to glevand/travis-ci--gimme that referenced this issue May 8, 2017

gimme: Add url for arm64 bootstrap binary
To allow gimme to run on arm64 machines add a URL to a hosted arm64 binary
release.  This is hopefully a temporary work-around until official arm64
go binaries are released.

Note that arm64 support was added to go1.5, so gimme cannot bootstrap from the
1.4 sources when run on an arm64 machine.

See: golang/go#19082 (proposal: build: distribute
linux/arm64 binaries for Go releases).

Signed-off-by: Geoff Levand <geoff@infradead.org>
@glevand

This comment has been minimized.

Show comment
Hide comment
@glevand

glevand May 8, 2017

We need at least one arm64 go binary release to use as the bootstrap compiler to build other go versions. As of now, we cannot build any version of go on an arm64 machine since arm64 support was added in go1.5, and go1.5 needs a go bootstrap compiler to build itself.

glevand commented May 8, 2017

We need at least one arm64 go binary release to use as the bootstrap compiler to build other go versions. As of now, we cannot build any version of go on an arm64 machine since arm64 support was added in go1.5, and go1.5 needs a go bootstrap compiler to build itself.

@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney May 9, 2017

Contributor
Contributor

davecheney commented May 9, 2017

@vielmetti

This comment has been minimized.

Show comment
Hide comment
@vielmetti

vielmetti May 9, 2017

@davecheney when you build the bootstrap compiler on another machine (according to your directions) you get this project by @hypriot and Dieter Reuter. https://github.com/hypriot/golang-armbuilds

vielmetti commented May 9, 2017

@davecheney when you build the bootstrap compiler on another machine (according to your directions) you get this project by @hypriot and Dieter Reuter. https://github.com/hypriot/golang-armbuilds

@glevand

This comment has been minimized.

Show comment
Hide comment
@glevand

glevand May 9, 2017

@davecheney I don't think it reasonable to require a user on an arm64 machine to have another machine to simply build a bootstrap program.

glevand commented May 9, 2017

@davecheney I don't think it reasonable to require a user on an arm64 machine to have another machine to simply build a bootstrap program.

@alexellis

This comment has been minimized.

Show comment
Hide comment
@alexellis

alexellis May 9, 2017

Is there any resistance to the proposal or was this additional information just for technical reference? I think this would benefit the community - well beyond the scope of Linaro.

alexellis commented May 9, 2017

Is there any resistance to the proposal or was this additional information just for technical reference? I think this would benefit the community - well beyond the scope of Linaro.

@gopherbot

This comment has been minimized.

Show comment
Hide comment
@gopherbot

gopherbot commented Jun 15, 2017

CL https://golang.org/cl/45913 mentions this issue.

@bradfitz

This comment has been minimized.

Show comment
Hide comment
@bradfitz

bradfitz Jun 15, 2017

Member

@broady, I'm running using pending CL 45913:

$ release -rev=952ecbe0a27aadd184ca3e2c342beb464d6b1653 -skip_tests -target=linux-arm64 -tools=6d30ceaab7ab5caffba4974a61e36d34268756ec -version=go1.9beta1 -watch
2017/06/15 19:45:42 linux-arm64: Start.
2017/06/15 19:45:42 linux-arm64: Creating buildlet.
2017/06/15 19:45:43 linux-arm64: Pushing source to buildlet.
....
Member

bradfitz commented Jun 15, 2017

@broady, I'm running using pending CL 45913:

$ release -rev=952ecbe0a27aadd184ca3e2c342beb464d6b1653 -skip_tests -target=linux-arm64 -tools=6d30ceaab7ab5caffba4974a61e36d34268756ec -version=go1.9beta1 -watch
2017/06/15 19:45:42 linux-arm64: Start.
2017/06/15 19:45:42 linux-arm64: Creating buildlet.
2017/06/15 19:45:43 linux-arm64: Pushing source to buildlet.
....
@bradfitz

This comment has been minimized.

Show comment
Hide comment
@bradfitz

bradfitz Jun 15, 2017

Member

Okay, it's uploaded now:

https://golang.org/dl/#unstable

The "Arch" column says "arm64" now. Maybe it should say something else. It looks out of place formatting-wise. ARMv8? ARMv8-A? AArch64? ARM has the most lovely names. ARMv8 matches the existing ARMv6 look-wise.

In any case, please test out the binary and report any problems.

If we don't hear happy reports, we won't ship it for the final release.

Member

bradfitz commented Jun 15, 2017

Okay, it's uploaded now:

https://golang.org/dl/#unstable

The "Arch" column says "arm64" now. Maybe it should say something else. It looks out of place formatting-wise. ARMv8? ARMv8-A? AArch64? ARM has the most lovely names. ARMv8 matches the existing ARMv6 look-wise.

In any case, please test out the binary and report any problems.

If we don't hear happy reports, we won't ship it for the final release.

glevand added a commit to glevand/travis-ci--gimme that referenced this issue Jun 15, 2017

gimme: Add url for arm64 bootstrap binary
To allow gimme to run on arm64 machines add a URL to a hosted go1.5.linux-arm64
binary release to use as a bootstrap compiler.  This is a temporary work-around
until official arm64 go binaries are released.

Note that arm64 support was added to go1.5, so gimme cannot use gcc to bootstrap
from go1.4 sources when run on an arm64 machine.

See: golang/go#19082 (build: distribute linux/arm64
binaries for Go releases).

Signed-off-by: Geoff Levand <geoff@infradead.org>

tianon added a commit to docker-library/golang that referenced this issue Jun 15, 2017

gopherbot pushed a commit to golang/build that referenced this issue Jun 15, 2017

cmd/release: add linux-arm64
Updates golang/go#19082

Change-Id: I02cd4f0e06e1f405761967dd550cf6c7a638835a
Reviewed-on: https://go-review.googlesource.com/45913
Reviewed-by: Chris Broadfoot <cbro@golang.org>
@nathangoulding

This comment has been minimized.

Show comment
Hide comment
@nathangoulding

nathangoulding Jun 15, 2017

@bradfitz My vote would be for arm64, and probably rename ARMv6 to arm32. Since there are a few 64-bit architecture now, perhaps amd64 and i386 instead of 64/32-bit.

nathangoulding commented Jun 15, 2017

@bradfitz My vote would be for arm64, and probably rename ARMv6 to arm32. Since there are a few 64-bit architecture now, perhaps amd64 and i386 instead of 64/32-bit.

@alexellis

This comment has been minimized.

Show comment
Hide comment
@alexellis

alexellis Jun 15, 2017

There are varying opinions on naming for ARM - ARM themselves communicating with the Docker/Moby project asked for -

  • arm32v6
  • arm32v7
  • arm64v8

etc. This has been reflected in the Docker manifest definitions and the prefixes for the naming of ARM-based container images.

aarch64 / arm64 / armv8 amongst others are the most popular I've seen elsewhere. Every project seems to do something different - which makes maintaining ports harder.

alexellis commented Jun 15, 2017

There are varying opinions on naming for ARM - ARM themselves communicating with the Docker/Moby project asked for -

  • arm32v6
  • arm32v7
  • arm64v8

etc. This has been reflected in the Docker manifest definitions and the prefixes for the naming of ARM-based container images.

aarch64 / arm64 / armv8 amongst others are the most popular I've seen elsewhere. Every project seems to do something different - which makes maintaining ports harder.

@bradfitz

This comment has been minimized.

Show comment
Hide comment
@bradfitz

bradfitz Jun 15, 2017

Member

We'll go with ARMv8 if anything. We're not repainting the rest of the house.

Member

bradfitz commented Jun 15, 2017

We'll go with ARMv8 if anything. We're not repainting the rest of the house.

@bradfitz

This comment has been minimized.

Show comment
Hide comment
@bradfitz

bradfitz Jun 15, 2017

Member

@nathangoulding, for anybody actually confused what "64-bit" or "32-bit" means, they can read the filename substring that says "amd64" or "386".

Member

bradfitz commented Jun 15, 2017

@nathangoulding, for anybody actually confused what "64-bit" or "32-bit" means, they can read the filename substring that says "amd64" or "386".

@nathangoulding

This comment has been minimized.

Show comment
Hide comment
@nathangoulding

nathangoulding Jun 15, 2017

@bradfitz More so for accuracy/consistency, especially as golang makes its way into predominantly ARM ecosystems. Fair enough though, and ARMv8 is a fine choice.

nathangoulding commented Jun 15, 2017

@bradfitz More so for accuracy/consistency, especially as golang makes its way into predominantly ARM ecosystems. Fair enough though, and ARMv8 is a fine choice.

tianon added a commit to infosiftr/stackbrew that referenced this issue Jun 15, 2017

Update docker-library images
- `docker`: 17.06.0-ce-rc4
- `golang`: add arm64! (golang/go#19082 (comment))
@benshi001

This comment has been minimized.

Show comment
Hide comment
@benshi001

benshi001 Jun 16, 2017

Member

I suggest aarch64. ARMv8 implies arm32 instruction set ver 8, and aarch64 implies the 64 bit mode instruction set, as a convention.

Please refer to linaro tool chain's naming.

https://www.linaro.org/downloads/

Member

benshi001 commented Jun 16, 2017

I suggest aarch64. ARMv8 implies arm32 instruction set ver 8, and aarch64 implies the 64 bit mode instruction set, as a convention.

Please refer to linaro tool chain's naming.

https://www.linaro.org/downloads/

@tianon

This comment has been minimized.

Show comment
Hide comment
@tianon

tianon Jun 16, 2017

If we don't hear happy reports, we won't ship it for the final release.

Happy to report that it's working pretty well here! 😇 👍 😀

(It successfully builds binaries as expected, and the resulting binaries run properly. ❤️)

Thanks for your work on this! 💯

tianon commented Jun 16, 2017

If we don't hear happy reports, we won't ship it for the final release.

Happy to report that it's working pretty well here! 😇 👍 😀

(It successfully builds binaries as expected, and the resulting binaries run properly. ❤️)

Thanks for your work on this! 💯

@emaste

This comment has been minimized.

Show comment
Hide comment
@emaste

emaste Jun 16, 2017

On a related note, I see there are FreeBSD go binaries for 32- and 64-bit x86, but not for 32- or 64-bit arm. Should I create a "build: distribute freebsd/arm64 binaries for Go releases" issue?

emaste commented Jun 16, 2017

On a related note, I see there are FreeBSD go binaries for 32- and 64-bit x86, but not for 32- or 64-bit arm. Should I create a "build: distribute freebsd/arm64 binaries for Go releases" issue?

@bradfitz

This comment has been minimized.

Show comment
Hide comment
@bradfitz

bradfitz Jun 16, 2017

Member

No, because we don't have a freebsd/arm64 port.

Member

bradfitz commented Jun 16, 2017

No, because we don't have a freebsd/arm64 port.

@emaste

This comment has been minimized.

Show comment
Hide comment
@emaste

emaste Jun 16, 2017

Ok, so first I'll open an issue for the FreeBSD/arm64 port - it will be useful to at least track requirements for bootstrapping.

emaste commented Jun 16, 2017

Ok, so first I'll open an issue for the FreeBSD/arm64 port - it will be useful to at least track requirements for bootstrapping.

@gopherbot

This comment has been minimized.

Show comment
Hide comment
@gopherbot

gopherbot commented Jun 28, 2017

CL https://golang.org/cl/46916 mentions this issue.

@vielmetti

This comment has been minimized.

Show comment
Hide comment
@vielmetti

vielmetti Jun 29, 2017

I take it that's a "yes", if so, thanks.

vielmetti commented Jun 29, 2017

I take it that's a "yes", if so, thanks.

@rhenwood-arm

This comment has been minimized.

Show comment
Hide comment
@rhenwood-arm

rhenwood-arm Jun 30, 2017

If we don't hear happy reports, we won't ship it for the final release.

My colleague, @naveena2016, shared test results with me. She used the AArch64 1.9 beta2 binary. Her effort included four different workloads that completed without issue. Performance could be characterized as 'similar to 1.8'.

Thanks for preparing this release!

rhenwood-arm commented Jun 30, 2017

If we don't hear happy reports, we won't ship it for the final release.

My colleague, @naveena2016, shared test results with me. She used the AArch64 1.9 beta2 binary. Her effort included four different workloads that completed without issue. Performance could be characterized as 'similar to 1.8'.

Thanks for preparing this release!

@eanger

This comment has been minimized.

Show comment
Hide comment
@eanger

eanger Jul 11, 2017

I've run some experiments to check out the behavior of the AArch64 1.9b2 binary, looking at the performance of the workloads in the Computer Language Benchmarks Game. All the benchmarks worked as expected. Here's a figure showing speedup of 1.9b2 over 1.6 for each of the benchmarks:
golang-1 6-vs-1 9b2
Thanks!

eanger commented Jul 11, 2017

I've run some experiments to check out the behavior of the AArch64 1.9b2 binary, looking at the performance of the workloads in the Computer Language Benchmarks Game. All the benchmarks worked as expected. Here's a figure showing speedup of 1.9b2 over 1.6 for each of the benchmarks:
golang-1 6-vs-1 9b2
Thanks!

@alexellis

This comment has been minimized.

Show comment
Hide comment
@alexellis

alexellis Jul 12, 2017

Nice @eagner - out of interest which machine did you run these on?

alexellis commented Jul 12, 2017

Nice @eagner - out of interest which machine did you run these on?

@eanger

This comment has been minimized.

Show comment
Hide comment
@eanger

eanger Jul 12, 2017

It was run on an A57 platform restricted to using a single core.

eanger commented Jul 12, 2017

It was run on an A57 platform restricted to using a single core.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.