New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: bootstrapping Go 1.n on macOS Sierra from source #16352

Closed
bradfitz opened this Issue Jul 13, 2016 · 78 comments

Comments

Projects
None yet
@bradfitz
Member

bradfitz commented Jul 13, 2016

@andlabs pointed out in #16272 (comment) that as of macOS Sierra, we'll have no way to bootstrap Go 1.N from source.

Maybe we need to backport @ianlancetaylor's time fix to Go 1.4?

@bradfitz

This comment has been minimized.

Show comment
Hide comment
@bradfitz

bradfitz Jul 13, 2016

Member

Nevermind. @ianlancetaylor says it works fine on Sierra. You can build Go 1.4 there and time won't work and tests will fail, but it will successfully build Go 1.7.

Member

bradfitz commented Jul 13, 2016

Nevermind. @ianlancetaylor says it works fine on Sierra. You can build Go 1.4 there and time won't work and tests will fail, but it will successfully build Go 1.7.

@andlabs

This comment has been minimized.

Show comment
Hide comment
@andlabs

andlabs Jul 13, 2016

Contributor

Good to know. I'm fine with untested 1.4s (the tests take too long to run on my virtual machines so I don't have a choice in those cases =P ); hopefully this won't break something else in the future.

Contributor

andlabs commented Jul 13, 2016

Good to know. I'm fine with untested 1.4s (the tests take too long to run on my virtual machines so I don't have a choice in those cases =P ); hopefully this won't break something else in the future.

@ianlancetaylor

This comment has been minimized.

Show comment
Hide comment
@ianlancetaylor

ianlancetaylor Jul 13, 2016

Contributor

I'm closing this because I believe that it works fine today.

Contributor

ianlancetaylor commented Jul 13, 2016

I'm closing this because I believe that it works fine today.

@bradfitz

This comment has been minimized.

Show comment
Hide comment
@bradfitz

bradfitz Aug 2, 2016

Member

As of Sierra Beta 4, this issue is again relevant.

Member

bradfitz commented Aug 2, 2016

As of Sierra Beta 4, this issue is again relevant.

@kirillDanshin

This comment has been minimized.

Show comment
Hide comment
@kirillDanshin

kirillDanshin Aug 3, 2016

I think we need to backport this fix in 1.4.4 as soon as possible.
we could also wait for Sierra stable release but now folks can't install Go from source.

kirillDanshin commented Aug 3, 2016

I think we need to backport this fix in 1.4.4 as soon as possible.
we could also wait for Sierra stable release but now folks can't install Go from source.

@bradfitz

This comment has been minimized.

Show comment
Hide comment
@bradfitz

bradfitz Aug 3, 2016

Member

@kirillDanshin, we're going to wait until Sierra is released. It's already changed twice during Sierra betas.

Member

bradfitz commented Aug 3, 2016

@kirillDanshin, we're going to wait until Sierra is released. It's already changed twice during Sierra betas.

@kirillDanshin

This comment has been minimized.

Show comment
Hide comment
@kirillDanshin

kirillDanshin Aug 3, 2016

ok @bradfitz ping me here when we'll going to backport the fix, I'll do it.

kirillDanshin commented Aug 3, 2016

ok @bradfitz ping me here when we'll going to backport the fix, I'll do it.

@ianlancetaylor

This comment has been minimized.

Show comment
Hide comment
@ianlancetaylor

ianlancetaylor Aug 3, 2016

Contributor

@kirillDanshin People using a beta version of Sierra and building Go from source should, I hope, be able to apply a patch to their 1.4.3 sources. It doesn't make sense for us to release 1.4.4 now, given that the code in Sierra has changed twice already.

Contributor

ianlancetaylor commented Aug 3, 2016

@kirillDanshin People using a beta version of Sierra and building Go from source should, I hope, be able to apply a patch to their 1.4.3 sources. It doesn't make sense for us to release 1.4.4 now, given that the code in Sierra has changed twice already.

@tattwamasi

This comment has been minimized.

Show comment
Hide comment
@tattwamasi

tattwamasi Aug 3, 2016

Has anyone filed a radar with Apple, and would it be worth it to? I was about to, and reference this and related issues, but will reference other IDs if already filed. (Edit: based on conversation in the other issues re: the change, seems like it isn't worth bringing up)

I had a working Go, but the b4 broke it, or I did with something else, so I was (re)installing with brew. Is there a easy(ish) way to build using the 1.4.3 patch with brew, or am I better off just pulling 1.4.3 from git and going from source?

tattwamasi commented Aug 3, 2016

Has anyone filed a radar with Apple, and would it be worth it to? I was about to, and reference this and related issues, but will reference other IDs if already filed. (Edit: based on conversation in the other issues re: the change, seems like it isn't worth bringing up)

I had a working Go, but the b4 broke it, or I did with something else, so I was (re)installing with brew. Is there a easy(ish) way to build using the 1.4.3 patch with brew, or am I better off just pulling 1.4.3 from git and going from source?

@bradfitz

This comment has been minimized.

Show comment
Hide comment
@bradfitz

bradfitz Aug 3, 2016

Member

Apple knows. The right people on both sides know the details. It's Go's fault, anyway, not Apple's.

Member

bradfitz commented Aug 3, 2016

Apple knows. The right people on both sides know the details. It's Go's fault, anyway, not Apple's.

@tattwamasi

This comment has been minimized.

Show comment
Hide comment
@tattwamasi

tattwamasi Aug 3, 2016

Thanks. And to follow up on the remainder of my comment, I didn't actually need to install from source, and had no reason to have to use brew, so I downloaded the pkg file for 1.7b5 from the main site.

It seems to work fine, and based on the git history should contain the fixes for Sierra (for now at least 😁 )
Now to rebuild Hugo and hopefully stop the frequent, but intermittent, crashes.

tattwamasi commented Aug 3, 2016

Thanks. And to follow up on the remainder of my comment, I didn't actually need to install from source, and had no reason to have to use brew, so I downloaded the pkg file for 1.7b5 from the main site.

It seems to work fine, and based on the git history should contain the fixes for Sierra (for now at least 😁 )
Now to rebuild Hugo and hopefully stop the frequent, but intermittent, crashes.

@bradfitz

This comment has been minimized.

Show comment
Hide comment
@bradfitz

bradfitz Sep 9, 2016

Member

macOS Sierra GM (Gold Master) came out yesterday. The time system call interface is unchanged from Beta 4.

I confirmed that all.bash passes on macOS 10.12 Sierra, using Go 1.7.1 as the GOROOT_BOOTSTRAP. It fails to bootstrap using Go 1.4.3.

So, should we cherry-pick the time fix back to Go 1.4.4 so there's a source-only way to bootstrap on macOS 10.12+?

The cherry picks would be:

fad2bbd (the first change, for Sierra beta2)
2da5633 (the fix for Sierra beta4, and final)
da070be (the syscall package too)

@ianlancetaylor, @rsc?

Member

bradfitz commented Sep 9, 2016

macOS Sierra GM (Gold Master) came out yesterday. The time system call interface is unchanged from Beta 4.

I confirmed that all.bash passes on macOS 10.12 Sierra, using Go 1.7.1 as the GOROOT_BOOTSTRAP. It fails to bootstrap using Go 1.4.3.

So, should we cherry-pick the time fix back to Go 1.4.4 so there's a source-only way to bootstrap on macOS 10.12+?

The cherry picks would be:

fad2bbd (the first change, for Sierra beta2)
2da5633 (the fix for Sierra beta4, and final)
da070be (the syscall package too)

@ianlancetaylor, @rsc?

@ianlancetaylor

This comment has been minimized.

Show comment
Hide comment
@ianlancetaylor

ianlancetaylor Sep 9, 2016

Contributor

I suppose I am mildly in favor of doing a 1.4.4 release for better Darwin support going forward. We should also consider #14795 (https://golang.org/cl/21190) and #13114 (https://golang.org/cl/16982).

Contributor

ianlancetaylor commented Sep 9, 2016

I suppose I am mildly in favor of doing a 1.4.4 release for better Darwin support going forward. We should also consider #14795 (https://golang.org/cl/21190) and #13114 (https://golang.org/cl/16982).

@mwhudson

This comment has been minimized.

Show comment
Hide comment
@mwhudson

mwhudson Sep 11, 2016

Contributor

I fixed #13114 on release-branch.go1.4 already (https://go-review.googlesource.com/c/21598/)

Contributor

mwhudson commented Sep 11, 2016

I fixed #13114 on release-branch.go1.4 already (https://go-review.googlesource.com/c/21598/)

gopherbot pushed a commit that referenced this issue Nov 8, 2016

doc: reference go1.4-bootstrap-20161024.tar.gz
Updates #16352

Change-Id: I214c87579ef21ced8d0ba94aa170dd7780afec4b
Reviewed-on: https://go-review.googlesource.com/32312
Reviewed-by: Ian Lance Taylor <iant@golang.org>

gopherbot pushed a commit that referenced this issue Nov 8, 2016

[release-branch.go1.7] doc: reference go1.4-bootstrap-20161024.tar.gz
Updates #16352

Change-Id: I214c87579ef21ced8d0ba94aa170dd7780afec4b
Reviewed-on: https://go-review.googlesource.com/32312
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-on: https://go-review.googlesource.com/32914
@bradfitz

This comment has been minimized.

Show comment
Hide comment
@bradfitz
Member

bradfitz commented Nov 10, 2016

@broady deployed the docs: https://golang.org/doc/install/source#go14

Closing this.

@bradfitz bradfitz closed this Nov 10, 2016

@hartzell

This comment has been minimized.

Show comment
Hide comment
@hartzell

hartzell Nov 19, 2016

Here's a Go 1.4 source-only snapshot for building on Sierra: https://storage.googleapis.com/golang/go1.4-bootstrap-20161024.tar.gz

@bradfitz -- My reading of the full set of comments here is that if there are other changes needed to keep 1.4 building in the future (even if Apple drops MacOS, etc...) there will be new date-stamped tarballs. Is that correct?

I'm touching up the spack packaging for the bootstrap and can either track the tip of the 1.4 release branch or use tarballs. The project has a mild preference for tarballs because one can verify them with digests.

hartzell commented Nov 19, 2016

Here's a Go 1.4 source-only snapshot for building on Sierra: https://storage.googleapis.com/golang/go1.4-bootstrap-20161024.tar.gz

@bradfitz -- My reading of the full set of comments here is that if there are other changes needed to keep 1.4 building in the future (even if Apple drops MacOS, etc...) there will be new date-stamped tarballs. Is that correct?

I'm touching up the spack packaging for the bootstrap and can either track the tip of the 1.4 release branch or use tarballs. The project has a mild preference for tarballs because one can verify them with digests.

hartzell added a commit to hartzell/spack that referenced this issue Nov 19, 2016

Update go-bootstrap package
The last C based Go src tree was the 1.4 series.  For a while they
were cutting new releases so that people could bootstrap from a C only
system.  Now they're recommending that you either use the release-1.4
branch or that you use a date-stamped tarball that they'll produce on
an as-needed basis.

There are several issues that keep 1.4.2 from building on a CentOS 7
system.

I've switched to the date based tarball.

The cgo bits were also mis-behaving, but they're not needed for the
bootstrapping task so I've set an environment variable that disables
them.

Details [on the install-from-source
page](https://golang.org/doc/install/source#go14) and these issues:

- golang/go#17545
- golang/go#16352.
@bradfitz

This comment has been minimized.

Show comment
Hide comment
@bradfitz

bradfitz Nov 19, 2016

Member

@bradfitz -- My reading of the full set of comments here is that if there are other changes needed to keep 1.4 building in the future (even if Apple drops MacOS, etc...) there will be new date-stamped tarballs. Is that correct?

Yup. That's the idea. If there are changes needed in the future.

Member

bradfitz commented Nov 19, 2016

@bradfitz -- My reading of the full set of comments here is that if there are other changes needed to keep 1.4 building in the future (even if Apple drops MacOS, etc...) there will be new date-stamped tarballs. Is that correct?

Yup. That's the idea. If there are changes needed in the future.

@hartzell

This comment has been minimized.

Show comment
Hide comment
@hartzell

hartzell commented Nov 19, 2016

@bradfitz Thanks!

tgamblin added a commit to spack/spack that referenced this issue Nov 24, 2016

Bugfix/update go packages (#2369)
* Update go-bootstrap package

The last C based Go src tree was the 1.4 series.  For a while they
were cutting new releases so that people could bootstrap from a C only
system.  Now they're recommending that you either use the release-1.4
branch or that you use a date-stamped tarball that they'll produce on
an as-needed basis.

There are several issues that keep 1.4.2 from building on a CentOS 7
system.

I've switched to the date based tarball.

The cgo bits were also mis-behaving, but they're not needed for the
bootstrapping task so I've set an environment variable that disables
them.

Details [on the install-from-source
page](https://golang.org/doc/install/source#go14) and these issues:

- golang/go#17545
- golang/go#16352.

* Update go package

Switched from pulling from the git repository to using the source
tarballs and added digest values.

Added support for 1.7.3, continued supporting 1.6.2, including patches
for a couple of problems (details in
[17545](golang/go#17545) and
[17986](golang/go#17986).

Dropped support for 1.5.4 and 1.4.2 because they no longer pass their
tests and the patches above to not apply.

citibeth added a commit to citibeth/spack that referenced this issue Dec 4, 2016

Bugfix/update go packages (#2369)
* Update go-bootstrap package

The last C based Go src tree was the 1.4 series.  For a while they
were cutting new releases so that people could bootstrap from a C only
system.  Now they're recommending that you either use the release-1.4
branch or that you use a date-stamped tarball that they'll produce on
an as-needed basis.

There are several issues that keep 1.4.2 from building on a CentOS 7
system.

I've switched to the date based tarball.

The cgo bits were also mis-behaving, but they're not needed for the
bootstrapping task so I've set an environment variable that disables
them.

Details [on the install-from-source
page](https://golang.org/doc/install/source#go14) and these issues:

- golang/go#17545
- golang/go#16352.

* Update go package

Switched from pulling from the git repository to using the source
tarballs and added digest values.

Added support for 1.7.3, continued supporting 1.6.2, including patches
for a couple of problems (details in
[17545](golang/go#17545) and
[17986](golang/go#17986).

Dropped support for 1.5.4 and 1.4.2 because they no longer pass their
tests and the patches above to not apply.

mbland added a commit to mbland/gvm that referenced this issue Dec 11, 2016

Add apply_patches function and initial patches
These patches are for two issues:

* Enabling Go to build on Alpine by disabling PIC per:
  golang/go#6940
  https://github.com/docker-library/golang/blob/master/1.7/alpine/no-pic.patch
* Fixing Go 1.4 bug appearing on macOS 10.12.1 per:
  golang/go#16352 (comment)

mbland added a commit to mbland/gvm that referenced this issue Dec 11, 2016

Add apply_patches function and initial patches
These patches are for two issues:

* Enabling Go to build on Alpine by disabling PIC per:
  golang/go#6940
  https://github.com/docker-library/golang/blob/master/1.7/alpine/no-pic.patch
* Fixing Go 1.4 bug appearing on macOS 10.12.1 per:
  golang/go#16352 (comment)

NexediGitlab pushed a commit to SlapOS/slapos that referenced this issue Feb 22, 2017

golang: Upgrade golang14 to latest source-only release so we can boot…
…strap it

Otherwise it started to fail this way:

[2017-02-22 11:49:22,982] INFO     --- FAIL: TestLoadFixed (0.00s)
[2017-02-22 11:49:22,982] INFO          time_test.go:929: Now().In(loc).Zone() = "-01", -3600, want "GMT+1", -3600
[2017-02-22 11:49:22,983] INFO     FAIL
[2017-02-22 11:49:22,983] INFO     FAIL time    2.380s

See golang/go#16352 (comment) and around
for details.
@gunsluo

This comment has been minimized.

Show comment
Hide comment

gunsluo commented May 11, 2017

mbland added a commit to mbland/dev-setup that referenced this issue Sep 5, 2017

Enable successful install on macOS 10.12.6
Of particular interest are the updates to `./go install languages go`
whereby the Go source repo is primed with the `GOROOT_BOOTSTRAP` version
in `_prepare_go_repo_for_bootstrap`. This is necessary due to the macOS
Sierra issues described in golang/go#16352.
The fix is only in the `release-branch.go1.4` branch not a tag, which
causes `gvm` not to find it without this priming step.
@StudioEtrange

This comment has been minimized.

Show comment
Hide comment
@StudioEtrange

StudioEtrange Nov 9, 2017

Still need a "real" binary release of 1.4.4, for some packaging system, some fast bootstraping purpose (without having to build go 1.4.4), or use gonative+gox tools on macos, and so on.

I dont know which problem does it generate to make a binary release instead of just source release ?

Please just release a binary go1.4.4 !

StudioEtrange commented Nov 9, 2017

Still need a "real" binary release of 1.4.4, for some packaging system, some fast bootstraping purpose (without having to build go 1.4.4), or use gonative+gox tools on macos, and so on.

I dont know which problem does it generate to make a binary release instead of just source release ?

Please just release a binary go1.4.4 !

@broady

This comment has been minimized.

Show comment
Hide comment
@broady

broady Nov 9, 2017

Member
Member

broady commented Nov 9, 2017

@StudioEtrange

This comment has been minimized.

Show comment
Hide comment
@StudioEtrange

StudioEtrange Nov 9, 2017

No, I cannot use 1.9.2 in some use case, for example for gonative+gox.
At this time gonative+gox on macos sierra is broken. And we just need a binary 1.4.4 release to not have to prebuilt a go1.4.4 before all our other task. Making too long bootstraping.

So, what is the exact problem of releasing 1.4.4 binary ?

StudioEtrange commented Nov 9, 2017

No, I cannot use 1.9.2 in some use case, for example for gonative+gox.
At this time gonative+gox on macos sierra is broken. And we just need a binary 1.4.4 release to not have to prebuilt a go1.4.4 before all our other task. Making too long bootstraping.

So, what is the exact problem of releasing 1.4.4 binary ?

@broady

This comment has been minimized.

Show comment
Hide comment
@broady

broady Nov 9, 2017

Member

Can you help me understand why you need a 1.4.x binary release?

You can use any recent version of Go to bootstrap an environment to compile from source.

You only need the 1.4.x source to bootstrap if you want to build entirely from source.
If you want a binary release to bootstrap, use the latest stable version (i.e., 1.9.2).

I thought there was an explanation of this somewhere, but I can't find it.

Member

broady commented Nov 9, 2017

Can you help me understand why you need a 1.4.x binary release?

You can use any recent version of Go to bootstrap an environment to compile from source.

You only need the 1.4.x source to bootstrap if you want to build entirely from source.
If you want a binary release to bootstrap, use the latest stable version (i.e., 1.9.2).

I thought there was an explanation of this somewhere, but I can't find it.

@StudioEtrange

This comment has been minimized.

Show comment
Hide comment
@StudioEtrange

StudioEtrange Nov 9, 2017

Ok
Crosscompiling chain use gonative+gox. (https://github.com/inconshreveable/gonative)
I think you know this set up.

gonative is a simple tool which creates a build of Go that can cross compile to all platforms while still using the Cgo-enabled versions of the stdlib packages. It does this by downloading the binary distributions for each platform and copying their libraries into the proper places.

Some go application use this setup to cross compile go application.

And for now, gonative needs go1.4 to work. Then it download binaries and prepare cross compiling toolchain, then we use gox to cross compile.

So, getting in first place go1.4.4 (so with patches for macos included) to be built before cross compilation take a while in case of bootstraping chain.

That's a use case of having 1.4.4 binary available

StudioEtrange commented Nov 9, 2017

Ok
Crosscompiling chain use gonative+gox. (https://github.com/inconshreveable/gonative)
I think you know this set up.

gonative is a simple tool which creates a build of Go that can cross compile to all platforms while still using the Cgo-enabled versions of the stdlib packages. It does this by downloading the binary distributions for each platform and copying their libraries into the proper places.

Some go application use this setup to cross compile go application.

And for now, gonative needs go1.4 to work. Then it download binaries and prepare cross compiling toolchain, then we use gox to cross compile.

So, getting in first place go1.4.4 (so with patches for macos included) to be built before cross compilation take a while in case of bootstraping chain.

That's a use case of having 1.4.4 binary available

@broady

This comment has been minimized.

Show comment
Hide comment
@broady

broady Nov 9, 2017

Member

Why does gonative require go1.4? I find that hard to believe. This sounds like an issue with gonative.

/cc @inconshreveable

Member

broady commented Nov 9, 2017

Why does gonative require go1.4? I find that hard to believe. This sounds like an issue with gonative.

/cc @inconshreveable

@StudioEtrange

This comment has been minimized.

Show comment
Hide comment
@StudioEtrange

StudioEtrange Nov 9, 2017

Dont know yet.
I investigate in the two direction

  • why gonative works only on 1.4.x
  • why there is not a go binary 1.4.4

StudioEtrange commented Nov 9, 2017

Dont know yet.
I investigate in the two direction

  • why gonative works only on 1.4.x
  • why there is not a go binary 1.4.4
@inconshreveable

This comment has been minimized.

Show comment
Hide comment
@inconshreveable

inconshreveable Nov 9, 2017

Contributor

@StudioEtrange gonative is very, very old and almost certainly no longer necessary or the right way to do whatever it is that you're trying to do. i no longer use it and you should consider it a deprecated tool. any issue with it also belongs on that repository and not here

Contributor

inconshreveable commented Nov 9, 2017

@StudioEtrange gonative is very, very old and almost certainly no longer necessary or the right way to do whatever it is that you're trying to do. i no longer use it and you should consider it a deprecated tool. any issue with it also belongs on that repository and not here

@rsc

This comment has been minimized.

Show comment
Hide comment
@rsc

rsc Nov 9, 2017

Contributor

Please don't comment on long-closed issues. We don't track them.

Contributor

rsc commented Nov 9, 2017

Please don't comment on long-closed issues. We don't track them.

@golang golang locked and limited conversation to collaborators Nov 9, 2017

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.