Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: setting CC to "ccache clang" does not work #11685

Open
tucnak opened this issue Jul 12, 2015 · 10 comments

Comments

@tucnak
Copy link

commented Jul 12, 2015

I am not able to build Go from Git (tag: go1.5beta1) with bootstrapping go1.4.2 on OSX 10.10.4, cos of wrong handling of $CC environmental variable.

Steps to reproduce:

  1. Get a custom CC, e.g. with ccache: export CC="ccache clang"
  2. ./all.bash

Expected result: built Go with exit status 1
Observed result: https://gist.github.com/tucnak/77c6d21d299f3c73ee7e

Looks like build script cuts off everything after space char in $CC, so instead of calling ccache clang -E -dM ... it does ccache -E -dM ... which is wrong. There must be a regression!

@ianlancetaylor ianlancetaylor changed the title OSX: Go from Git won't get built with custom $CC build: setting CC to "ccache clang" does not work Jul 12, 2015

@ianlancetaylor ianlancetaylor added this to the Go1.6 milestone Jul 12, 2015

@dominikh

This comment has been minimized.

Copy link
Member

commented Jul 12, 2015

This affects building any package with cgo and isn't specific to building Go itself. It's a regression that was introduced in commit b38fa89

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Jul 12, 2015

The build doesn't discard the options after the space character--you can see them in your gist in GOGCCFLAGS. But you are correct that the build assumes that the first word in $CC is special, and that it doesn't need to keep the second work immediately after the first word as is true for something like "ccache clang".

I doubt this will be fixed for the 1.5 release. For now you will have to work around the problem, either by using a shell script or by omitting "ccache". Frankly, "ccache" isn't buying you anything here, so I recommend omitting it until this is fixed.

@dominikh

This comment has been minimized.

Copy link
Member

commented Jul 12, 2015

For what it is worth, this was already semi-broken in Go 1.4, where it executes the following commands for CC="myccache gcc":

myccache gcc -E -dM -xc -m64 -I /tmp/go-build460939936/command-line-arguments/_obj/ -
myccache gcc -w -Wno-error -o/tmp/go-build460939936/command-line-arguments/_obj//_cgo_.o -gdwarf-2 -c -xc -I /tmp/go-build460939936/command-line-arguments/_obj/ -m64 -
myccache gcc -w -Wno-error -o/tmp/go-build460939936/command-line-arguments/_obj//_cgo_.o -gdwarf-2 -c -xc -I /tmp/go-build460939936/command-line-arguments/_obj/ -m64 -
myccache -I /tmp gcc -fPIC -m64 -pthread -fmessage-length=0 -print-libgcc-file-name

Where the last command isn't correct. So this is only partly a regression towards 1.4. It used to work, however, in 2012, where 7ae6872 specifically added support for it.

@ghost

This comment has been minimized.

Copy link

commented Oct 14, 2015

In my opinion, this isn't a golang bug.

cgo should just print a warning when CC contains spaces, so that the user can fix the actual cause of the problem.

Solution:

$ cat ~/bin/ccache-gcc 
#!/bin/bash
exec ccache /usr/bin/gcc "$@"

$ cat ~/bin/ccache-g++
#!/bin/bash
exec ccache /usr/bin/g++ "$@"

$ cat ~/.bashrc  (relevant lines only)
export PATH="${HOME}/bin:${PATH}"
export CC="ccache-gcc"
export CXX="ccache-g++"

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Oct 14, 2015

@atom-symbol I don't think I agree. I think you should be able to use spaces in $CC, and my understanding is that it used to work. It's not a major issue--as you say, you can always use a shell script--but I think we should fix it so that it works.

@ghost

This comment has been minimized.

Copy link

commented Oct 14, 2015

@ianlancetaylor CC="xxx yyy zzz" is ambiguous because it is uncertain whether the executable is "xxx" or "xxx yyy zzz". One way to decide is first to look whether there exists "xxx yyy zzz" executable, if it doesn't than for check existence of "xxx yyy", etc. go build does not know beforehand whether the name has spaces in it or not.

In the absolute, fixing this issue is impossible because solving it would require go build to know the mind's meaning of spaces in "xxx yyy zzz". Even if the build checks for the existence of "$PATH/xxx yyy zzz", it doesn't guarantee that the person really meant "$PATH/xxx yyy zzz" because the existence of "$PATH/xxx yyy zzz" may just be a coincidence that the person did not took into account.

@tucnak

This comment has been minimized.

Copy link
Author

commented Oct 14, 2015

@atom-symbol CC="xxx yyy zzz" doesn't seem ambiguous to me, it's pretty clear for those familiar with unix. I also can't workaround CC with spaces, since it's a must-have for my build environment in Qt. IMHO, any sort of valid CC command should be interpreted by Go fine. And as @ianlancetaylor pointed out, it's been working inb4...

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Oct 14, 2015

@atom-symbol I agree with @tucnak. It's not ambiguous. Spaces separate arguments. That is historically how the CC environment variable works on Unix systems. Programs like make and autoconf support CC with spaces, and people expect to be able to use them.

@mdempsky

This comment has been minimized.

Copy link
Member

commented Dec 7, 2015

There's a handful of problems.

First, in cmd/go when it sees a multifield CC value like "ccache clang", it resets CC="ccache" and moves "clang" into GOGCCFLAGS. (It also assumes it's able to insert arguments after the first field like "ccache -I objdir clang".)

Second, cmd/cgo doesn't use GOGCCFLAGS, so it doesn't see the "clang" argument.

Third, when you run "go env", it runs mkEnv again based on the updated environment, so the "clang" string disappears completely from the output of CC="ccache clang" go env, contrary to the environment actually provided to subtools.

Lastly, cmd/dist/test.go is currently written to assume $CC (and go env CC's output; it uses both for some reason) is a single field string, and it does use $GOGCCFLAGS.

@mdempsky mdempsky modified the milestones: Go1.7, Go1.6 Dec 8, 2015

@mdempsky mdempsky self-assigned this Dec 8, 2015

@mdempsky

This comment has been minimized.

Copy link
Member

commented Dec 8, 2015

This has never worked, and ccache isn't particularly useful for Go, so it doesn't seem like a blocker for Go 1.6. I'll look into it for Go 1.7.

@rsc rsc modified the milestones: Unplanned, Go1.7 May 17, 2016

glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 19, 2016
package/go-bootstrap: Build host tools with host CC
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 19, 2016
package/go: Build host tools with host CC
The go build system doesn't have the notion of cross compiling, but just the
notion of architecture.  When the host and target architectures are different
it expects to be given a target cross compiler in CC_FOR_TARGET.  When the
architectures are the same it will use CC_FOR_TARGET for both host and target
compilation.  To work around this limitation, when the host and target
architectures are the same build a set of host binaries and tools with
CC_FOR_TARGET set to the host compiler.  Also, the go build system is not
compatible with ccache, so use HOSTCC_NOCCACHE.  See
golang/go#11685.

Fixes build errors like these:

  host/usr/bin/go: No such file or directory

  http://autobuild.buildroot.net/results/6664189a6f3a815978e8d0a1d7ef408ca47e2874/

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 19, 2016
package/go-bootstrap: Build host tools with host CC
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 19, 2016
package/go: Build host tools with host CC
The go build system doesn't have the notion of cross compiling, but just the
notion of architecture.  When the host and target architectures are different
it expects to be given a target cross compiler in CC_FOR_TARGET.  When the
architectures are the same it will use CC_FOR_TARGET for both host and target
compilation.  To work around this limitation, when the host and target
architectures are the same build a set of host binaries and tools with
CC_FOR_TARGET set to the host compiler.  Also, the go build system is not
compatible with ccache, so use HOSTCC_NOCCACHE.  See
golang/go#11685.

Fixes build errors like these:

  host/usr/bin/go: No such file or directory

  http://autobuild.buildroot.net/results/6664189a6f3a815978e8d0a1d7ef408ca47e2874/

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 19, 2016
package/go-bootstrap: Build host tools with host CC
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 19, 2016
package/go: Build host tools with host CC
The go build system doesn't have the notion of cross compiling, but just the
notion of architecture.  When the host and target architectures are different
it expects to be given a target cross compiler in CC_FOR_TARGET.  When the
architectures are the same it will use CC_FOR_TARGET for both host and target
compilation.  To work around this limitation, when the host and target
architectures are the same build a set of host binaries and tools with
CC_FOR_TARGET set to the host compiler.  Also, the go build system is not
compatible with ccache, so use HOSTCC_NOCCACHE.  See
golang/go#11685.

Fixes build errors like these:

  host/usr/bin/go: No such file or directory

  http://autobuild.buildroot.net/results/6664189a6f3a815978e8d0a1d7ef408ca47e2874/

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 19, 2016
package/go: Build host tools with host CC
The go build system doesn't have the notion of cross compiling, but just the
notion of architecture.  When the host and target architectures are different
it expects to be given a target cross compiler in CC_FOR_TARGET.  When the
architectures are the same it will use CC_FOR_TARGET for both host and target
compilation.  To work around this limitation, when the host and target
architectures are the same build a set of host binaries and tools with
CC_FOR_TARGET set to the host compiler.  Also, the go build system is not
compatible with ccache, so use HOSTCC_NOCCACHE.  See
golang/go#11685.

Fixes build errors like these:

  host/usr/bin/go: No such file or directory

  http://autobuild.buildroot.net/results/6664189a6f3a815978e8d0a1d7ef408ca47e2874/

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 19, 2016
package/go-bootstrap: Build host tools with host CC
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 19, 2016
package/go: Build host tools with host CC
The go build system doesn't have the notion of cross compiling, but just the
notion of architecture.  When the host and target architectures are different
it expects to be given a target cross compiler in CC_FOR_TARGET.  When the
architectures are the same it will use CC_FOR_TARGET for both host and target
compilation.  To work around this limitation, when the host and target
architectures are the same build a set of host binaries and tools with
CC_FOR_TARGET set to the host compiler.  Also, the go build system is not
compatible with ccache, so use HOSTCC_NOCCACHE.  See
golang/go#11685.

Fixes build errors like these:

  host/usr/bin/go: No such file or directory

  http://autobuild.buildroot.net/results/6664189a6f3a815978e8d0a1d7ef408ca47e2874/

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 19, 2016
package/go-bootstrap: Build host tools with host CC
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 19, 2016
package/go-bootstrap: Build host tools with host CC
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 24, 2016
package/go: Build host tools with host CC
The go build system doesn't have the notion of cross compiling, but just the
notion of architecture.  When the host and target architectures are different
it expects to be given a target cross compiler in CC_FOR_TARGET.  When the
architectures are the same it will use CC_FOR_TARGET for both host and target
compilation.  To work around this limitation, when the host and target
architectures are the same build a set of host binaries and tools with
CC_FOR_TARGET set to the host compiler.  Also, the go build system is not
compatible with ccache, so use HOSTCC_NOCCACHE.  See
golang/go#11685.

Fixes build errors like these:

  host/usr/bin/go: No such file or directory

  http://autobuild.buildroot.net/results/6664189a6f3a815978e8d0a1d7ef408ca47e2874/

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 24, 2016
package/go-bootstrap: Build host tools with host CC
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 26, 2016
package/go: Build host tools with host CC
The go build system doesn't have the notion of cross compiling, but just the
notion of architecture.  When the host and target architectures are different
it expects to be given a target cross compiler in CC_FOR_TARGET.  When the
architectures are the same it will use CC_FOR_TARGET for both host and target
compilation.  To work around this limitation build and install a set of
compiler and tool binaries built with CC_FOR_TARGET set to the host compiler.
Also, the go build system is not compatible with ccache, so use
HOSTCC_NOCCACHE.  See golang/go#11685.

Fixes build errors like these:

  host/usr/bin/go: No such file or directory

  http://autobuild.buildroot.net/results/6664189a6f3a815978e8d0a1d7ef408ca47e2874/

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 26, 2016
package/go: Build host tools with host CC
The go build system doesn't have the notion of cross compiling, but just the
notion of architecture.  When the host and target architectures are different
it expects to be given a target cross compiler in CC_FOR_TARGET.  When the
architectures are the same it will use CC_FOR_TARGET for both host and target
compilation.  To work around this limitation build and install a set of
compiler and tool binaries built with CC_FOR_TARGET set to the host compiler.
Also, the go build system is not compatible with ccache, so use
HOSTCC_NOCCACHE.  See golang/go#11685.

Fixes build errors like these:

  host/usr/bin/go: No such file or directory

  http://autobuild.buildroot.net/results/6664189a6f3a815978e8d0a1d7ef408ca47e2874/

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 26, 2016
package/go-bootstrap: Build host tools with host CC
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
woodsts pushed a commit to woodsts/buildroot that referenced this issue May 26, 2016
package/go: Build host tools with host CC
The go build system doesn't have the notion of cross compiling, but just the
notion of architecture.  When the host and target architectures are different
it expects to be given a target cross compiler in CC_FOR_TARGET.  When the
architectures are the same it will use CC_FOR_TARGET for both host and target
compilation.  To work around this limitation build and install a set of
compiler and tool binaries built with CC_FOR_TARGET set to the host compiler.
Also, the go build system is not compatible with ccache, so use
HOSTCC_NOCCACHE.  See golang/go#11685.

Fixes build errors like these:

  host/usr/bin/go: No such file or directory

  http://autobuild.buildroot.net/results/6664189a6f3a815978e8d0a1d7ef408ca47e2874/

Signed-off-by: Geoff Levand <geoff@infradead.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 26, 2016
package/go-bootstrap: Build host tools with host CC
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 31, 2016
package/go-bootstrap: Build host tools with host CC
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
buildroot-auto-update pushed a commit to buildroot/buildroot that referenced this issue Jul 4, 2016
package/go-bootstrap: Build host tools with host CC
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
BWhitten added a commit to LairdCP/wb-buildroot that referenced this issue Dec 14, 2017
package/go: Build host tools with host CC
The go build system doesn't have the notion of cross compiling, but just the
notion of architecture.  When the host and target architectures are different
it expects to be given a target cross compiler in CC_FOR_TARGET.  When the
architectures are the same it will use CC_FOR_TARGET for both host and target
compilation.  To work around this limitation build and install a set of
compiler and tool binaries built with CC_FOR_TARGET set to the host compiler.
Also, the go build system is not compatible with ccache, so use
HOSTCC_NOCCACHE.  See golang/go#11685.

Fixes build errors like these:

  host/usr/bin/go: No such file or directory

  http://autobuild.buildroot.net/results/6664189a6f3a815978e8d0a1d7ef408ca47e2874/

Signed-off-by: Geoff Levand <geoff@infradead.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
BWhitten added a commit to LairdCP/wb-buildroot that referenced this issue Dec 14, 2017
package/go-bootstrap: Build host tools with host CC
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Risca added a commit to Risca/CHIP-buildroot that referenced this issue Sep 30, 2018
package/go: Build host tools with host CC
The go build system doesn't have the notion of cross compiling, but just the
notion of architecture.  When the host and target architectures are different
it expects to be given a target cross compiler in CC_FOR_TARGET.  When the
architectures are the same it will use CC_FOR_TARGET for both host and target
compilation.  To work around this limitation build and install a set of
compiler and tool binaries built with CC_FOR_TARGET set to the host compiler.
Also, the go build system is not compatible with ccache, so use
HOSTCC_NOCCACHE.  See golang/go#11685.

Fixes build errors like these:

  host/usr/bin/go: No such file or directory

  http://autobuild.buildroot.net/results/6664189a6f3a815978e8d0a1d7ef408ca47e2874/

Signed-off-by: Geoff Levand <geoff@infradead.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Risca added a commit to Risca/CHIP-buildroot that referenced this issue Sep 30, 2018
package/go-bootstrap: Build host tools with host CC
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.