cmd/go: generating C header files for transitive dependencies is awkward

[bcmills]

As discovered in CL 208117 (#30316), cmd/go (as of Go 1.14) lacks a good way to generate C header files for Go libraries in order to use them from C.

go install -buildmode=c-archive and go install -buildmode=c-shared will, as a side-effect, generate header files for the requested package. However, -buildmode=c-archive and -buildmode=c-shared today require the named package to be a package main, and those commands do not generate headers for the (transitive) Go dependencies of the named package.

The -i flag does generate headers for transitive dependencies, but it also has a second, unwanted side effect: it builds (and attempts to install) libraries for the transitive dependencies of the named package, including transitive dependencies in the standard library. That fails if the user cannot write to GOROOT/pkg, such as when a non-root user is working with a go tool installed as root.

As a workaround, one can run go tool cgo -exportheader, but go tool cgo accepts a list of files rather than a list of packages, and generates a number of additional outputs (normally in an _obj subdirectory) that the C caller does not directly need. So we're still doing extra work, just in a different location.

We should see what we can do to make it easier to generate C headers without unnecessary or unwanted side-effects.

CC @ianlancetaylor @jayconrod

[bcmills]

One possibility might be to add another -buildmode, such as -buildmode=c-header, that does whatever work is needed (and possibly writes extra artifacts into GOCACHE) but only emits (or installs) header files rather than entire archives.

That could be used as a more direct substitute for the go tool cgo call with specific package arguments.

[bcmills]

Another option might be to make the -i flag suppress errors for standard-library targets, so that the go install -i command would work even if GOROOT is not writable.

[gopherbot]

Change https://golang.org/cl/208117 mentions this issue: misc/cgo/testcarchive: avoid writing to GOROOT in tests

[gopherbot]

Change https://golang.org/cl/208119 mentions this issue: misc/cgo/testcshared: avoid writing to GOROOT in tests

[ianlancetaylor]

We could have go install install all headers for imported packages, even without -i. That would be a change in behavior, but only for packages that are not main but have exported functions, which is an uncommon case.

[bcmills]

Maybe, but the import itself seems like a bit of a non-sequitur: if the C code is using identifiers from package a, then it seems like the build process should mention package a rather than some other package b that happens to import a.

On the other hand, I suppose that the C library must link against some (shared or archive) file generated from a main package, so having go install generate all of the headers for identifiers linked into that main package doesn't seem terrible.