Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: optimize CGO caching and dead code removal #38886

Open
Keithcat1 opened this issue May 5, 2020 · 6 comments
Open

cmd/go: optimize CGO caching and dead code removal #38886

Keithcat1 opened this issue May 5, 2020 · 6 comments

Comments

@Keithcat1
Copy link

@Keithcat1 Keithcat1 commented May 5, 2020

I'm writing this with the assumption that each CGO-generated wrapper for each C function is stored in its own C file.
Editing and compiling a CGO package can take a minute or 2, even for one little change.
It should be possible to cache the generated .o files and only recompile them if the code for that function changes.
When linking the CGO wrappers, Go should avoid linking unused wrappers to reduce size.

@ianlancetaylor ianlancetaylor changed the title Optimize CGO caching and dead code removal cmd/cgo: optimize CGO caching and dead code removal May 5, 2020
@ianlancetaylor ianlancetaylor changed the title cmd/cgo: optimize CGO caching and dead code removal cmd/go: optimize CGO caching and dead code removal May 5, 2020
@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented May 5, 2020

One C file is generated for each Go file that uses import "C". It is not the case that we use a separate C file for each C function wrapper. Though I suppose we could if it seems useful.

CC @bcmills @jayconrod

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented May 5, 2020

Related #9887

Currently, I think we cache the generated .c files but not the .o files produced by the C compiler.

We should have a separate action for each .c file compilation, and we should cache the outputs of those actions.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented May 5, 2020

Note that proper caching of C object files require detecting whether any included .h files change. This does arise when people update to a newer version of some C dependency.

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented May 6, 2020

Good point. We should be incorporating hashes of included .h files into the cache keys for compiled .a files at least. I don't believe we currently are.

@Keithcat1
Copy link
Author

@Keithcat1 Keithcat1 commented May 23, 2020

Maybe we could only run the C compiler when the package is actually being compiled in order to get the values of C constants and make sure C functions are called properly.
Then, at link time, the Go linker linkes the entire program instead of using the C linker. We could probably use syscall to call C functions at runtime from CGO Go code wrappers.
This would have several benefits. Faster linking speed since no C linker is being invoked, the Go linker probably wouldn't need to spend time building a C linker compatible object file, dead code elimination could probably be done even for statically linked C code (at least on a per object file basis), and cross compiling might be supported if you have static libraries for other platforms. The Go package "github.com/veandco/go-sdl2/sdl" provides static libraries for all the platforms SDL2 runs on.
The linker would need support for reading a few different formats of object files though.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented May 23, 2020

We run the C linker, rather than using only the Go linker, because the Go linker doesn't support C++ global constructors or C++ exception handling or a few other cases that the C linker handles. We could support those things in the Go linker, but it's not trivial.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.