Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gccgo: export data linked into binary #26204

Open
cherrymui opened this issue Jul 3, 2018 · 9 comments

Comments

@cherrymui
Copy link
Contributor

commented Jul 3, 2018

With gccgo,

$ go version
go version go1.10.3 gccgo (GCC) 9.0.0 20180622 (experimental) linux/amd64
$ gccgo --version
gccgo (GCC) 9.0.0 20180622 (experimental)
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
$ go build -gccgoflags="-static-libgo" hello.go
$ objdump -h hello

hello:     file format elf64-x86-64

Sections:
Idx Name          Size      VMA               LMA               File off  Algn
...
 31 .go_export    000363b8  0000000000000000  0000000000000000  002b460b  2**0
                  CONTENTS, READONLY
...

Dumping out the .go_export section, it looks like indeed export data. I think it is not needed at run time. It probably should not be linked into the binary.

-static-libgo is just for demonstration. The export data also present in the default dynamically linked binary, or completely -static binary, though the size varies.

With some big binaries, like kubernetes, the export data can be quite big (over 40% of the binary size).

cc @ianlancetaylor

@gopherbot gopherbot added this to the Gccgo milestone Jul 3, 2018

@cherrymui

This comment has been minimized.

Copy link
Contributor Author

commented Jul 3, 2018

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Jul 4, 2018

We use the system linker. We want the export data to be present in a shared library, but not in an executable. I don't know of a way to tell the linker to do that.

Perhaps we could have the go tool run objcopy --remove-section .go_export EXECUTABLE.

@cherrymui

This comment has been minimized.

Copy link
Contributor Author

commented Jul 4, 2018

Executable produced by gollvm doesn't have the export data. The .o and .a files have the .go_export sections, but they don't get linked into the final executable. Gollvm invokes the linker as

/usr/bin/ld.gold -o /tmp/go-build295362102/b001/exe/a.out --eh-frame-hdr -dynamic-linker /lib64/ld-linux-x86-64.so.2 /usr/lib/gcc/x86_64-linux-gnu/7.3.0/../../../x86_64-linux-gnu/crt1.o /usr/lib/gcc/x86_64-linux-gnu/7.3.0/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/7.3.0/crtbegin.o -( --whole-archive /tmp/go-build295362102/b001/_pkg_.a --no-whole-archive -) --build-id=0x756b496c416977546553707855656f4d455539372f6d7a496b493163304963724574445653723442502f6f57667a39426869472d3133595a42556f5250452f756b496c416977546553707855656f4d45553937 -m elf_x86_64 -L/usr/local/google/home/cherryyz/w/gollvm/bin/../lib64 -lgobegin -Bstatic -lgo -Bdynamic -L/usr/lib/gcc/x86_64-linux-gnu/7.3.0 -L/usr/lib/gcc/x86_64-linux-gnu/7.3.0/../../../x86_64-linux-gnu -lpthread -lm --wrap=pthread_create -lgcc_s -lgcc -lc -lgcc_s -lgcc /usr/lib/gcc/x86_64-linux-gnu/7.3.0/crtend.o /usr/lib/gcc/x86_64-linux-gnu/7.3.0/../../../x86_64-linux-gnu/crtn.o

gccgo linking uses collect2, as

collect2 --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o /tmp/go-build236096397/b001/exe/a.out /usr/lib/x86_64-linux-gnu/crt1.o /usr/lib/x86_64-linux-gnu/crti.o /usr/local/google/home/cherryyz/w/gccgo2/lib/gcc/x86_64-pc-linux-gnu/9.0.0/crtbegin.o -L/usr/local/google/home/cherryyz/w/gccgo2/lib/gcc/x86_64-pc-linux-gnu/9.0.0 -L/usr/local/google/home/cherryyz/w/gccgo2/lib/gcc/x86_64-pc-linux-gnu/9.0.0/../../../../lib64 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/local/google/home/cherryyz/w/gccgo2/lib/gcc/x86_64-pc-linux-gnu/9.0.0/../../.. -( --whole-archive /tmp/go-build236096397/b001/_pkg_.a --no-whole-archive -) --build-id=0x5732536f5472715a564e36356b4d46326f42762d2f4a4961684b5f6e78615f2d64597278507a794a392f786d31684579334e452d383664597a596654474e2f5732536f5472715a564e36356b4d46326f42762d -lgobegin -Bstatic -lgo -Bdynamic -lpthread -lm --wrap=pthread_create -lgcc_s -lgcc -lc -lgcc_s -lgcc /usr/local/google/home/cherryyz/w/gccgo2/lib/gcc/x86_64-pc-linux-gnu/9.0.0/crtend.o /usr/lib/x86_64-linux-gnu/crtn.o

The flags looks pretty similar. I don't know what makes the difference.

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Jul 4, 2018

Does a shared library produced by gollvm have the export data?

@cherrymui

This comment has been minimized.

Copy link
Contributor Author

commented Jul 5, 2018

I'm not sure what is the "right" way to produce a shared library with gccgo (or gollvm). I tried

$ gccgo -shared -o libp.so p.go

(where p.go is not main package)

This works, and libp.so contains the export data. I can then import it, like

$ gccgo -I. -c main.go
$ gccgo main.o libp.so
$ LD_LIBRARY_PATH=$PWD:$GCCGOINSTALLDIR/lib64 ./a.out

(where main.go import package p)

With gollvm, libp.so doesn't have the export data, and I cannot do the above.

$ llvm-goc -shared -o libp.so p.go
$ llvm-goc -I. -c main.go 
main.go:3:9: error: ./libp.so exists but does not contain any Go export data
main.go:3:9: error: libp.so exists but does not contain any Go export data
main.go:3:9: error: import file 'p' not found
main.go:5:15: error: reference to undefined name 'p'

I looked into this. The reason is that when generating the object file, the .go_export section has the "e" (exclude) flag set. Gccgo doesn't set this flag. If I don't set this flag, shared libraries will have the export data, so will executables. Is there a way to instruct the linker to drop a section when we are linking executable?

In "normal" builds (i.e. using the go tool, instead of invoking gccgo or llvm-goc manually), does it need to look up the export data from shared libraries? When?

cc @thanm

@thanm

This comment has been minimized.

Copy link
Member

commented Jul 6, 2018

I'm wondering how important it is to support this use case -- do people actually rely on this behavior?

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Jul 6, 2018

I also don't know how important it is to put export information in a shared library. Historically it worked naturally with gccgo -o libpkg.so -shared PKGFILES. This assumed that you built most of your packages that way. Then import "pkg" would find libpkg.so and pull the export data from the .go_export section and everything would work smoothly. If we start adding SHF_EXCLUDE to the .go_export section, that approach would stop working.

But I don't know if anybody actually does that. Now that gccgo includes the Go tool, it should in principle work to use go install -buildmode=shared PKG. That will install two different files: PKGPATH/libPKG.a and libPKGPATH.so. With this model, gccgo should find libPKG.a and extract the export information from there.

@thanm

This comment has been minimized.

Copy link
Member

commented Jul 6, 2018

My preference would be to keep the current behavior (not incorporate the export data when linking) -- perhaps as a fallback we could add a command line flag to enable it (in case someone really needs this)?

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Jul 6, 2018

I guess a command line option would be consistent with the general GCC approach to these matters.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.