Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/link: internal linker fails to build openshift/installer with "program too large, call relocation distance" and segfaults on linux/arm64 #40492

Open
jcajka opened this issue Jul 30, 2020 · 12 comments
Labels
Milestone

Comments

@jcajka
Copy link
Contributor

@jcajka jcajka commented Jul 30, 2020

What version of Go are you using (go version)?

go version devel +85afa2eb19 Tue Jul 28 16:59:04 2020 +0000 linux/arm64
Also seen with go1.15rc1/1.14.6/1.13.14

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

linux/arm64

What did you do?

git clone https://github.com/openshift/installer
pushd installer
go build -mod=vendor -o bin/openshift-install ./cmd/openshift-install

What did you expect to see?

Build succeed.

What did you see instead?

For the record following is with the master branch go and gcc-10.1.1, gibc-2.31 and binutils-2.34 on (Fedora 32) host.

# go build -mod=vendor -o bin/openshift-install ./cmd/openshift-install                                                                                                                            
# github.com/openshift/installer/cmd/openshift-install                                                                                                                                                                                        
net(.text): program too large, call relocation distance = 146365316                                                                                                                                                                           
net(.text): program too large, call relocation distance = 146365296                                                                                                                                                                           
net(.text): program too large, call relocation distance = 146365196                                                                                                                                                                           
net(.text): program too large, call relocation distance = 146365108                                                                                                                                                                           
net(.text): program too large, call relocation distance = 146365120                                                                                                                                                                           
net(.text): program too large, call relocation distance = 146365084                                                                                                                                                                           
net(.text): program too large, call relocation distance = 146365056                                                                                                                                                                           
net(.text): program too large, call relocation distance = 146364940                                                                                                                                                                           
runtime/cgo(.text): program too large, call relocation distance = 146364852                                                                                                                                                                   
runtime/cgo(.text): program too large, call relocation distance = 146364808                                                                                                                                                                   
runtime/cgo(.text): program too large, call relocation distance = 146364812                                                                                                                                                                   
runtime/cgo(.text): program too large, call relocation distance = 146364824                                                                                                                                                                   
runtime/cgo(.text): program too large, call relocation distance = 146364804                                                                                                                                                                   
runtime/cgo(.text): program too large, call relocation distance = 146364792                                                                                                                                                                   
runtime/cgo(.text): program too large, call relocation distance = 146364788                                                                                                                                                                   
runtime/cgo(.text): program too large, call relocation distance = 146364680                                                                                                                                                                   
runtime/cgo(.text): program too large, call relocation distance = 146364712                                                                                                                                                                   
runtime/cgo(.text): program too large, call relocation distance = 146364680                                                                                                                                                                   
runtime/cgo(.text): program too large, call relocation distance = 146364612                                                                                                                                                                   
runtime/cgo(.text): program too large, call relocation distance = 146364624                                                                                                                                                                   
runtime/cgo(.text): program too large, call relocation distance = 146364564                                                                                                                                                                   
/root/upstream/go/pkg/tool/linux_arm64/link: too many errors                                                                                                                                                                                  
unexpected fault address 0xffff03aca4ca                                                                                                                                                                                                       
fatal error: fault                                                                                                                                                                                                                            
[signal SIGSEGV: segmentation violation code=0x1 addr=0xffff03aca4ca pc=0xe10b0]                                                                                                                                                              
                                                                                                                                                                                                                                              
goroutine 305 [running]:                                                                                                                                                                                                                      
runtime.throw(0x29ff4c, 0x5)                                                                                                                                                                                                                  
        /root/upstream/go/src/runtime/panic.go:1116 +0x54 fp=0x4000037540 sp=0x4000037510 pc=0x41594                                                                                                                                          
runtime.sigpanic()                                                                                                                                                                                                                            
        /root/upstream/go/src/runtime/signal_unix.go:727 +0x3b8 fp=0x4000037570 sp=0x4000037540 pc=0x577d8                                                                                                                                    
encoding/binary.littleEndian.PutUint32(...)                                                                                                                                                                                                   
        /root/upstream/go/src/encoding/binary/binary.go:73                                                                                                                                                                                    
encoding/binary.(*littleEndian).PutUint32(0x489780, 0xffff03aca4ca, 0x4, 0xe6a7302, 0x126c)                                                                                                                                                   
        :1 +0x40 fp=0x40000375a0 sp=0x4000037580 pc=0xe10b0                                                                                                                                                                    
cmd/link/internal/ld.relocsym2(0x4000071880, 0x40007c2000, 0x4000071890, 0x40000718d8, 0x401fcbc480) 
        /root/upstream/go/src/cmd/link/internal/ld/data2.go:481 +0x620 fp=0x4000037750 sp=0x40000375a0 pc=0x171db0
cmd/link/internal/ld.(*Link).reloc2.func2(0x4000071880, 0x4000071880, 0x40007c2000, 0x4000071890, 0x40000718d8, 0x403bcb66f0)
        /root/upstream/go/src/cmd/link/internal/ld/data2.go:505 +0x68 fp=0x40000377a0 sp=0x4000037750 pc=0x1cd128
runtime.goexit()
        /root/upstream/go/src/runtime/asm_arm64.s:1136 +0x4 fp=0x40000377a0 sp=0x40000377a0 pc=0x726b4
created by cmd/link/internal/ld.(*Link).reloc2
        /root/upstream/go/src/cmd/link/internal/ld/data2.go:503 +0xe0

goroutine 1 [semacquire]:
sync.runtime_Semacquire(0x403bcb66f8)
        /root/upstream/go/src/runtime/sema.go:56 +0x38
sync.(*WaitGroup).Wait(0x403bcb66f0)
        /root/upstream/go/src/sync/waitgroup.go:130 +0x6c
cmd/link/internal/ld.(*Link).reloc2(0x4000071880)
        /root/upstream/go/src/cmd/link/internal/ld/data2.go:517 +0x12c
cmd/link/internal/ld.Main(0x448740, 0x10, 0x20, 0x1, 0x1f, 0x1e, 0x2a77c4, 0x14, 0x2aaf46, 0x1a, ...)
        /root/upstream/go/src/cmd/link/internal/ld/main.go:346 +0x1228
main.main()
        /root/upstream/go/src/cmd/link/main.go:68 +0x188

goroutine 80 [syscall]:
syscall.Syscall(0xd7, 0xfffefaed8000, 0x172997cc, 0x0, 0x40006dca28, 0x56c8d05f, 0xec02fd3640213e57)
        /root/upstream/go/src/syscall/asm_linux_arm64.s:9 +0x10
syscall.munmap(0xfffefaed8000, 0x172997cc, 0xffff121717cb, 0x4004e85ce8)
        /root/upstream/go/src/syscall/zsyscall_linux_arm64.go:957 +0x3c
syscall.(*mmapper).Munmap(0x4447a0, 0xfffefaed8000, 0x172997cc, 0x172997cc, 0x0, 0x0)
        /root/upstream/go/src/syscall/syscall_unix.go:94 +0x148
syscall.Munmap(...)
        /root/upstream/go/src/syscall/syscall_linux.go:1003 
cmd/link/internal/ld.(*OutBuf).munmap(0x4000136070)
        /root/upstream/go/src/cmd/link/internal/ld/outbuf_mmap.go:39 +0x54
cmd/link/internal/ld.(*OutBuf).copyHeap(0x4000136070, 0x403941a000)
        /root/upstream/go/src/cmd/link/internal/ld/outbuf.go:152 +0x54
cmd/link/internal/ld.(*OutBuf).Close(0x4000136070, 0x1c3ea8, 0x40317b4270)
        /root/upstream/go/src/cmd/link/internal/ld/outbuf.go:116 +0x124
cmd/link/internal/ld.linknew.func1()
        /root/upstream/go/src/cmd/link/internal/ld/sym.go:61 +0x4c
cmd/link/internal/ld.runAtExitFuncs()
        /root/upstream/go/src/cmd/link/internal/ld/util.go:24 +0x40
cmd/link/internal/ld.Exit(0x2)
        /root/upstream/go/src/cmd/link/internal/ld/util.go:31 +0x20
cmd/link/internal/ld.Exitf(0x2a4c73, 0xf, 0x0, 0x0, 0x0)
        /root/upstream/go/src/cmd/link/internal/ld/util.go:39 +0x100
cmd/link/internal/ld.afterErrorAction(...)
        /root/upstream/go/src/cmd/link/internal/ld/util.go:50
cmd/link/internal/ld.Errorf(0x403f1b4280, 0x2b350e, 0x30, 0x40006dcd88, 0x1, 0x1)
        /root/upstream/go/src/cmd/link/internal/ld/util.go:66 +0x100
cmd/link/internal/arm64.archreloc(0x4000071880, 0x40000718d8, 0x4077122560, 0x403f1b4280, 0x94000000, 0x910da273, 0x1)
        /root/upstream/go/src/cmd/link/internal/arm64/asm.go:634 +0x4dc
cmd/link/internal/ld.relocsym2(0x4000071880, 0x40007c2000, 0x4000071890, 0x40000718d8, 0x403f1b4280)
        /root/upstream/go/src/cmd/link/internal/ld/data2.go:147 +0xe38
cmd/link/internal/ld.(*Link).reloc2.func1(0x4000071880, 0x4000071880, 0x40007c2000, 0x4000071890, 0x40000718d8, 0x403bcb66f0)
        /root/upstream/go/src/cmd/link/internal/ld/data2.go:498 +0x78
created by cmd/link/internal/ld.(*Link).reloc2
        /root/upstream/go/src/cmd/link/internal/ld/data2.go:495 +0x9c

Using linkmode=external seems to work around the issue.
This reminds me of the similar issues with the external linker(on non-amd64 architectures) in the past #17028 . I'm bit surprised that this didn't got hit by any other "big binary" projects as kube, openshift,....

@davecheney
Copy link
Contributor

@davecheney davecheney commented Jul 30, 2020

How big are we talking? If you compile it for amd64 how larger is the image?

@jcajka
Copy link
Contributor Author

@jcajka jcajka commented Jul 30, 2020

@davecheney ~436MB executable on arm64 with external linker, with defaults 468MB on amd64

@davecheney
Copy link
Contributor

@davecheney davecheney commented Jul 30, 2020

That’s staggering. I thought hyperkube was large.

@jcajka
Copy link
Contributor Author

@jcajka jcajka commented Jul 30, 2020

TBH same here, assumed it will be in the "normal" ~100MB range, not 3-4 times larger.

@jcajka
Copy link
Contributor Author

@jcajka jcajka commented Jul 30, 2020

To note it is not stripped with DWARF. And disabling the DWARF have no effect on the build success.

@networkimprov
Copy link

@networkimprov networkimprov commented Jul 30, 2020

@gopherbot remove WaitingForInfo

cc @thanm @aclements @cherrymui

@aclements
Copy link
Member

@aclements aclements commented Jul 30, 2020

Also cc @jeremyfaller .

Looks like we may have a trampoline insertion issue?

@aclements aclements added this to the Go1.15 milestone Jul 30, 2020
@aclements
Copy link
Member

@aclements aclements commented Jul 30, 2020

Note that this was also present in 1.14 and 1.13, so this isn't a regression. I've tentatively put this in the 1.15 milestone, but we probably won't fix this for the initial release (it may make a point release).

@cherrymui
Copy link
Contributor

@cherrymui cherrymui commented Jul 30, 2020

We don't have trampoline support on ARM64. We could implement it (which shouldn't be too hard). But probably too late for 1.15.

@cherrymui
Copy link
Contributor

@cherrymui cherrymui commented Jul 30, 2020

DWARF does not matter. It is solely about program text.

@jcajka
Copy link
Contributor Author

@jcajka jcajka commented Aug 3, 2020

@cherrymui yeah, it would have been really weird if that had any impact

@aclements it would be cool to have it fixed in the go1.15, but it seems that the GC with external linker is ok atm(in all upstream supported versions of GC). So this doesn't seem to me as really critical issue atm.

@aclements
Copy link
Member

@aclements aclements commented Aug 3, 2020

Thanks for the follow-up. Since 1.15 is just about out the door at this point, I'll move this to 1.16.

@aclements aclements modified the milestones: Go1.15, Go1.16 Aug 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants
You can’t perform that action at this time.