Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/link: crashes on ppc64le and others #25863

Closed
jcajka opened this issue Jun 13, 2018 · 15 comments

Comments

Projects
None yet
8 participants
@jcajka
Copy link
Contributor

commented Jun 13, 2018

What version of Go are you using (go version)?

go version devel +9ef9765c16 Tue Jun 12 11:08:14 2018 +0000 linux/ppc64le

Does this issue reproduce with the latest release?

Affects master, bisected and the issues started with commit

[bd83774593bca66cc899d5180c77680bc907fab8] cmd/link: separate virtual address layout from file layout

What operating system and processor architecture are you using (go env)?

GOARCH="ppc64le"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOEXE=""
GOHOSTARCH="ppc64le"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/root/go"
GORACE=""
GOROOT="/usr/lib/golang"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/golang/pkg/tool/linux_ppc64le"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build008088815=/tmp/go-build -gno-record-gcc-switches"

What did you do?

git clone https://github.com/golang/go
cd go/src
GOROOT_BOOTSTRAP=/usr/lib/golang ./all.bash

What did you expect to see?

Build/Bootstrap of GC(all.bash) passing

What did you see instead?

Building Go cmd/dist using /usr/lib/golang/.
Building Go toolchain1 using /usr/lib/golang/.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
go tool dist: FAILED: /root/go/pkg/tool/linux_ppc64le/go_bootstrap install -gcflags=all= -ldflags=all= -i cmd/asm cmd/cgo cmd/compile cmd/link: signal: segmentation fault

Notes

Doesn't seems to be related to the boostrap compiler as have been using 9ef9765 for bisect and also tried few builds with go1.10 and go1.9, this issue seems to span all Fedora releases.

CC'ed @laboger

@tklauser

This comment has been minimized.

@tklauser

This comment has been minimized.

Copy link
Member

commented Jun 13, 2018

...and the linux/arm64, linux/arm, linux/mips, linux/s390x builders fail as well, though not with a segfault during bootstrap but during various tests:

linux/arm: https://build.golang.org/log/ce0ac926b3379a141c368e72328120ff16e75928
linux/arm64: https://build.golang.org/log/36143a71f0998633c7bd61705d38e301e2450ae9
linux/mips: https://build.golang.org/log/2647b5b5501fbe006587658f098778fcc16d1a02
linux/s390x: https://build.golang.org/log/981714c0a467c108dbdf281a531f72a04d7aed92

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Jun 13, 2018

@ianlancetaylor ianlancetaylor added this to the Go1.11 milestone Jun 13, 2018

@ianlancetaylor ianlancetaylor changed the title GC boostrap fails on ppc64le with segfault cmd/link: crashes on ppc64le and others Jun 13, 2018

@laboger

This comment has been minimized.

Copy link
Contributor

commented Jun 13, 2018

If I look at the readelf output for go_bootstrap after this change, the section names are garbage. Otherwise the rest looks the same.

Section Headers:
  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 0] ^I^J^K^L<95>^Q       NULL             0000000000000000  00000000
       0000000000000000  0000000000000000           0     0     0
  [ 1] ^J^K^L<95>^Q         PROGBITS         0000000000011000  00001000
       00000000002497c0  0000000000000000  AX       0     0     16
  [ 2]                   PROGBITS         0000000000260000  00251000
       00000000001031c0  0000000000000000   A       0     0     32
  [ 3] ^E^E^E^Fé^Q       STRTAB           0000000000000000  003531c0
       00000000000000fa  0000000000000000           0     0     1
  [ 4]                   PROGBITS         00000000003632c0  003542c0
       00000000000022e4  0000000000000000   A       0     0     32
  [ 5]                   PROGBITS         00000000003655a8  003565a8
       00000000000008b8  0000000000000000   A       0     0     8
  [ 6] ^C^C^C^C^C^C^C^C¨ PROGBITS         0000000000365e60  00356e60
       0000000000000000  0000000000000000   A       0     0     1
  [ 7]                   PROGBITS         0000000000365e60  00356e60
       00000000001921ff  0000000000000000   A       0     0     32
  [ 8]                   PROGBITS         0000000000500000  004f1000
       0000000000011e60  0000000000000000  WA       0     0     32
  [ 9] ^D^D^D^E^G^G<9c>^Q   PROGBITS         0000000000511e60  00502e60
       000000000000df90  0000000000000000  WA       0     0     32
  [10] <9c>^Q               NOBITS           000000000051fe00  00510e00
       0000000000024730  0000000000000000  WA       0     0     32
  [11]                   NOBITS           0000000000544540  00535540
       0000000000002978  0000000000000000  WA       0     0     32
  [12] ^B^B^B^B^C^C^D^D  PROGBITS         0000000000550000  00511000
       00000000000001d3  0000000000000000           0     0     1
  [13] ^KÐ^Q             PROGBITS         00000000005501d3  005111d3
       000000000007a448  0000000000000000           0     0     1
  [14]                   PROGBITS         00000000005ca61b  0058b61b
       000000000004bdc4  0000000000000000           0     0     1
  [15] ^B^B^B^B^B^B^B^C  PROGBITS         00000000006163df  005d73df
       0000000000017d2f  0000000000000000           0     0     1
  [16] ^D^D^F^G^H^J^J^J  PROGBITS         000000000062e10e  005ef10e
       0000000000021482  0000000000000000           0     0     1
  [17]                   PROGBITS         000000000064f590  00610590
       000000000000003b  0000000000000000           0     0     1
  [18] ^G^H^J^PÅ^Q       PROGBITS         000000000064f5cb  006105cb
       0000000000156fe7  0000000000000000           0     0     1
  [19] ^C^D^E^E^G^G^H^I  PROGBITS         00000000007a65b2  007675b2
       0000000000183b8f  0000000000000000           0     0     1
  [20]                   PROGBITS         000000000092a141  008eb141
       000000000008dbb0  0000000000000000           0     0     1
  [21] ^E^E^E^E^E^E^F¯^Q SYMTAB           0000000000000000  00980000
       0000000000034cf8  0000000000000018          22   393     8
  [22] ^Q                STRTAB           0000000000000000  009b4cf8
       000000000003c0c3  0000000000000000           0     0     1
@laboger

This comment has been minimized.

Copy link
Contributor

commented Jun 13, 2018

How could this pass the trybots for ppc64x?

@tklauser

This comment has been minimized.

Copy link
Member

commented Jun 13, 2018

I think there are currently no trybots for linux/ppc64x

/cc @bradfitz

@laboger

This comment has been minimized.

Copy link
Contributor

commented Jun 13, 2018

Last time I asked, I was told there was some compile-only testing done, but obviously not since in this case the toolchain doesn't even build.

@bradfitz

This comment has been minimized.

Copy link
Member

commented Jun 13, 2018

@laboger, @tklauser, we do indeed compile for ppc64 on trybots, but it's cross-compiled from Linux, not run on real ppc64 hardware, which seems to be the issue here. Looks like it does cross-compile fine from Linux.

The definition from x/build/dashboard/builders.go is:

        addMiscCompile := func(suffix, rx string) { 
                addBuilder(BuildConfig{ 
                        Name:        "misc-compile" + suffix, 
                        HostType:    "host-linux-jessie", 
                        TryBot:      true, 
                        TryOnly:     true, 
                        CompileOnly: true, 
                        Notes:       "Runs buildall.sh to cross-compile std packages for " + rx + ", but doesn't run any tests.",
                        allScriptArgs: []string{
                                // Filtering pattern to buildall.bash:                                                                                                                
                                rx,
                        },
                })
        }
        addMiscCompile("", "^(linux-arm64|linux-s390x|solaris-amd64|darwin-386)$") // 4 ports                                                                                         
        addMiscCompile("-mobile", "(^android|darwin-arm64)")                       // 5 ports                                                                                         
        addMiscCompile("-nacl", "^nacl")                                           // 3                                                                                               
        addMiscCompile("-mips", "^linux-mips")                                     // 4                                                                                               
        addMiscCompile("-ppc", "^linux-ppc64")                                     // 2                                                                                               
        addMiscCompile("-plan9", "^plan9-")                                        // 3                                                                                               
        addMiscCompile("-freebsd", "^freebsd-")                                    // 3                                                                                               
        addMiscCompile("-netbsd", "^netbsd-")                                      // 3                                                                                               
        addMiscCompile("-openbsd", "^openbsd-")                                    // 3                                                                                               

Again, the goal of TryBots is to be fast and have decent coverage to catch most mistakes, but not all.

@heschik

This comment has been minimized.

Copy link
Contributor

commented Jun 13, 2018

Cross compiling doesn't crash, but the resulting binaries are still broken. It'd be hard to know that without running them though.

The most obvious problem is that the location of .shstrtab is wrong, which is why the section names are gibberish. In the hello world program I built, .shstrtab in the ELF headers:

  [ 3]                   STRTAB           0000000000000000  000d3d00
       0000000000000193  0000000000000000           0     0     1

But looking at the actual binary, the correct location is 0xd4d00. I'll poke around for my own curiosity but Austin is probably better suited for this.

@aclements

This comment has been minimized.

Copy link
Member

commented Jun 13, 2018

Actually the whole section table appears to be wrong. Before this CL for the cmd/go binary "There are 37 section headers, starting at offset 0x270", after this CL "There are 24 section headers, starting at offset 0x1c8". And the section header types don't match up at all (which you would expect if it was just the names that were broken).

@heschik

This comment has been minimized.

Copy link
Contributor

commented Jun 13, 2018

Okay. I was able to fix the problems with the section headers, As always with the linker I'm using a mix of guesswork and brute force, but I think the immediate problem is in datoff(). It only looks at Segtext and Segdata, but .shstrtab is in Segrodata. On amd64, rodata comes right after text (other than a bit of padding) and so that doesn't matter, but on arm64 and ppc64le there seems to be a sizeable gap for some reason. Changing datoff() to check against Segrodata fixes the section headers:

	for _, seg := range []*sym.Segment{&Segdwarf, &Segdata, &Segrodata, &Segtext} {
		if uint64(addr) >= seg.Vaddr {
			return int64(uint64(addr) - Segdata.Vaddr + Segdata.Fileoff)
		}
	}

but probably the bigger problem is the gap. I presume it's new after Austin's CL, though I didn't check.

Austin, I think you were probably external linking and the external threw away a bunch of sections that it didn't know what to do with? I only see 24 sections in my binaries.

@aclements

This comment has been minimized.

Copy link
Member

commented Jun 13, 2018

FWIW, when I originally wrote the patch I verified that binaries were identical on all supported OSes and arches. Either I did that wrong (certainly possible) or something happened in the rebase to master.

@aclements

This comment has been minimized.

Copy link
Member

commented Jun 13, 2018

@heschik, you're right that it looked completely different for me because I was external linking.

You're also right that there are some gaps that were added by my CL. In some cases (the strings test binary on linux/mips64) the binaries are even different sizes.

@aclements

This comment has been minimized.

Copy link
Member

commented Jun 13, 2018

The gaps are happening because previously we computed Fileoff as round(seg.Vaddr) - prev.Vaddr + prev.Fileoff and now we're computing it as round(seg.Vaddr - prev.Vaddr) + prev.Fileoff. The previous computation was "more correct" in the sense the it applied the rounding to virtual addresses rather than file offsets, but of course the point of the change was to get away from depending on the virtual address layout to match the file layout.

@gopherbot

This comment has been minimized.

Copy link

commented Jun 13, 2018

Change https://golang.org/cl/118716 mentions this issue: cmd/link: separate virtual address layout from file layout

@gopherbot gopherbot closed this in 8998c55 Jun 15, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.