New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/link: build broken on plan9/amd64 since CL 20928 #14894

Open
0intro opened this Issue Mar 21, 2016 · 16 comments

Comments

Projects
None yet
7 participants
@0intro
Member

0intro commented Mar 21, 2016

Since CL 20928, the build is broken on plan9/amd64.

It always fail when reading symbols from the buffer in readsym, but not always at the same place.

Example 1:

cmd/compile
# cmd/compile
2016/03/21 01:38:22 bad pcdata -2
cmd/compile/internal/ssa/gen
cmd/cover
cmd/dist
cmd/doc
cmd/fix
cmd/go
# cmd/go
panic: runtime error: slice bounds out of range

goroutine 1 [running]:
panic(0x42af80, 0x8872010)
    /tmp/gobuilder/plan9-amd64-9front-36d5650a1c60/go/src/runtime/panic.go:483 +0x3fd
bootstrap/link/internal/ld.rddata(0xa941ec0, 0x95652d8, 0x0, 0x0, 0x0)
    /tmp/gobuilder/plan9-amd64-9front-36d5650a1c60/go/src/cmd/link/internal/ld/objfile.go:490 +0xd7
bootstrap/link/internal/ld.readsym(0x8ce4000, 0xa941ec0, 0x95652d8, 0x8873140, 0x8, 0xa8e7aa0, 0x54)
    /tmp/gobuilder/plan9-amd64-9front-36d5650a1c60/go/src/cmd/link/internal/ld/objfile.go:290 +0x1251
bootstrap/link/internal/ld.ldobjfile(0x8ce4000, 0xa941ec0, 0x8873140, 0x8, 0x1e5b84, 0xa8e7aa0, 0x54)
    /tmp/gobuilder/plan9-amd64-9front-36d5650a1c60/go/src/cmd/link/internal/ld/objfile.go:165 +0xd84
bootstrap/link/internal/ld.ldobj(0xa941ec0, 0x8873140, 0x8, 0x1e5bf1, 0xa8e7aa0, 0x54, 0x88955e0, 0x4c, 0x1, 0x0)
    /tmp/gobuilder/plan9-amd64-9front-36d5650a1c60/go/src/cmd/link/internal/ld/lib.go:1350 +0x1413
bootstrap/link/internal/ld.objfile(0x887f880)
    /tmp/gobuilder/plan9-amd64-9front-36d5650a1c60/go/src/cmd/link/internal/ld/lib.go:846 +0xe2a
bootstrap/link/internal/ld.loadlib()
    /tmp/gobuilder/plan9-amd64-9front-36d5650a1c60/go/src/cmd/link/internal/ld/lib.go:523 +0x5d4
bootstrap/link/internal/ld.Ldmain()
    /tmp/gobuilder/plan9-amd64-9front-36d5650a1c60/go/src/cmd/link/internal/ld/pobj.go:187 +0x160c
bootstrap/link/internal/amd64.Main()
    /tmp/gobuilder/plan9-amd64-9front-36d5650a1c60/go/src/cmd/link/internal/amd64/obj.go:44 +0x1e
main.main()
    /tmp/gobuilder/plan9-amd64-9front-36d5650a1c60/go/src/cmd/link/main.go:27 +0x374

See http://build.golang.org/log/c3fee7600dc4ac359dd804577a201beb008d64d5

Where src/cmd/link/internal/ld/objfile.go:290 is:

pc.Pcfile.P = rddata(f, buf)

Example 2:

cmd/compile/internal/ssa/gen
# cmd/compile/internal/ssa/gen
panic: runtime error: index out of range

goroutine 1 [running]:
panic(0x371d40, 0x8870000)
    /tmp/gobuilder/plan9-amd64-9front-060a6915d4e3/go/src/runtime/panic.go:500 +0x181
bootstrap/link/internal/ld.rdsym(0x8cf8000, 0x8e11ce0, 0x3ac71d, 0x7, 0x8e1fbf0)
    /tmp/gobuilder/plan9-amd64-9front-060a6915d4e3/go/src/cmd/link/internal/ld/objfile.go:470 +0x56
bootstrap/link/internal/ld.readsym(0x8cf8000, 0x8e11ce0, 0x908b4c0, 0x3ac71d, 0x7, 0x8892ea0, 0x53)
    /tmp/gobuilder/plan9-amd64-9front-060a6915d4e3/go/src/cmd/link/internal/ld/objfile.go:274 +0x5c2
bootstrap/link/internal/ld.ldobjfile(0x8cf8000, 0x8e11ce0, 0x3ac71d, 0x7, 0x16202c, 0x8892ea0, 0x53)
    /tmp/gobuilder/plan9-amd64-9front-060a6915d4e3/go/src/cmd/link/internal/ld/objfile.go:164 +0x75d
bootstrap/link/internal/ld.ldobj(0x8e11ce0, 0x3ac71d, 0x7, 0x1620a8, 0x8892ea0, 0x53, 0x8880280, 0x4b, 0x1, 0x0)
    /tmp/gobuilder/plan9-amd64-9front-060a6915d4e3/go/src/cmd/link/internal/ld/lib.go:1350 +0x6d8
bootstrap/link/internal/ld.objfile(0x887c2a0)
    /tmp/gobuilder/plan9-amd64-9front-060a6915d4e3/go/src/cmd/link/internal/ld/lib.go:846 +0x501
bootstrap/link/internal/ld.loadlib()
    /tmp/gobuilder/plan9-amd64-9front-060a6915d4e3/go/src/cmd/link/internal/ld/lib.go:523 +0x165
bootstrap/link/internal/ld.Ldmain()
    /tmp/gobuilder/plan9-amd64-9front-060a6915d4e3/go/src/cmd/link/internal/ld/pobj.go:187 +0x122c
bootstrap/link/internal/amd64.Main()
    /tmp/gobuilder/plan9-amd64-9front-060a6915d4e3/go/src/cmd/link/internal/amd64/obj.go:44 +0x1e
main.main()
    /tmp/gobuilder/plan9-amd64-9front-060a6915d4e3/go/src/cmd/link/main.go:27 +0x281

See http://build.golang.org/log/48db2f11592b3b99c9ec92ac9e764d12c2c10a34

Where src/cmd/link/internal/ld/objfile.go:274 is:

Gotype:  rdsym(ctxt, f, pkg),

Example 3:

cmd/yacc
# cmd/yacc
panic: runtime error: slice bounds out of range

goroutine 1 [running]:
panic(0x371c80, 0x8870010)
    /tmp/gobuilder/plan9-amd64-9front-4402ee9fa3fd/go/src/runtime/panic.go:500 +0x181
bootstrap/link/internal/ld.rddata(0x8dabf80, 0x907b4b8, 0x24, 0x9370e00, 0x24)
    /tmp/gobuilder/plan9-amd64-9front-4402ee9fa3fd/go/src/cmd/link/internal/ld/objfile.go:431 +0x85
bootstrap/link/internal/ld.readsym(0x8cf8000, 0x8dabf80, 0x907b4b8, 0x3ac636, 0x7, 0x8892de0, 0x53)
    /tmp/gobuilder/plan9-amd64-9front-4402ee9fa3fd/go/src/cmd/link/internal/ld/objfile.go:287 +0x810
bootstrap/link/internal/ld.ldobjfile(0x8cf8000, 0x8dabf80, 0x3ac636, 0x7, 0x16202c, 0x8892de0, 0x53)
    /tmp/gobuilder/plan9-amd64-9front-4402ee9fa3fd/go/src/cmd/link/internal/ld/objfile.go:164 +0x75d
bootstrap/link/internal/ld.ldobj(0x8dabf80, 0x3ac636, 0x7, 0x1620a8, 0x8892de0, 0x53, 0x8880190, 0x4b, 0x1, 0x0)
    /tmp/gobuilder/plan9-amd64-9front-4402ee9fa3fd/go/src/cmd/link/internal/ld/lib.go:1350 +0x6d8
bootstrap/link/internal/ld.objfile(0x887c230)
    /tmp/gobuilder/plan9-amd64-9front-4402ee9fa3fd/go/src/cmd/link/internal/ld/lib.go:846 +0x501
bootstrap/link/internal/ld.loadlib()
    /tmp/gobuilder/plan9-amd64-9front-4402ee9fa3fd/go/src/cmd/link/internal/ld/lib.go:523 +0x165
bootstrap/link/internal/ld.Ldmain()
    /tmp/gobuilder/plan9-amd64-9front-4402ee9fa3fd/go/src/cmd/link/internal/ld/pobj.go:187 +0x122c
bootstrap/link/internal/amd64.Main()
    /tmp/gobuilder/plan9-amd64-9front-4402ee9fa3fd/go/src/cmd/link/internal/amd64/obj.go:44 +0x1e
main.main()
    /tmp/gobuilder/plan9-amd64-9front-4402ee9fa3fd/go/src/cmd/link/main.go:27 +0x281

http://build.golang.org/log/63f0bac67454da50570bb9cb661f9058ff9a063e

Where src/cmd/link/internal/ld/objfile.go:287 is:

pc.Pcdata[i].P = rddata(f, buf)

@0intro 0intro added the OS-Plan9 label Mar 21, 2016

@0intro 0intro self-assigned this Mar 21, 2016

@0intro 0intro added this to the Go1.7 milestone Mar 21, 2016

@0intro

This comment has been minimized.

Member

0intro commented Mar 21, 2016

@0intro

This comment has been minimized.

Member

0intro commented Mar 21, 2016

CC @4ad

@0intro

This comment has been minimized.

Member

0intro commented Mar 21, 2016

I've looked at the builder history and noticed a few cases where
similar errors in cmd/link happened when running the tests.

I think this issue in another consequence of the same issue.

For example, on 2016-02-29:

--- FAIL: TestIssue6480 (0.30s)
    go_test.go:244: running testgo [test -c -test.bench=XXX errors]
    go_test.go:263: standard error:
    go_test.go:264: # testmain
        2016/02/29 00:23:46 -5894 out of range for uint8
        panic: -5894 out of range for uint8

        goroutine 1 [running]:
        panic(0x3ec960, 0x99f6cc0)
            /tmp/gobuilder/plan9-amd64-9front-75cc05fa557b/go/src/runtime/panic.go:483 +0x3fd
        log.Panicf(0x493bc0, 0x19, 0x91f0c68, 0x1, 0x1)
            /tmp/gobuilder/plan9-amd64-9front-75cc05fa557b/go/src/log/log.go:327 +0xdd
        cmd/link/internal/ld.rduint8(0x8cba120, 0x24)
            /tmp/gobuilder/plan9-amd64-9front-75cc05fa557b/go/src/cmd/link/internal/ld/objfile.go:422 +0xfa
        cmd/link/internal/ld.readsym(0x88b0240, 0x8cba120, 0x8c32aca, 0x4, 0x9368460, 0x50)
            /tmp/gobuilder/plan9-amd64-9front-75cc05fa557b/go/src/cmd/link/internal/ld/objfile.go:245 +0xccd
        cmd/link/internal/ld.ldobjfile(0x88b0240, 0x8cba120, 0x8c32aca, 0x4, 0x725ba, 0x9368460, 0x50)
            /tmp/gobuilder/plan9-amd64-9front-75cc05fa557b/go/src/cmd/link/internal/ld/objfile.go:147 +0xa67
        cmd/link/internal/ld.ldobj(0x8cba120, 0x8c32aca, 0x4, 0x72627, 0x9368460, 0x50, 0x93fb180, 0x48, 0x1, 0x0)
            /tmp/gobuilder/plan9-amd64-9front-75cc05fa557b/go/src/cmd/link/internal/ld/lib.go:1339 +0x155f
        cmd/link/internal/ld.objfile(0x941a2a0)
            /tmp/gobuilder/plan9-amd64-9front-75cc05fa557b/go/src/cmd/link/internal/ld/lib.go:835 +0xee0
        cmd/link/internal/ld.loadlib()
            /tmp/gobuilder/plan9-amd64-9front-75cc05fa557b/go/src/cmd/link/internal/ld/lib.go:510 +0x5cb
        cmd/link/internal/ld.Ldmain()
            /tmp/gobuilder/plan9-amd64-9front-75cc05fa557b/go/src/cmd/link/internal/ld/pobj.go:187 +0x162c
        cmd/link/internal/amd64.Main()
            /tmp/gobuilder/plan9-amd64-9front-75cc05fa557b/go/src/cmd/link/internal/amd64/obj.go:44 +0x1e
        main.main()
            /tmp/gobuilder/plan9-amd64-9front-75cc05fa557b/go/src/cmd/link/main.go:27 +0x374

    go_test.go:273: go [test -c -test.bench=XXX errors] failed unexpectedly: exit status: 'testgo 292097: 2'
FAIL
FAIL    cmd/go  43.226s

Where src/cmd/link/internal/ld/objfile.go:245 is:

r.Siz = rduint8(f)

See http://build.golang.org/log/b6a6656739598fd705fc25bab896f998bb3db8b4

@mwhudson

This comment has been minimized.

Contributor

mwhudson commented Mar 21, 2016

Urgh. Can I reproduce this somehow? The only thing I can really think of is to print out debugging spew about how much data is written into an object file and how much is read out again. I suppose the call to Bread in readsym should check its return value, at least... although it seems more likely that the problem is that it's trying to read too many bytes from the data block, rather than failing to read all the data.

@0intro

This comment has been minimized.

Member

0intro commented Mar 22, 2016

The only thing I can really think of is to print out debugging spew about how much data is written into an object file and how much is read out again.

Yes, this is what I wanted to do. However the problem disappears when I add print statements.
There is definitely something broken. I'll investigate.

@mwhudson

This comment has been minimized.

Contributor

mwhudson commented Mar 23, 2016

OK, so I'll hold off doing anything while you do your investigations -- if that's ok with you.

@mdempsky

This comment has been minimized.

Member

mdempsky commented Mar 29, 2016

@0intro Any progress?

@0intro

This comment has been minimized.

Member

0intro commented Mar 29, 2016

Not yet. I can reproduce the issue on 9front's amd64 kernel (which the builder is running), but not on 9k, though the interfaces are similar.

I get the same issue when the linker has been cross-compiled from Linux. The issue goes away when I add print statements in src/cmd/link/internal/ld/objfile.go or when I run go build -v.

Any help would be appreciated. I can provide a QEMU image running 9front/amd64 if needed.

@mdempsky

This comment has been minimized.

Member

mdempsky commented Mar 29, 2016

If you can provide dummy proof instructions on how to repro the issue with either QEMU or VMware, I can help investigate.

@0intro

This comment has been minimized.

Member

0intro commented Mar 29, 2016

I've built a disk image running 9front. It is available on http://9legacy.org/download/go/2016-03-29/9front.qcow2.bz2 (516 MB)

You can run it under QEMU with the following command:

qemu-system-x86_64 -machine accel=kvm -net user -net nic,model=virtio -m 3072 -vga std -drive if=none,id=hd,file=9front.qcow2 -device virtio-scsi-pci,id=scsi -device scsi-hd,drive=hd -boot c

After booting the image, you get rio running with a terminal open.
An installation of Go (current master branch) is available in /usr/go. So you can run:

cd /usr/go/src && all.rc

Other installations of Go 1.4 to 1.7 (pre-built) are available in /usr/glenda/go1.[4567]. In the current profile, the GOROOT_BOOTSTRAP variable is set to /usr/glenda/go1.7.

You can edit files with sam or acme. A minimal set of Git commands is available, so you can run git pull to update the tree, or git checkout to check another branch/tag/sha1 or git clone to clone a repository.

I've noticed that I can reproduce the issue only if the virtual machine has been configured with more than 2480 MB of memory, approximately.
The builder used to have 4 GB of memory, and I've just lowered it to 2 GB to keep the build going.

@mdempsky

This comment has been minimized.

Member

mdempsky commented Mar 30, 2016

@0intro Thanks. I was able to get the VM running and to run make.rc.

Unfortunately, I've run make.rc 3 times so far, and it's successfully built each time. I'll continue trying, but at the moment I can't repro the failure.

What version of QEMU are you using? I appear to have:

qemu-x86_64 version 2.0.0 (Debian 2.0.0+dfsg-2ubuntu1.22), Copyright (c) 2003-2008 Fabrice Bellard

provided by Ubuntu.

@mdempsky

This comment has been minimized.

Member

mdempsky commented Mar 30, 2016

Nevermind, I managed to repro it after a few tries.

@bradfitz bradfitz modified the milestones: Go1.7Maybe, Go1.7 May 10, 2016

@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Jun 9, 2016

@mdempsky Any idea what is happening here?

@mdempsky

This comment has been minimized.

Member

mdempsky commented Jun 9, 2016

@ianlancetaylor Unfortunately not yet. I was able to repro it a few times following @0intro's instructions, but got side tracked figuring out how to work effectively within Plan 9.

@adg adg modified the milestones: Unplanned, Go1.7Maybe Jun 27, 2016

@adg

This comment has been minimized.

Contributor

adg commented Jun 27, 2016

Moved to unplanned as this will not block any release, as plan9 is not a 'first class port'.

@driusan

This comment has been minimized.

driusan commented Feb 17, 2017

I've been using 9front/amd64 as my dev machine for about a week now, and I get errors of this sort about once every 100-150 times when I run go install. Is there anything I can do to help debug this? And is it possible that it has something to do with the fact that 9front uses cwfs instead of fossil?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment