Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: crash on ARMv5TE when running tests #5466

Closed
gopherbot opened this issue May 14, 2013 · 34 comments
Closed

runtime: crash on ARMv5TE when running tests #5466

gopherbot opened this issue May 14, 2013 · 34 comments

Comments

@gopherbot
Copy link
Contributor

by bga@bug-br.org.br:

What steps will reproduce the problem?
If possible, include a link to a program on play.golang.org.
1. Download the Go 1.1 source code from the usual location.
2. Try to compile it on an ARM5 device (in my case a QNAP TS-219 running Debian Linux)
by running all.bash (make sure to set GOARM=5, just in case).
3. Wait for it to compile and the tests to be executed.

What is the expected output?

It compiles and tests get xecuted without issues.

What do you see instead?

[...]
ok      crypto/hmac 0.069s
ok      crypto/md5  0.065s
SIGILL: illegal instruction
PC=0x4e5154


goroutine 1 [semacquire]:
sync.runtime_Semacquire(0x10c8ed88)
    /usr/local/go/src/pkg/runtime/zsema_linux_arm.c:165 +0x30
sync.(*WaitGroup).Wait(0x10cfd940)
    /usr/local/go/src/pkg/sync/waitgroup.go:109 +0x118
main.(*builder).do(0x108573c0, 0x10d028c0)
    /usr/local/go/src/cmd/go/build.go:715 +0x428
main.runTest(0x4df438, 0x10668010, 0x3, 0x3)
    /usr/local/go/src/cmd/go/test.go:330 +0x1ff8
main.main()
    /usr/local/go/src/cmd/go/main.go:156 +0x4ec

goroutine 2 [syscall]:

goroutine 4 [syscall]:
created by os/signal.init·1
    /usr/local/go/src/pkg/os/signal/signal_unix.go:27 +0x44

goroutine 6 [running]:
syscall.Syscall()
    /usr/local/go/src/pkg/syscall/asm_linux_arm.s:15 +0x8
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:712 +0x40c

goroutine 26 [finalizer wait]:

goroutine 14 [chan receive]:
main.func·026()
    /usr/local/go/src/cmd/go/signal.go:21 +0x34
created by main.processSignals
    /usr/local/go/src/cmd/go/signal.go:23 +0xc0

trap    0x6
error   0x0
oldmask 0x0
r0      0x1
r1      0x40dfcd80
r2      0x40dfc954
r3      0x9e4a0046
r4      0x12fe131d
r5      0x4e5154
r6      0x1
r7      0xf0
r8      0x2
r9      0x1066f400
r10     0x1065f360
fp      0x2b
ip      0x400b59f0
sp      0x40dfc95c
lr      0x7f0e0
pc      0x4e5154
cpsr    0x80000010
fault   0x0

Which compiler are you using (5g, 6g, 8g, gccgo)?

Whatever all.bash uses by default when running the tests after Go is built.

Which operating system are you using?

Debian Linux
uname -a output : Linux blindio 2.6.32-5-kirkwood #1 Sun May 6 16:57:51 UTC 2012
armv5tel GNU/Linux

Which version are you using?  (run 'go version')

Go 1.1.

Please provide any additional information below.

cat /proc/cpuinfo
Processor   : Feroceon 88FR131 rev 1 (v5l)
BogoMIPS    : 1199.30
Features    : swp half thumb fastmult edsp 
CPU implementer : 0x56
CPU architecture: 5TE
CPU variant : 0x2
CPU part    : 0x131
CPU revision    : 1

Hardware    : QNAP TS-119/TS-219
Revision    : 0000
Serial      : 0000000000000000

cat /proc/cpu/alignment
User:       0
System:     0
Skipped:    0
Half:       0
Word:       0
DWord:      0
Multi:      0
User faults:    0 (ignored)

Dmesg does not seem to have anything interesting. just in case, I am pasting it here too:

dmesg
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Linux version 2.6.32-5-kirkwood (Debian 2.6.32-45) (dannf@debian.org)
(gcc version 4.3.5 (Debian 4.3.5-4) ) #1 Sun May 6 16:57:51 UTC 2012
[    0.000000] CPU: Feroceon 88FR131 [56251311] revision 1 (ARMv5TE), cr=00053977
[    0.000000] CPU: VIVT data cache, VIVT instruction cache
[    0.000000] Machine: QNAP TS-119/TS-219
[    0.000000] Ignoring unrecognised tag 0x41000403
[    0.000000] Memory policy: ECC disabled, Data cache writeback
[    0.000000] On node 0 totalpages: 131072
[    0.000000] free_area_init_node: node 0, pgdat c03d50c4, node_mem_map c0442000
[    0.000000]   Normal zone: 1024 pages used for memmap
[    0.000000]   Normal zone: 0 pages reserved
[    0.000000]   Normal zone: 130048 pages, LIFO batch:31
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages:
130048
[    0.000000] Kernel command line: console=ttyS0,115200 root=/dev/ram
initrd=0xa00000,0x900000 ramdisk=32768
[    0.000000] PID hash table entries: 2048 (order: 1, 8192 bytes)
[    0.000000] Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
[    0.000000] Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
[    0.000000] Memory: 256MB 256MB = 512MB total
[    0.000000] Memory: 505984KB available (3516K code, 583K data, 124K init, 0K highmem)
[    0.000000] SLUB: Genslabs=11, HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000] NR_IRQS:114
[    0.000000] Console: colour dummy device 80x30
[    0.000137] Calibrating delay loop... 1199.30 BogoMIPS (lpj=5996544)
[    0.230092] Security Framework initialized
[    0.230111] SELinux:  Disabled at boot.
[    0.230138] Mount-cache hash table entries: 512
[    0.230454] Initializing cgroup subsys ns
[    0.230470] Initializing cgroup subsys cpuacct
[    0.230480] Initializing cgroup subsys devices
[    0.230489] Initializing cgroup subsys freezer
[    0.230498] Initializing cgroup subsys net_cls
[    0.230557] CPU: Testing write buffer coherency: ok
[    0.231337] devtmpfs: initialized
[    0.233041] regulator: core version 0.5
[    0.233263] NET: Registered protocol family 16
[    0.233794] Kirkwood: MV88F6281-A1, TCLK=200000000.
[    0.233807] Feroceon L2: Enabling L2
[    0.233840] Feroceon L2: Cache support initialised.
[    0.234274] initial MPP regs: 21112222 11313311 00551110 00000000 00000000 00000000
00000000
[    0.234297]   final MPP regs: 21552222 03313311 00551110 00000000 00000000 00000000
00000000
[    0.234748] Kirkwood PCIe port 0: 
[    0.234754] link up
[    0.234761] PCI: bus0 uses PCIe port 0
[    0.234865] pci 0000:00:00.0: reg 10 64bit mmio pref: [0xf1000000-0xf10fffff]
[    0.234880] pci 0000:00:00.0: reg 18 32bit mmio: [0x000000-0x1fffffff]
[    0.234907] pci 0000:00:00.0: supports D1 D2
[    0.234991] pci 0000:00:01.0: reg 24 32bit mmio: [0x40000000-0x40001fff]
[    0.235007] pci 0000:00:01.0: reg 30 32bit mmio pref: [0x000000-0x00ffff]
[    0.235044] pci 0000:00:01.0: PME# supported from D3hot
[    0.235055] pci 0000:00:01.0: PME# disabled
[    0.235110] pci 0000:00:01.1: reg 10 io port: [0xfff8-0xffff]
[    0.235125] pci 0000:00:01.1: reg 14 io port: [0xfffc-0xffff]
[    0.235139] pci 0000:00:01.1: reg 18 io port: [0xfff8-0xffff]
[    0.235154] pci 0000:00:01.1: reg 1c io port: [0xfffc-0xffff]
[    0.235168] pci 0000:00:01.1: reg 20 io port: [0xfff0-0xffff]
[    0.235254] PCI: bus0: Fast back to back transfers disabled
[    0.236623] bio: create slab <bio-0> at 0
[    0.236909] vgaarb: loaded
[    0.237359] Switching to clocksource orion_clocksource
[    0.241227] NET: Registered protocol family 2
[    0.241464] IP route cache hash table entries: 4096 (order: 2, 16384 bytes)
[    0.242228] TCP established hash table entries: 16384 (order: 5, 131072 bytes)
[    0.242580] TCP bind hash table entries: 16384 (order: 4, 65536 bytes)
[    0.242758] TCP: Hash tables configured (established 16384 bind 16384)
[    0.242767] TCP reno registered
[    0.242939] NET: Registered protocol family 1
[    0.243121] Unpacking initramfs...
[    0.609095] Freeing initrd memory: 9216K
[    0.609195] NetWinder Floating Point Emulator V0.97 (double precision)
[    0.609442] audit: initializing netlink socket (disabled)
[    0.609476] type=2000 audit(0.600:1): initialized
[    0.615995] VFS: Disk quotas dquot_6.5.2
[    0.616247] Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
[    0.616346] JFFS2 version 2.2. (NAND) (SUMMARY)  © 2001-2006 Red Hat, Inc.
[    0.616932] msgmni has been set to 1006
[    0.618694] alg: No test for stdrng (krng)
[    0.618822] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
[    0.618835] io scheduler noop registered
[    0.618842] io scheduler anticipatory registered
[    0.618850] io scheduler deadline registered
[    0.619028] io scheduler cfq registered (default)
[    0.625102] Serial: 8250/16550 driver, 2 ports, IRQ sharing disabled
[    0.625572] serial8250.0: ttyS0 at MMIO 0xf1012000 (irq = 33) is a 16550A
[    0.976681] console [ttyS0] enabled
[    0.980445] serial8250.1: ttyS1 at MMIO 0xf1012100 (irq = 34) is a 16550A
[    0.988234] m25p80 spi0.0: m25p128 (16384 Kbytes)
[    0.992963] Creating 6 MTD partitions on "spi_flash":
[    0.998063] 0x000000000000-0x000000080000 : "U-Boot"
[    1.003654] 0x000000200000-0x000000400000 : "Kernel"
[    1.009138] 0x000000400000-0x000000d00000 : "RootFS1"
[    1.014654] 0x000000d00000-0x000001000000 : "RootFS2"
[    1.020183] 0x000000080000-0x0000000c0000 : "U-Boot Config"
[    1.026212] 0x0000000c0000-0x000000200000 : "NAS Config"
[    1.032224] mice: PS/2 mouse device common for all mice
[    2.047398] rtc-mv rtc-mv: internal RTC not ticking
[    2.052394] i2c /dev entries driver
[    2.057795] rtc-s35390a 0-0030: rtc core: registered rtc-s35390a as rtc0
[    2.064745] cpuidle: using governor ladder
[    2.069004] cpuidle: using governor menu
[    2.073005] mv_xor_shared mv_xor_shared.0: Marvell shared XOR driver
[    2.079420] mv_xor_shared mv_xor_shared.1: Marvell shared XOR driver
[    2.117419] mv_xor mv_xor.0: Marvell XOR: ( xor cpy )
[    2.157417] mv_xor mv_xor.1: Marvell XOR: ( xor fill cpy )
[    2.197418] mv_xor mv_xor.2: Marvell XOR: ( xor cpy )
[    2.237419] mv_xor mv_xor.3: Marvell XOR: ( xor fill cpy )
[    2.244081] TCP cubic registered
[    2.247325] NET: Registered protocol family 17
[    2.251821] Gating clock of unused units
[    2.251829] before: 0x00c7c1dd
[    2.251836]  after: 0x00c7c1cd
[    2.252164] registered taskstats version 1
[    2.257683] rtc-s35390a 0-0030: setting system clock to 2013-05-14 13:55:25 UTC
(1368539725)
[    2.266211] Initalizing network drop monitor service
[    2.271275] Freeing init memory: 124K
[    2.357840] udev[44]: starting version 164
[    2.646260] SCSI subsystem initialized
[    2.728189] libata version 3.00 loaded.
[    2.745904] sata_mv sata_mv.0: version 1.28
[    2.750262] sata_mv sata_mv.0: slots 32 ports 2
[    2.755514] scsi0 : sata_mv
[    2.759011] scsi1 : sata_mv
[    2.762270] ata1: SATA max UDMA/133 irq 21
[    2.766383] ata2: SATA max UDMA/133 irq 21
[    3.277411] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl F300)
[    3.317438] ata1.00: ATA-8: ST31000528AS, CC38, max UDMA/133
[    3.323129] ata1.00: 1953525168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[    3.387489] ata1.00: configured for UDMA/133
[    3.392081] scsi 0:0:0:0: Direct-Access     ATA      ST31000528AS     CC38 PQ: 0
ANSI: 5
[    3.907407] ata2: SATA link up 3.0 Gbps (SStatus 123 SControl F300)
[    3.967439] ata2.00: ATA-8: WDC WD10EARS-00Y5B1, 80.00A80, max UDMA/133
[    3.974086] ata2.00: 1953525168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[    4.017490] ata2.00: configured for UDMA/133
[    4.022054] scsi 1:0:0:0: Direct-Access     ATA      WDC WD10EARS-00Y 80.0 PQ: 0
ANSI: 5
[    4.064829] sd 0:0:0:0: [sda] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB)
[    4.072991] sd 1:0:0:0: [sdb] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB)
[    4.081812] sd 0:0:0:0: [sda] Write Protect is off
[    4.086627] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[    4.086767] sd 1:0:0:0: [sdb] Write Protect is off
[    4.091605] sd 1:0:0:0: [sdb] Mode Sense: 00 3a 00 00
[    4.092046] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't
support DPO or FUA
[    4.101258] sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't
support DPO or FUA
[    4.111463]  sda:
[    4.113315]  sdb: sdb1
[    4.129921] sd 1:0:0:0: [sdb] Attached SCSI disk
[    4.134641]  sda1
[    4.138460] sd 0:0:0:0: [sda] Attached SCSI disk
[    4.430995] md: raid1 personality registered for level 1
[    4.451046] mdadm: sending ioctl 1261 to a partition!
[    4.456128] mdadm: sending ioctl 1261 to a partition!
[    4.461963] mdadm: sending ioctl 1261 to a partition!
[    4.467045] mdadm: sending ioctl 1261 to a partition!
[    4.496496] md: md0 stopped.
[    4.500576] mdadm: sending ioctl 1261 to a partition!
[    4.505657] mdadm: sending ioctl 1261 to a partition!
[    4.513756] mdadm: sending ioctl 1261 to a partition!
[    4.518867] mdadm: sending ioctl 1261 to a partition!
[    4.533558] mdadm: sending ioctl 1261 to a partition!
[    4.538683] mdadm: sending ioctl 1261 to a partition!
[    4.552676] md: bind<sdb1>
[    4.555706] md: bind<sda1>
[    4.563216] raid1: raid set md0 active with 2 out of 2 mirrors
[    4.569257] md0: detected capacity change from 0 to 1000201904128
[    4.579552]  md0: unknown partition table
[    4.819999] device-mapper: uevent: version 1.0.3
[    4.826776] device-mapper: ioctl: 4.15.0-ioctl (2009-04-01) initialised:
dm-devel@redhat.com
[    5.447294] EXT4-fs (dm-0): INFO: recovery required on readonly filesystem
[    5.454260] EXT4-fs (dm-0): write access will be enabled during recovery
[    7.785537] EXT4-fs (dm-0): orphan cleanup on readonly fs
[    7.791205] EXT4-fs (dm-0): ext4_orphan_cleanup: deleting unreferenced inode 31982811
[    7.791437] EXT4-fs (dm-0): 1 orphan inode deleted
[    7.796269] EXT4-fs (dm-0): recovery complete
[    8.009557] EXT4-fs (dm-0): mounted filesystem with ordered data mode
[    9.766690] udev[192]: starting version 164
[   10.304695] input: gpio-keys as /devices/platform/gpio-keys/input/input0
[   10.461444] ahci 0000:00:01.0: version 3.0
[   10.477480] ahci 0000:00:01.0: AHCI 0001.0000 32 slots 2 ports 3 Gbps 0x3 impl SATA
mode
[   10.485622] ahci 0000:00:01.0: flags: 64bit ncq pm led clo pmp pio slum part 
[   10.494907] scsi2 : ahci
[   10.498114] scsi3 : ahci
[   10.501132] ata3: SATA max UDMA/133 abar m8192@0xe0010000 port 0xe0010100 irq 9
[   10.508701] ata4: SATA max UDMA/133 abar m8192@0xe0010000 port 0xe0010180 irq 9
[   10.634212] MV-643xx 10/100/1000 ethernet driver version 1.4
[   10.640200] mv643xx_eth smi: probed
[   10.659350] net eth0: port 0 with MAC address 00:08:9b:c4:46:d9
[   10.735404] usbcore: Unknown symbol current_is_keventd
[   10.741212] usbcore: Unknown symbol add_device_randomness
[   10.765253] __ratelimit: 2 callbacks suppressed
[   10.769850] mdadm: sending ioctl 1261 to a partition!
[   10.774930] mdadm: sending ioctl 1261 to a partition!
[   10.877491] ata3: SATA link down (SStatus 0 SControl 300)
[   10.897445] ata4: SATA link down (SStatus 0 SControl 300)
[   12.385717] mdadm: sending ioctl 1261 to a partition!
[   12.471829] loop: module loaded
[   12.524739] mdadm: sending ioctl 1261 to a partition!
[   12.529898] mdadm: sending ioctl 1261 to a partition!
[   13.105438] Adding 1499128k swap on /dev/mapper/blindio-swap.  Priority:-1 extents:1
across:1499128k 
[   15.671572] NET: Registered protocol family 10
[   15.679566] ADDRCONF(NETDEV_UP): eth0: link is not ready
[   18.255415] eth0: link up, 1000 Mb/s, full duplex, flow control disabled
[   18.262430] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   28.577394] eth0: no IPv6 routers present
[   30.340013] VFS (dm-0): Writes happened before quota was turned on thus quota
information is probably inconsistent. Please run quotacheck(8).
[   30.458883] VFS (dm-0): Writes happened before quota was turned on thus quota
information is probably inconsistent. Please run quotacheck(8).

(Yes, this is all of it)

Let me know if you need more information.
@bradfitz
Copy link
Contributor

Comment 1:

Owner changed to @minux.

@minux
Copy link
Member

minux commented May 14, 2013

Comment 2:

could you use gdb or objdump to obtain a disassembly listing around 0x4e5154?
for example, you could do this in gdb(1):
gdb $GOROOT/bin/go
x/10i 0x4e5148
or use objdump(1):
objdump -d --start-address=0x4e5148 --stop-address=0x4e516c $GOROOT/bin/go

Status changed to WaitingForReply.

@gopherbot
Copy link
Contributor Author

Comment 3 by bga@bug-br.org.br:

Sorry, I already started the compilation froma  clean slate. I will let you know when it
finishes.

@gopherbot
Copy link
Contributor Author

Comment 4 by bga@bug-br.org.br:

Ok, it crashed again, but now in a different point.
Also, it did not abort the tests and continued for a while until it seems it crashed
again:
First crash (formatting was weird):
[...]
ok      regexp  0.845s
ok      regexp/syntax   4.786s
--- FAIL: TestCgoSignalDeadlock (1.12 seconds)
    crash_cgo_test.go:21: expected "OK\n", but got "SIGILL: illegal instruction\nPC=0x4e5414\n\n\ngoroutine 1 [semacquire]:\nruntime.park(0x62fa8, 0x4ec6b8, 0x4dd018)\n\t/usr/local/go/src/pkg/runtime/proc.c:1175 +0x40\nsemacquireimpl(0x1074f810, 0x1)\n\t/usr/local/go/src/pkg/runtime/zsema_linux_arm.c:113 +0x120\nsync.runtime_Semacquire(0x1074f810)\n\t/usr/local/go/src/pkg/runtime/zsema_linux_arm.c:165 +0x30\nsync.(*WaitGroup).Wait(0x107969a0)\n\t/usr/local/go/src/pkg/sync/waitgroup.go:109 +0x118\nmain.(*builder).do(0x106f4940, 0x1076b310)\n\t/usr/local/go/src/cmd/go/build.go:715 +0x428\nmain.runRun(0x4df3e0, 0x10668010, 0x1, 0x1)\n\t/usr/local/go/src/cmd/go/run.go:86 +0x838\nmain.main()\n\t/usr/local/go/src/cmd/go/main.go:156 +0x4ec\nruntime.main()\n\t/usr/local/go/src/pkg/runtime/proc.c:182 +0x78\nruntime.goexit()\n\t/usr/local/go/src/pkg/runtime/proc.c:1223\n\ngoroutine 2 [syscall]:\nruntime.goexit()\n\t/usr/local/go/src/pkg/runtime/proc.c:1223\n\ngoroutine 3 [syscall]:\nruntime.entersyscallblock()\n\t/usr/local/go/src/pkg/runtime/proc.c:1333 +0xc0\nruntime.MHeap_Scavenger()\n\t/usr/local/go/src/pkg/runtime/mheap.c:453 +0xfc\nruntime.goexit()\n\t/usr/local/go/src/pkg/runtime/proc.c:1223\ncreated by runtime.main\n\t/usr/local/go/src/pkg/runtime/proc.c:165\n\ngoroutine 4 [syscall]:\nruntime.entersyscallblock()\n\t/usr/local/go/src/pkg/runtime/proc.c:1333 +0xc0\ncreated by os/signal.init·1\n\t/usr/local/go/src/pkg/os/signal/signal_unix.go:27 +0x44\n\ngoroutine 6 [running]:\nsyscall.Syscall()\n\t/usr/local/go/src/pkg/syscall/asm_linux_arm.s:15 +0x8\ncreated by main.(*builder).do\n\t/usr/local/go/src/cmd/go/build.go:712 +0x40c\n\ntrap    0x6\nerror   0x0\noldmask 0x0\nr0      0x1\nr1      0xfff8335f\nr2      0x40dfc954\nr3      0x57daa3c4\nr4      0x12fe1d91\nr5      0x4e5154\nr6      0x3e\nr7      0xf0\nr8      0x2\nr9      0x1066f400\nr10     0x1065f360\nfp      0x2b\nip      0x400b59f0\nsp      0x40dfc95c\nlr      0x4e51d6\npc      0x4e5414\ncpsr    0x20000010\nfault   0x0\n"
FAIL
FAIL    runtime 55.997s
?       runtime/cgo [no test files]
ok      runtime/debug   0.061s
ok      runtime/pprof   1.887s
?       runtime/race    [no test files]
[...]
And second crash:
[...]
ok      time    5.640s
ok      unicode 0.066s
ok      unicode/utf16   0.057s
SIGILL: illegal instruction
PC=0x4e5238
goroutine 1 [semacquire]:
runtime.park(0x62fa8, 0x4ec478, 0x4dd018)
    /usr/local/go/src/pkg/runtime/proc.c:1175 +0x40
semacquireimpl(0x10b86798, 0x1)
    /usr/local/go/src/pkg/runtime/zsema_linux_arm.c:113 +0x120
sync.runtime_Semacquire(0x10b86798)
    /usr/local/go/src/pkg/runtime/zsema_linux_arm.c:165 +0x30
sync.(*WaitGroup).Wait(0x10b8aec0)
    /usr/local/go/src/pkg/sync/waitgroup.go:109 +0x118
main.(*builder).do(0x1085a6c0, 0x10b6da80)
    /usr/local/go/src/cmd/go/build.go:715 +0x428
main.runTest(0x4df438, 0x10668010, 0x3, 0x3)
    /usr/local/go/src/cmd/go/test.go:330 +0x1ff8
main.main()
    /usr/local/go/src/cmd/go/main.go:156 +0x4ec
runtime.main()
    /usr/local/go/src/pkg/runtime/proc.c:182 +0x78
runtime.goexit()
    /usr/local/go/src/pkg/runtime/proc.c:1223
goroutine 2 [syscall]:
runtime.goexit()
    /usr/local/go/src/pkg/runtime/proc.c:1223
goroutine 3 [syscall]:
runtime.entersyscallblock()
    /usr/local/go/src/pkg/runtime/proc.c:1333 +0xc0
runtime.MHeap_Scavenger()
    /usr/local/go/src/pkg/runtime/mheap.c:453 +0xfc
runtime.goexit()
    /usr/local/go/src/pkg/runtime/proc.c:1223
created by runtime.main
    /usr/local/go/src/pkg/runtime/proc.c:165
goroutine 4 [syscall]:
runtime.entersyscallblock()
    /usr/local/go/src/pkg/runtime/proc.c:1333 +0xc0
created by os/signal.init·1
    /usr/local/go/src/pkg/os/signal/signal_unix.go:27 +0x44
goroutine 6 [running]:
syscall.Syscall()
    /usr/local/go/src/pkg/syscall/asm_linux_arm.s:15 +0x8
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:712 +0x40c
goroutine 13 [syscall]:
runtime.entersyscallblock()
    /usr/local/go/src/pkg/runtime/proc.c:1333 +0xc0
timerproc()
    /usr/local/go/src/pkg/runtime/ztime_linux_arm.c:194 +0xd4
runtime.goexit()
    /usr/local/go/src/pkg/runtime/proc.c:1223
created by addtimer
    /usr/local/go/src/pkg/runtime/ztime_linux_arm.c:82
goroutine 14 [chan receive]:
runtime.park(0x62fa8, 0x10be0688, 0x4e1baa)
    /usr/local/go/src/pkg/runtime/proc.c:1175 +0x40
runtime.chanrecv(0x293488, 0x10be0660, 0x405f7fd0, 0x0, 0x0, ...)
    /usr/local/go/src/pkg/runtime/chan.c:366 +0x258
runtime.chanrecv1()
    /usr/local/go/src/pkg/runtime/chan.c:458 +0x30
main.func·026()
    /usr/local/go/src/cmd/go/signal.go:21 +0x34
runtime.goexit()
    /usr/local/go/src/pkg/runtime/proc.c:1223
created by main.processSignals
    /usr/local/go/src/cmd/go/signal.go:23 +0xc0
goroutine 280 [finalizer wait]:
runtime.park(0x0, 0x0, 0x4e2d91)
    /usr/local/go/src/pkg/runtime/proc.c:1175 +0x40
runfinq()
    /usr/local/go/src/pkg/runtime/mgc0.c:2182 +0x68
runtime.goexit()
    /usr/local/go/src/pkg/runtime/proc.c:1223
created by runtime.gc
    /usr/local/go/src/pkg/runtime/mgc0.c:1886
trap    0x6
error   0x0
oldmask 0x0
r0      0x0
r1      0x1
r2      0x4e6fd0
r3      0x0
r4      0x0
r5      0x4e5154
r6      0x1
r7      0xf0
r8      0x0
r9      0x1066f400
r10     0x1065f360
fp      0x4ea0fc
ip      0x400b59f0
sp      0x40dfc94c
lr      0x7f0e0
pc      0x4e5238
cpsr    0x40000010
fault   0x0

@minux
Copy link
Member

minux commented May 14, 2013

Comment 5:

i still want to know the exact instruction that caused the SIGILL:
use gdb(1):
gdb $GOROOT/bin/go
x/10i 0x4e5230
or use objdump(1):
objdump -d --start-address=0x4e5230 --stop-address=0x4e5254 $GOROOT/bin/go
thank you.

@gopherbot
Copy link
Contributor Author

Comment 6 by bga@bug-br.org.br:

Something looks fishy it seems:
bga@blindio:~/go/src$ gdb ../bin/go
GNU gdb (GDB) 7.0.1-debian
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>;
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "arm-linux-gnueabi".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>;...
Reading symbols from /home/bga/go/bin/go...done.
(gdb) x/10i 0x4e5230
0x4e5230 <go/build.Default>:  andeq   r0, r0, r0
0x4e5234 <go/build.Default+4>:    andeq   r0, r0, r0
0x4e5238 <go/build.Default+8>:    andeq   r0, r0, r0
0x4e523c <go/build.Default+12>:   andeq   r0, r0, r0
0x4e5240 <go/build.Default+16>:   andeq   r0, r0, r0
0x4e5244 <go/build.Default+20>:   andeq   r0, r0, r0
0x4e5248 <go/build.Default+24>:   andeq   r0, r0, r0
0x4e524c <go/build.Default+28>:   andeq   r0, r0, r0
0x4e5250 <go/build.Default+32>:   andeq   r0, r0, r0
0x4e5254 <go/build.Default+36>:   andeq   r0, r0, r0
bga@blindio:~/go/src$ objdump -d --start-address=0x4e5230 --stop-address=0x4e5254
../bin/go
../bin/go:     file format elf32-littlearm

@davecheney
Copy link
Contributor

Comment 7:

The crash you are seeing is not the go tool, but the crash of the test binary (the name
of the package that failed will follow directly after the crash dump, matching the
string ^FAIL). If you cd to that package, run go test -c, then run the resulting
$PKG.test binary you should seen an identical crash, as well as be able to use gdb to
dissect the instruction.
As you go binary appears to be working, could you also include the output of `go env`
when replying.

@minux
Copy link
Member

minux commented May 14, 2013

Comment 8:

i'm confused. the cpu is executing from data section?
somehow runtime.pack passed garbage function pointer to runtime.mcall?

@minux
Copy link
Member

minux commented May 14, 2013

Comment 9:

Labels changed: added priority-soon, arch-arm, removed priority-triage.

Status changed to Accepted.

@gopherbot
Copy link
Contributor Author

Comment 11 by bga@bug-br.org.br:

Ok. Based on the crash, it seem it is the runtime package that crashed, right? So I did:
bga@blindio:~/go/src/pkg/runtime$ go version
go version go1.1 linux/arm
bga@blindio:~/go/src/pkg/runtime$ go test -c
bga@blindio:~/go/src/pkg/runtime$ ./runtime.test 
PASS
No crashes.
As requested:
bga@blindio:~/go/src/pkg/runtime$ go env
GOARCH="arm"
GOBIN=""
GOCHAR="5"
GOEXE=""
GOHOSTARCH="arm"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/bga/go-dev"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/linux_arm"
CC="gcc"
GOGCCFLAGS="-g -O2 -fPIC -marm -pthread"
CGO_ENABLED="1"
In case it is relevant, I was compiling go from my home dir and setting GOROOT_FINAL to
/usr/local/go so, to be able to compile the test and run it, I created a symlink
/usr/local/go pointing to /home/bga/go

@davecheney
Copy link
Contributor

Comment 12:

> In case it is relevant, I was compiling go from my home dir and setting GOROOT_FINAL
to /usr/local/go so, to be able to compile the test and run it, I created a symlink
/usr/local/go pointing to /home/bga/go
^ I wouldn't do _any_ of that. 
For simplicities sake, remove any reference to any go installation on your system,
including unsetting GOROOT and try again.

@gopherbot
Copy link
Contributor Author

Comment 13 by bga@bug-br.org.br:

Wait. I only created the symlink to do what you suggested (compiling and running the
tests) otherwise it would not find any packages.
other than that, i was simply following the instructions on how to compile go from
source without doing anything out of the ordinary.
But I will do as you suggest anyway. Will report back soon.

@davecheney
Copy link
Contributor

Comment 14:

I am sorry that I left you with the suggestion you should symlink things. Reviewing the
history of this ticket, I cannot see where I suggested that. 
I am concerned that you are not following the source installation procedure as outlined
on the website, and this may be compounding the problem you are experiencing.
Specifically if you have a mixture of go 1.0.x and 1.1 on your system, this might
explain why you are finding binaries with armv6 instructions in them

@gopherbot
Copy link
Contributor Author

Comment 15 by bga@bug-br.org.br:

You misunderstood me. What you suggested was that I compile the test manually and
execute it. I had GOROOT set to /usr/local/go from a previous installation but this
directory does not exist anymore as I wiped it clean before trying to compile Go.
Just to make it clear, there is no go installation on my system other than the directory
where I am compiling it.
In any case, I did as you suggested and removed all Go related environment variables:
bga@blindio:~/go/src$ set | grep GO
bga@blindio:~/go/src$
bga@blindio:~/go/src$ ../bin/go env
GOARCH="arm"
GOBIN=""
GOCHAR="5"
GOEXE=""
GOHOSTARCH="arm"
GOHOSTOS="linux"
GOOS="linux"
GOPATH=""
GORACE=""
GOROOT="/home/bga/go"
GOTOOLDIR="/home/bga/go/pkg/tool/linux_arm"
CC="gcc"
GOGCCFLAGS="-g -O2 -fPIC -marm -pthread"
CGO_ENABLED="1"
bga@blindio:~/go/src$
Then I tried to compile again.:
[...]
ok      html/template   0.513s
ok      image   0.676s
ok      image/color     0.104s
ok      image/draw      0.242s
ok      image/gif       0.065s
ok      image/jpeg      19.569s
ok      image/png       0.309s
ok      index/suffixarray       0.106s
SIGILL: illegal instruction
PC=0x4e5414
goroutine 1 [semacquire]:
sync.runtime_Semacquire(0x10e71da0)
        /home/bga/go/src/pkg/runtime/zsema_linux_arm.c:165 +0x30
sync.(*WaitGroup).Wait(0x10a1e360)
        /home/bga/go/src/pkg/sync/waitgroup.go:109 +0x118
main.(*builder).do(0x10841080, 0x10b59a10)
        /home/bga/go/src/cmd/go/build.go:715 +0x428
main.runTest(0x4df438, 0x10668010, 0x3, 0x3)
        /home/bga/go/src/cmd/go/test.go:330 +0x1ff8
main.main()
        /home/bga/go/src/cmd/go/main.go:156 +0x4ec
goroutine 2 [syscall]:
goroutine 4 [syscall]:
created by os/signal.init·1
        /home/bga/go/src/pkg/os/signal/signal_unix.go:27 +0x44
goroutine 6 [running]:
syscall.Syscall()
        /home/bga/go/src/pkg/syscall/asm_linux_arm.s:15 +0x8
created by main.(*builder).do
        /home/bga/go/src/cmd/go/build.go:712 +0x40c
goroutine 74 [finalizer wait]:
goroutine 14 [chan receive]:
main.func·026()
        /home/bga/go/src/cmd/go/signal.go:21 +0x34
created by main.processSignals
        /home/bga/go/src/cmd/go/signal.go:23 +0xc0
trap    0x6
error   0x0
oldmask 0x0
r0      0x1
r1      0x40dfcd80
r2      0xcb0f1f
r3      0x53e6049f
r4      0x12fe28d3
r5      0x4e5154
r6      0x3e
r7      0xf0
r8      0x2
r9      0x1066f400
r10     0x1065f360
fp      0x2b
ip      0x400b59f0
sp      0x40dfc95c
lr      0x7f0e0
pc      0x4e5414
cpsr    0x20000010
fault   0x0
Anything else I should try?

@davecheney
Copy link
Contributor

Comment 16:

I'm sorry I have nothing left to suggest. I am traveling at the moment, I will be able
to test on my QNAP 119P-II in a few days.

@minux
Copy link
Member

minux commented May 16, 2013

Comment 17:

i will need access to a similar system to determine the real cause.

@gopherbot
Copy link
Contributor Author

Comment 18 by bga@bug-br.org.br:

I can provide you with access to my NAS.

@davecheney
Copy link
Contributor

Comment 19:

I have been unable to reproduce the problem on my QNAP 119P-II (http://build.golang.org/
the linux-arm-arm5 builder)

@minux
Copy link
Member

minux commented May 16, 2013

Comment 20:

i will contact you off-list.
i will need gdb 7 and strace on your host.

@gopherbot
Copy link
Contributor Author

Comment 21 by bga@bug-br.org.br:

Hi minux. Did you figure anything out? Do you still need access to the NAS?

@minux
Copy link
Member

minux commented May 17, 2013

Comment 22:

i can't reproduce the SIGILL. but running "go test misc/cgo/test" could trigger this:
SIGSEGV: segmentation violation
PC=0x3ff971cc
goroutine 1 [running]:
syscall.Syscall()
        /home/minux/go/src/pkg/syscall/asm_linux_arm.s:15 +0x8
goroutine 2 [syscall]:
runtime.entersyscallblock()
        /home/minux/go/src/pkg/runtime/proc.c:1333 +0xc0
runtime.MHeap_Scavenger()
        /home/minux/go/src/pkg/runtime/mheap.c:453 +0xfc
runtime.goexit()
        /home/minux/go/src/pkg/runtime/proc.c:1223
created by runtime.main
        /home/minux/go/src/pkg/runtime/proc.c:165
goroutine 33 [finalizer wait]:
runtime.park(0x0, 0x0, 0x27e059)
        /home/minux/go/src/pkg/runtime/proc.c:1175 +0x40
runfinq()
        /home/minux/go/src/pkg/runtime/mgc0.c:2182 +0x68
runtime.goexit()
        /home/minux/go/src/pkg/runtime/proc.c:1223
created by runtime.gc
        /home/minux/go/src/pkg/runtime/mgc0.c:1886
trap    0xe
error   0x80000005
oldmask 0x0
r0      0x1
r1      0x27f420
r2      0x10352b88
r3      0xaacb5759
r4      0x12fef105
r5      0x27f424
r6      0x1
r7      0xf0
r8      0x2
r9      0x10345000
r10     0x103361b0
fp      0x6
ip      0x1037bffd
sp      0x10352b90
lr      0x68e34
pc      0x3ff971cc
cpsr    0x80000010
fault   0x3ff971cc
note: it's cmd/go that panics, not the test.
the pc is incorrect, but i'm still not sure what smashes pc.
@dave, do you want to take a look? could you please run "go test ../misc/cgo/test" in a
loop and see if
you can reproduce the crash?

@davecheney
Copy link
Contributor

Comment 23:

@minux will do.
As a side note, my arm 5 builder, which is the same hardware, has been running builds
successfully (see the dashboard).
@OP, as an experiment, grab the binary distribution from here,
http://dave.cheney.net/unofficial-arm-tarballs, and see if you can run the tests. go
test std should do the job.

@gopherbot
Copy link
Contributor Author

Comment 24 by bga@bug-br.org.br:

I downloaded the pre-compiled package and installed it. Ran the tests and it did not
crash, but I got 3 timeouts:
*** Test killed: ran too long (10m0s).
FAIL    archive/zip 600.019s
*** Test killed: ran too long (10m0s).
FAIL    crypto/dsa  600.005s
*** Test killed: ran too long (10m0s).
FAIL    math/big    600.006s
Considering the crashes I was seeing seemed to be related to semaphores, could these
timeouts be a deadlock that may have the same root cause of the crashes?

@davecheney
Copy link
Contributor

Comment 25:

Oh, this was my mistake, ./all.bash runs go test -short std, I forgot to tell you to run
the tests in short mode. Those failures are expected when you don't pass -short due to
the slow fpu emulation.

@gopherbot
Copy link
Contributor Author

Comment 26 by bga@bug-br.org.br:

Ran it with -short and it worked except fr this:
2013/05/21 09:14:50 cgi: copy error: write tcp 127.0.0.1:45865: connection reset by peer
--- FAIL: TestCopyError (15.12 seconds)
    host_test.go:351: post-conn.Close, expected child to be gone
FAIL
FAIL    net/http/cgi    16.883s
Any idea why compiling Go on my system could result on the crashes I was seeing?

@minux
Copy link
Member

minux commented May 21, 2013

Comment 27:

perhaps a buggy gcc?
in fact, on netbsd/arm i do encounter a case where gcc -O2 generates bad code for cmd/gc
and in turn causes test failures.
i suggest you replace the toolchain in pkg/tools/linux_arm with those from the binary
distribution
and rerun the test to see if this makes a difference.
i originally suspect bad interaction between kernel and sync/atomic, but if "go test
-short std"
could pass with dave's prebuilt package then i don't think it is the case.

@gopherbot
Copy link
Contributor Author

Comment 28 by bga@bug-br.org.br:

I ran the tests again )no changes whatsoever, just executed a second run with exactly
the same setup as teh previous one. I got a crash again now:
ok      net/http/cgi    1.858s
ok      net/http/cookiejar  0.137s
ok      net/http/fcgi   0.111s
ok      net/http/httptest   0.128s
SIGSEGV: segmentation violation
PC=0x4e5154
goroutine 1 [semacquire]:
sync.runtime_Semacquire(0x10d43730)
    /usr/local/go/src/pkg/runtime/zsema_linux_arm.c:165 +0x30
sync.(*WaitGroup).Wait(0x10ddfa40)
    /usr/local/go/src/pkg/sync/waitgroup.go:109 +0x118
main.(*builder).do(0x10837300, 0x10b42150)
    /usr/local/go/src/cmd/go/build.go:715 +0x428
main.runTest(0x4df438, 0x10668010, 0x2, 0x2)
    /usr/local/go/src/cmd/go/test.go:330 +0x1ff8
main.main()
    /usr/local/go/src/cmd/go/main.go:156 +0x4ec
goroutine 2 [syscall]:
goroutine 4 [syscall]:
created by os/signal.init·1
    /usr/local/go/src/pkg/os/signal/signal_unix.go:27 +0x44
goroutine 6 [running]:
syscall.Syscall()
    /usr/local/go/src/pkg/syscall/asm_linux_arm.s:15 +0x8
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:712 +0x40c
goroutine 14 [chan receive]:
main.func·026()
    /usr/local/go/src/cmd/go/signal.go:21 +0x34
created by main.processSignals
    /usr/local/go/src/cmd/go/signal.go:23 +0xc0
goroutine 259 [finalizer wait]:
trap    0xe
error   0x5
oldmask 0x0
r0      0x1
r1      0x40dfcd80
r2      0x40dfc954
r3      0xb7dfac8f
r4      0x13002aa2
r5      0x4e5154
r6      0x1
r7      0xf0
r8      0x2
r9      0x1066f400
r10     0x1065f360
fp      0x2b
ip      0x400b59f0
sp      0x40dfc95c
lr      0x7f0e0
pc      0x4e5154
cpsr    0x80000010
fault   0xb84e515c
Looks like exactly the same crash, so it is still there even with the precompiled
packages.

@davecheney
Copy link
Contributor

Comment 29:

I am at a loss for what to suggest. We have two ARMv5 builders which are working fine,
and I cannot replicate the problem. 
Which OS distribution and kernel are you running, possibly I can construct an
environment in a chroot.

@gopherbot
Copy link
Contributor Author

Comment 30 by bga@bug-br.org.br:

It is Debian Squeeze running on a QNAP 219P.
The kernel is: 2.6.32-5-kirkwood
Note the change is a straight Debian Squeeze for the QNAP device in question. There are
no customizations (other than adding/removing some packages) including no
custom-compiled kernel.

@davecheney
Copy link
Contributor

Comment 31:

The only difference from you setup to mine is i've upgraded to debian/wheezy and am
running 
dfc@qnap:~$ uname -a
Linux qnap 3.2.0-4-kirkwood #1 Debian 3.2.41-2+deb7u2 armv5tel GNU/Linux

@minux
Copy link
Member

minux commented May 28, 2013

Comment 32:

hi @dga, could you please try the newer kernel debian provides?

@gopherbot
Copy link
Contributor Author

Comment 33 by bga@bug-br.org.br:

I will go ahead and upgrade to Wheezy. Will let you know how it turns out.

@gopherbot
Copy link
Contributor Author

Comment 34 by bga@bug-br.org.br:

Ok. After upgrading I ran the tests 3 times. No crashes whatsoever. I guess this was it.

@minux
Copy link
Member

minux commented May 28, 2013

Comment 35:

@bga, glad that you found the solution.
i will update the https://code.google.com/p/go-wiki/wiki/GoArm page about
the potentially buggy kernel (2.6.32-5-kirkwood) on QNAP 219P.
btw, please remove my account on your machine if you haven't done so,
and thank you for providing the help.

Status changed to Fixed.

This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants