Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: ARMv5: Illegal instruction #20127

Closed
jingqiuELE opened this issue Apr 26, 2017 · 9 comments

Comments

Projects
None yet
6 participants
@jingqiuELE
Copy link

commented Apr 26, 2017

What version of Go are you using (go version)?

#go version
go version go1.8.1 linux/arm

What operating system and processor architecture are you using (go env)?

# go env
GOARCH="arm"
GOBIN=""
GOEXE=""
GOHOSTARCH="arm"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/linux_arm"
GCCGO="gccgo"
GOARM="7"
CC="gcc"
GOGCCFLAGS="-fPIC -marm -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build526789265=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"

What did you do?

#cat hello.go
package main

import "fmt"

func main() {
        fmt.Println("Hello, world!")
}
# GOARM=5 go build
# readelf -h hello
ELF Header:
  Magic:   7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00 
  Class:                             ELF32
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              EXEC (Executable file)
  Machine:                           ARM
  Version:                           0x1
  Entry point address:               0x6132c
  Start of program headers:          52 (bytes into file)
  Start of section headers:          276 (bytes into file)
  Flags:                             0x5000002, has entry point, Version5 EABI
  Size of this header:               52 (bytes)
  Size of program headers:           32 (bytes)
  Number of program headers:         7
  Size of section headers:           40 (bytes)
  Number of section headers:         23
  Section header string table index: 3

Then upload the "hello" executable to an ARM board, which doesn't have "vfp" or "fp" features:

# cat /proc/cpuinfo
processor       : 0
model name      : ARMv7 Processor rev 1 (v7l)
BogoMIPS        : 398.13
Features        : half thumb fastmult edsp tls 
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x4
CPU part        : 0xc09
CPU revision    : 1

processor       : 1
model name      : ARMv7 Processor rev 1 (v7l)
BogoMIPS        : 398.13
Features        : half thumb fastmult edsp tls 
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x4
CPU part        : 0xc09
CPU revision    : 1

Run the program:

# ./hello.armv5 
potentially unexpected fatal signal 4.
CPU: 1 PID: 1578 Comm: hello.armv5 Tainted: P                4.1.38 #2
Hardware name: BCM963138
task: df50da20 ti: d8ba6000 task.ti: d8ba6000
PC is at 0x43ebc
LR is at 0x5e8bc
pc : [<00043ebc>]    lr : [<0005e8bc>]    psr: 20000010
sp : bed6ad84  ip : cafebabe  fp : 000b5060
r10: 00103020  r9 : 00000000  r8 : 00103218
r7 : 00000014  r6 : 00000000  r5 : 00000000  r4 : 00000000
r3 : 00000000  r2 : 00000000  r1 : bed5b1a0  r0 : 00000000
Flags: nzCv  IRQs on  FIQs on  Mode USER_32  ISA ARM  Segment user
Control: 10c5387d  Table: 177d004a  DAC: 00000015
CPU: 1 PID: 1578 Comm: hello.armv5 Tainted: P                4.1.38 #2
Hardware name: BCM963138
[<c00118f4>] (unwind_backtrace) from [<c000d63c>] (show_stack+0x10/0x14)
[<c000d63c>] (show_stack) from [<c0448584>] (dump_stack+0x8c/0xa0)
[<c0448584>] (dump_stack) from [<c00276fc>] (get_signal+0x4b4/0x5ac)
[<c00276fc>] (get_signal) from [<c000cdf4>] (do_signal+0xc8/0x3ac)
[<c000cdf4>] (do_signal) from [<c000d27c>] (do_work_pending+0x94/0xa4)
[<c000d27c>] (do_work_pending) from [<c000a46c>] (work_pending+0xc/0x20)
Illegal instruction

What did you expect to see?

The program prints "hello, world!" and exit without error.

What did you see instead?

Illegal instruction signal received.

@davecheney

This comment has been minimized.

Copy link
Contributor

commented Apr 26, 2017

@bradfitz bradfitz changed the title ARMv5: Illegal instruction cmd/compile: ARMv5: Illegal instruction Apr 26, 2017

@bradfitz bradfitz added this to the Go1.9 milestone Apr 26, 2017

@bradfitz

This comment has been minimized.

Copy link
Member

commented Apr 26, 2017

@jingqiuELE

This comment has been minimized.

Copy link
Author

commented Apr 26, 2017

Using the helloworld program, I couldn't reproduce the "Illegal Instruction" error using gdb remote debugging.
There is a slight chance that it exits without error.
This leads me to think that the offending code is not in the main function.
From the backtrace above: PC is at 0x43ebc
Disassembly hello.armv5 looking for above code address, I found:

00043ea0 <runtime.check>:
   43ea0:       e59a1008        ldr     r1, [sl, #8]
   43ea4:       e15d0001        cmp     sp, r1
   43ea8:       9a00013f        bls     443ac <runtime.check+0x50c>
   43eac:       e52de044        str     lr, [sp, #-68]! ; 0xffffffbc
   43eb0:       e3a00000        mov     r0, #0
   43eb4:       e58d0028        str     r0, [sp, #40]   ; 0x28
   43eb8:       e59fb4fc        ldr     fp, [pc, #1276] ; 443bc <runtime.check+0x51c>
   **43ebc:       ed9b0a00        vldr    s0, [fp]**
   43ec0:       ed8d0a09        vstr    s0, [sp, #36]   ; 0x24
   43ec4:       ed8d0a08        vstr    s0, [sp, #32]

If I build the same program on x86 for armv5:
GOARCH=arm GOARM=5 GOOS=linux go build
the section looks like below:

0004404c <runtime.check>:
   4404c:       e59a1008        ldr     r1, [sl, #8]
   44050:       e15d0001        cmp     sp, r1
   44054:       9a000123        bls     444e8 <runtime.check+0x49c>
   44058:       e52de044        str     lr, [sp, #-68]! ; 0xffffffbc
   4405c:       e3a00000        mov     r0, #0
   44060:       e58d0028        str     r0, [sp, #40]   ; 0x28
   44064:       eb00769b        bl      61ad8 <_sfloat>
   44068:       e59fb488        ldr     fp, [pc, #1160] ; 444f8 <runtime.check+0x4ac>
   **4406c:       ed9b0a00        vldr    s0, [fp]**
   44070:       ed8d0a09        vstr    s0, [sp, #36]   ; 0x24
   44074:       ed8d0a08        vstr    s0, [sp, #32]

The one built on x86 has _sfloat macro inserted before the vldr instruction in runtime_check, whereas the one built on armhf does not.
I suspect that the linker doesn't respect the GOARM environment variable when doing cross compiling in armhf.

@dcheney-atlassian

This comment has been minimized.

Copy link

commented Apr 26, 2017

I suspect that the linker doesn't respect the GOARM environment variable when doing cross compiling in armhf.

Values of GOARM greater than 5 are hard coded to assume a floating point unit is availble. If your hardware does not have a VFP unit, you must always compiled with GOARM=5

@jingqiuELE

This comment has been minimized.

Copy link
Author

commented Apr 26, 2017

I'm sorry if I have not describe the issue concise enough. GOARM=5 was specified in all of the above scenarios. Specifically the build command we used is always:
GOARCH=arm GOARM=5 GOOS=linux go build

I did a test with a slightly modified version of helloworld program, with a simple float point operation in the main program and disassemble the binary built on armv7 platform with GOARM=5. The instructions under main func clearly has the _sfloat macro before the float point operations, however, in the same file, the <runtime.check> function does not have the _sfloat macro inserted before the float point instructions:

package main

import "fmt"

func main() {
        i := 0.1
        fmt.Printf("Hello, world! i=%v\n", i*0.2)
}

Assembly dump of generated binary:

0008893c <main.main>:
   8893c:       e59a1008        ldr     r1, [sl, #8]
   88940:       e15d0001        cmp     sp, r1
   88944:       9a00001b        bls     889b8 <main.main+0x7c>
   88948:       e52de034        str     lr, [sp, #-52]! ; 0xffffffcc
   8894c:       ebff63ec        bl      61904 <_sfloat>
   88950:       e59fb070        ldr     fp, [pc, #112]  ; 889c8 <main.main+0x8c>
   88954:       ed9b0b00        vldr    d0, [fp]

00043ea0 <runtime.check>:
   43ea0:       e59a1008        ldr     r1, [sl, #8]
   43ea4:       e15d0001        cmp     sp, r1
   43ea8:       9a00013f        bls     443ac <runtime.check+0x50c>
   43eac:       e52de044        str     lr, [sp, #-68]! ; 0xffffffbc
   43eb0:       e3a00000        mov     r0, #0
   43eb4:       e58d0028        str     r0, [sp, #40]   ; 0x28
   43eb8:       e59fb4fc        ldr     fp, [pc, #1276] ; 443bc <runtime.check+0x51c>
   43ebc:       ed9b0a00        vldr    s0, [fp]
   43ec0:       ed8d0a09        vstr    s0, [sp, #36]   ; 0x24

@cherrymui

This comment has been minimized.

Copy link
Contributor

commented Apr 26, 2017

@jingqiuELE

This comment has been minimized.

Copy link
Author

commented Apr 26, 2017

Thank you, using the
GOARCH=arm GOARM=5 GOOS=linux go build -a
does produce the correct runtime and the program can run on ARMv5 without SIGILL.
Does it mean that I should clean build whenever the value of GOARM is changed?

@bradfitz

This comment has been minimized.

Copy link
Member

commented Apr 26, 2017

Yes. That is #9737

I'm going to close this as a dup of that.

@bradfitz bradfitz closed this Apr 26, 2017

@jingqiuELE

This comment has been minimized.

Copy link
Author

commented Apr 27, 2017

Thank you. Using
GOARCH=arm GOARM=5 GOOS=linux go build -i --installsuffix armv5
would cache armv5 runtime, thus speed up the build.

@golang golang locked and limited conversation to collaborators Apr 27, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.