x/sys/cpu: SIGILL on arm64 in readARM64Registers

[bradfitz]

On a Synology DS220J (Realtek RTD1296), an arm64 CPU, with x/sys@v0.1.0:

SIGILL: illegal instruction
PC=0x4e0d70 m=0 sigcode=1
instruction bytes: 0x0 0x6 0x38 0xd5 0xe0 0x7 0x0 0xf9 0xc0 0x3 0x5f 0xd6 0x0 0x0 0x0 0x0

goroutine 1 [running, locked to thread]:
golang.org/x/sys/cpu.getisar0()
	golang.org/x/sys@v0.1.0/cpu/cpu_arm64.s:14 fp=0x4000093510 sp=0x4000093510 pc=0x4e0d70
golang.org/x/sys/cpu.readARM64Registers()
	golang.org/x/sys@v0.1.0/cpu/cpu_arm64.go:65 +0x2c fp=0x4000093550 sp=0x4000093510 pc=0x4e05fc
golang.org/x/sys/cpu.doinit()
	golang.org/x/sys@v0.1.0/cpu/cpu_linux_arm64.go:38 +0x24 fp=0x4000093560 sp=0x4000093550 pc=0x4e08e4
golang.org/x/sys/cpu.archInit(...)
	golang.org/x/sys@v0.1.0/cpu/cpu_arm64.go:48
golang.org/x/sys/cpu.init.0()
	golang.org/x/sys@v0.1.0/cpu/cpu.go:199 +0x20 fp=0x4000093570 sp=0x4000093560 pc=0x4dfc80
runtime.doInit(0xfe7740)
	runtime/proc.go:6321 +0x128 fp=0x40000936b0 sp=0x4000093570 pc=0x56c38
runtime.doInit(0xfe9980)
	runtime/proc.go:6298 +0x68 fp=0x40000937f0 sp=0x40000936b0 pc=0x56b78
runtime.doInit(0xfee380)
	runtime/proc.go:6298 +0x68 fp=0x4000093930 sp=0x40000937f0 pc=0x56b78
runtime.doInit(0xff4da0)
	runtime/proc.go:6298 +0x68 fp=0x4000093a70 sp=0x4000093930 pc=0x56b78
runtime.doInit(0xff50a0)
	runtime/proc.go:6298 +0x68 fp=0x4000093bb0 sp=0x4000093a70 pc=0x56b78
runtime.doInit(0xffa960)
	runtime/proc.go:6298 +0x68 fp=0x4000093cf0 sp=0x4000093bb0 pc=0x56b78
runtime.doInit(0xff58a0)
	runtime/proc.go:6298 +0x68 fp=0x4000093e30 sp=0x4000093cf0 pc=0x56b78
runtime.doInit(0xff7080)
	runtime/proc.go:6298 +0x68 fp=0x4000093f70 sp=0x4000093e30 pc=0x56b78
runtime.main()
	runtime/proc.go:233 +0x1f8 fp=0x4000093fd0 sp=0x4000093f70 pc=0x48bb8
runtime.goexit()
	runtime/asm_arm64.s:1172 +0x4 fp=0x4000093fd0 sp=0x4000093fd0 pc=0x7a824

(originally reported in tailscale/tailscale#5793)

Although that's for x/sys v0.1, there have been no cpu changes since:

$ git diff -r v0.1.0..v0.3.0 cpu
$

@tklauser, looks like you touched that code last in 85b82a3

/cc @golang/runtime

[bradfitz]

Apparently that Realtek RTD1296 CPU is:

• ARMv8-A (64-bit)
• 4x ARM Cortex-A53 @ 1.4 GHz

[bcmills]

(attn @golang/arm)

[cherrymui]

The instruction is mrs x0, ID_AA64ISAR0_EL1. Maybe that system register isn't accessible in user space on your machine? I assume the OS is Linux?

[prattmic]

I think EL1 here is meant to refer to kernel mode. My copy of the ARM manual, section "D13.2.61 ID_AA64ISAR0_EL1, AArch64 Instruction Set Attribute Register 0" even says that access from EL0 (user mode) either traps or is undefined behavior. But clearly it must work on most systems.

[bradfitz]

Looks like it's:

Linux fanli-218play 4.4.180+ #42661 SMP Mon Jun 27 15:09:04 CST 2022 aarch64 GNU/Linux synology_rtd1296_ds218play

On a https://www.synology.com/en-us/products/DS218play

[bradfitz]

FWIW, on Synology, this program runs as non-root but with some additional capabilities (as in capabilities(7)).

[bradfitz]

Looks like this is the same CPU across:

https://www.synology.com/en-us/products/DS220j#specs
https://www.synology.com/en-us/products/DS218#specs
https://www.synology.com/en-us/products/DS418#specs
https://www.synology.com/en-us/products/DS218play#specs
https://www.synology.com/en-us/products/DS118#specs

... which are the 5 of the top 6 arm64 Synology models per our stats. (the DS120j has a Marvell Armada 3700 88F3720)

[prattmic]

It looks like these instructions do trap. Linux emulates the instructions and returns to userspace: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=77c97b4ee21290f5f083173d957843b615abbff2

It looks like emulation support was added in Linux 4.11, so it makes sense that 4.4 is crashing. From the patch, it seems that we should check for HWCAP_CPUID before executing these instructions.

[bradfitz]

Out of curiosity, I checked: no Synology model (for any architecture) runs a kernel newer than 4.4.x. Top kernel versions on Synology we see used:

kernel	percent
4.4.180+	75
3.10.108	13
4.4.59+	10
3.10.105	8
3.2.101	3
3.2.40	3
3.10.102	2
2.6.32.12	2

But who knows how much was cherry-picked into those +. Apparently not that commit you referenced :)

[bradfitz]

So in this case I guess the /proc/self/auxv read is failing, causing the readARM64Registers call, which then gets the SIGILL:

func doinit() {
        if err := readHWCAP(); err != nil {
                // failed to read /proc/self/auxv, try reading registers directly                                                                               
                readARM64Registers()
                return
        }
...

But these same hwcap flags are available in /proc/cpuinfo too, no?

pi@pi400arm64:~ $ uname -r
5.10.103-v8+
pi@pi400arm64:~ $ cat /proc/cpuinfo  | grep ^Features
Features	: fp asimd evtstrm crc32 cpuid
Features	: fp asimd evtstrm crc32 cpuid
Features	: fp asimd evtstrm crc32 cpuid
Features	: fp asimd evtstrm crc32 cpuid

Should we try parsing that first as another fallback before the mrs x0, ID_AA64ISAR0_EL1?

Or would y'all prefer looking at the kernel version and hope that's indicative of reality?

[bradfitz]

Related issue with x/sys/cpu on arm64 but on OpenBSD is #42517