Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: signal: illegal instruction on ios/arm64 #35851

Open
eliasnaur opened this issue Nov 26, 2019 · 12 comments
Open

runtime: signal: illegal instruction on ios/arm64 #35851

eliasnaur opened this issue Nov 26, 2019 · 12 comments

Comments

@eliasnaur
Copy link
Contributor

@eliasnaur eliasnaur commented Nov 26, 2019

https://build.golang.org/log/8da8b3d360f7b226bbc021f29c3d9617d35773f9

ok  	cmd/addr2line	0.028s
ok  	cmd/api	0.071s
ok  	cmd/asm/internal/asm	0.628s
ok  	cmd/asm/internal/lex	0.031s
/tmp/workdir-host-darwin-arm64-corellium-ios/go/pkg/tool/darwin_arm64/vet: signal: illegal instruction
ok  	cmd/compile	0.048s
ok  	cmd/compile/internal/gc	0.103s
FAIL
go tool dist: Failed: exit status 1
@dmitshur
Copy link
Member

@dmitshur dmitshur commented Nov 26, 2019

Strangely, it failed at an earlier step on the next commit (https://build.golang.org/log/186ffe8d4a32f24c719cb49eddf6e37e73957239), but has been passing since then. The commit that fixed it doesn't seem related.

@dmitshur dmitshur added this to the Backlog milestone Nov 26, 2019
@cherrymui
Copy link
Contributor

@cherrymui cherrymui commented Nov 26, 2019

@dmitshur that is #35800, which should be fixed with CL 208818.

@dmitshur
Copy link
Member

@dmitshur dmitshur commented Nov 26, 2019

@cherrymui Thanks. The darwin-arm64-corellium builder was passing for many commits between 01f15b6 and 67f0f83 (CL 208818). Do you know if that is it because this issue was sporadic? Do you think anything more needs to be done for this issue?

@cherrymui
Copy link
Contributor

@cherrymui cherrymui commented Nov 26, 2019

Yeah, #35800 is sporadic, happens when a signal lands in some unlucky time. This also looks like a flake. I don't know what the cause is.

@bcmills
Copy link
Member

@bcmills bcmills commented Feb 20, 2020

2020-02-19T21:34:59-1e43298/darwin-arm64-corellium

go build cmd/compile/internal/ssa: /tmp/workdir-host-darwin-arm64-corellium-ios/go/pkg/tool/darwin_arm64/compile: signal: illegal instruction
FAIL	cmd/compile/internal/ssa [build failed]
@markmentovai
Copy link

@markmentovai markmentovai commented Dec 2, 2020

The workaround in #42774 will probably address this, but it relies on sigaltstack, which isn’t working on ios-arm64 until iOS 14.

You ought to be able to use sigaltstack and SA_ONSTACK on iOS when the architecture is x86_64 (meaning the iOS simulator) or arm64 starting with iOS 14. #42774 (comment). If you do that, then the mlock workaround should keep this bug at bay on newer OS versions.

Apple FB8922558 aims to get this fixed in the kernel, because it is a kernel bug.

@eliasnaur
Copy link
Contributor Author

@eliasnaur eliasnaur commented Dec 2, 2020

I've upgraded all builders to iOS 14. I'm not sure what versions are covered by your "on newer OS versions", but I think Go should run on iOS < 14, at least until iOS 15 is out.

@eliasnaur
Copy link
Contributor Author

@eliasnaur eliasnaur commented Dec 2, 2020

Continuing the discussion from #42774 (comment) here.

If there is someway we can reproduce the failure more easily (like running the io benchmarks in this issue), that would be helpful. Then we can play with the workaround and other ideas and see if it is effective.

It may also be helpful if you could get a crash report.

@cherrymui I tried running ./all.bash while running the io benchmark concurrently. all.bash succeeded but the benchmark failed with an memory range error:

 ../bin/go test io -run='^$' -short -bench=BenchmarkCopy -count=1000
goos: ios
goarch: arm64
pkg: io
BenchmarkCopyNSmall-2   	 1000000	      1279 ns/op
BenchmarkCopyNSmall-2   	runtime: memory allocated by OS [0x280000000, 0x2a4000000) not in usable address space: base outside usable address space
fatal error: memory reservation exceeds address space limit

runtime stack:
runtime.throw(0x104135539, 0x2e)
	/var/root/goroot2/src/runtime/panic.go:1112 +0x54
runtime.(*mheap).sysAlloc(0x10417be20, 0x20000000, 0x16bddb368, 0x10404b888)
	/var/root/goroot2/src/runtime/malloc.go:720 +0x630
runtime.(*mheap).grow(0x10417be20, 0x10000, 0x0)
	/var/root/goroot2/src/runtime/mheap.go:1346 +0x74
runtime.(*mheap).allocSpan(0x10417be20, 0x10000, 0x100, 0x105d45690)
	/var/root/goroot2/src/runtime/mheap.go:1173 +0x648
runtime.(*mheap).alloc.func1()
	/var/root/goroot2/src/runtime/mheap.go:910 +0x4c
runtime.systemstack(0x16bddb4a8)
	/var/root/goroot2/src/runtime/asm_arm64.s:248 +0xa0
runtime.mstart()
	/var/root/goroot2/src/runtime/proc.go:1183

goroutine 1021 [running]:
runtime.systemstack_switch()
	/var/root/goroot2/src/runtime/asm_arm64.s:193 +0x8 fp=0x130335b20 sp=0x130335b10 pc=0x10408b008
runtime.(*mheap).alloc(0x10417be20, 0x10000, 0x105d40101, 0x0)
	/var/root/goroot2/src/runtime/mheap.go:904 +0x64 fp=0x130335b70 sp=0x130335b20 pc=0x10404ae64
runtime.(*mcache).allocLarge(0x1044485b8, 0x1ffffe00, 0x104190101, 0x104196b68)
	/var/root/goroot2/src/runtime/mcache.go:224 +0x90 fp=0x130335be0 sp=0x130335b70 pc=0x10403c0a0
runtime.mallocgc(0x1ffffe00, 0x1041ada20, 0x104448501, 0x135)
	/var/root/goroot2/src/runtime/malloc.go:1078 +0x82c fp=0x130335c90 sp=0x130335be0 pc=0x10403392c
runtime.makeslice(0x1041ada20, 0x1ffffe00, 0x1ffffe00, 0x1)
	/var/root/goroot2/src/runtime/slice.go:98 +0x74 fp=0x130335cc0 sp=0x130335c90 pc=0x104071204
bytes.makeSlice(0x1ffffe00, 0x0, 0x0, 0x0)
	/var/root/goroot2/src/bytes/buffer.go:229 +0x5c fp=0x130335d00 sp=0x130335cc0 pc=0x1040daaec
bytes.(*Buffer).grow(0x130092050, 0x200, 0x0)
	/var/root/goroot2/src/bytes/buffer.go:142 +0x130 fp=0x130335d50 sp=0x130335d00 pc=0x1040da570
bytes.(*Buffer).Write(0x130092050, 0x118676000, 0x200, 0x200, 0x0, 0x0, 0x0)
	/var/root/goroot2/src/bytes/buffer.go:172 +0xc4 fp=0x130335d80 sp=0x130335d50 pc=0x1040da7a4
io.copyBuffer(0x1041dcc78, 0x130092050, 0x1041dcb58, 0x118655860, 0x118676000, 0x200, 0x200, 0x200, 0x0, 0x0)
	/var/root/goroot2/src/io/io.go:425 +0x1a8 fp=0x130335e00 sp=0x130335d80 pc=0x1040a43c8
io.Copy(...)
	/var/root/goroot2/src/io/io.go:382
io.CopyN(0x1041dcc78, 0x130092050, 0x1041dca98, 0x1302ec060, 0x200, 0x200, 0x0, 0x0)
	/var/root/goroot2/src/io/io.go:358 +0x84 fp=0x130335e60 sp=0x130335e00 pc=0x1040a40e4
io_test.BenchmarkCopyNSmall(0x13031e480)
	/var/root/goroot2/src/io/io_test.go:181 +0x198 fp=0x130335ee0 sp=0x130335e60 pc=0x10411f368
testing.(*B).runN(0x13031e480, 0x928bf)
	/var/root/goroot2/src/testing/benchmark.go:192 +0xfc fp=0x130335f50 sp=0x130335ee0 pc=0x1040dce4c
testing.(*B).launch(0x13031e480)
	/var/root/goroot2/src/testing/benchmark.go:325 +0xcc fp=0x130335fd0 sp=0x130335f50 pc=0x1040dd4cc
runtime.goexit()
	/var/root/goroot2/src/runtime/asm_arm64.s:1130 +0x4 fp=0x130335fd0 sp=0x130335fd0 pc=0x10408d5c4
created by testing.(*B).doBench
	/var/root/goroot2/src/testing/benchmark.go:280 +0x48

goroutine 1 [chan receive]:
testing.(*B).doBench(0x13031e480, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/var/root/goroot2/src/testing/benchmark.go:281 +0x5c
testing.(*benchContext).processBench(0x130016030, 0x13031e480)
	/var/root/goroot2/src/testing/benchmark.go:580 +0x194
testing.(*B).run(0x13031e240)
	/var/root/goroot2/src/testing/benchmark.go:272 +0x58
testing.(*B).Run(0x13031e000, 0x10412f254, 0x13, 0x1041d9830, 0xbfea243223ce0000)
	/var/root/goroot2/src/testing/benchmark.go:668 +0x330
testing.runBenchmarks.func1(0x13031e000)
	/var/root/goroot2/src/testing/benchmark.go:541 +0x74
testing.(*B).runN(0x13031e000, 0x1)
	/var/root/goroot2/src/testing/benchmark.go:192 +0xfc
testing.runBenchmarks(0x10412c11c, 0x2, 0x130016018, 0x10416d700, 0x2, 0x2, 0x104175720)
	/var/root/goroot2/src/testing/benchmark.go:550 +0x34c
testing.(*M).Run(0x130152000, 0x0)
	/var/root/goroot2/src/testing/testing.go:1425 +0x434
main.main()
	_testmain.go:171 +0x14c
exit status 2
FAIL	io	2.086s
FAIL

Another possibility could be #41702. The workaround (CL https://go-review.googlesource.com/c/go/+/262817 and https://go-review.googlesource.com/c/go/+/262438) only covers GOOS="darwin". Maybe we should extend them ios as well.

This issue being the iOS version of #41702 sounds plausible to me, since both crashes in this issue are from an exec (vet and compile).

@gopherbot
Copy link

@gopherbot gopherbot commented Dec 3, 2020

Change https://golang.org/cl/275293 mentions this issue: runtime: avoid receiving preemotion signal while exec'ing

gopherbot pushed a commit that referenced this issue Dec 4, 2020
The iOS kernel has the same problem as the macOS kernel. Extend
the workaround of #41702 (CL 262438 and CL 262817) to iOS.

Updates #35851.

Change-Id: I7ccec00dc96643c08c5be8b385394856d0fa0f64
Reviewed-on: https://go-review.googlesource.com/c/go/+/275293
Trust: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
@bcmills bcmills changed the title runtime: signal: illegal instruction on darwin/arm64 runtime: signal: illegal instruction on ios-arm64 Dec 11, 2020
@bcmills bcmills changed the title runtime: signal: illegal instruction on ios-arm64 runtime: signal: illegal instruction on ios/arm64 Dec 11, 2020
@cherrymui
Copy link
Contributor

@cherrymui cherrymui commented Dec 11, 2020

Yeah, apparently that was not. I'm planning to do more experiments.

@gopherbot
Copy link

@gopherbot gopherbot commented Dec 21, 2020

Change https://golang.org/cl/279489 mentions this issue: runtime: use sigaltstack+mlock on iOS

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants