Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: signal: illegal instruction on ios/arm64 #35851

Open
eliasnaur opened this issue Nov 26, 2019 · 28 comments
Open

runtime: signal: illegal instruction on ios/arm64 #35851

eliasnaur opened this issue Nov 26, 2019 · 28 comments

Comments

@eliasnaur
Copy link
Contributor

@eliasnaur eliasnaur commented Nov 26, 2019

https://build.golang.org/log/8da8b3d360f7b226bbc021f29c3d9617d35773f9

ok  	cmd/addr2line	0.028s
ok  	cmd/api	0.071s
ok  	cmd/asm/internal/asm	0.628s
ok  	cmd/asm/internal/lex	0.031s
/tmp/workdir-host-darwin-arm64-corellium-ios/go/pkg/tool/darwin_arm64/vet: signal: illegal instruction
ok  	cmd/compile	0.048s
ok  	cmd/compile/internal/gc	0.103s
FAIL
go tool dist: Failed: exit status 1
@dmitshur
Copy link
Contributor

@dmitshur dmitshur commented Nov 26, 2019

Strangely, it failed at an earlier step on the next commit (https://build.golang.org/log/186ffe8d4a32f24c719cb49eddf6e37e73957239), but has been passing since then. The commit that fixed it doesn't seem related.

@dmitshur dmitshur added this to the Backlog milestone Nov 26, 2019
@cherrymui
Copy link
Contributor

@cherrymui cherrymui commented Nov 26, 2019

@dmitshur that is #35800, which should be fixed with CL 208818.

@dmitshur
Copy link
Contributor

@dmitshur dmitshur commented Nov 26, 2019

@cherrymui Thanks. The darwin-arm64-corellium builder was passing for many commits between 01f15b6 and 67f0f83 (CL 208818). Do you know if that is it because this issue was sporadic? Do you think anything more needs to be done for this issue?

@cherrymui
Copy link
Contributor

@cherrymui cherrymui commented Nov 26, 2019

Yeah, #35800 is sporadic, happens when a signal lands in some unlucky time. This also looks like a flake. I don't know what the cause is.

@bcmills
Copy link
Member

@bcmills bcmills commented Feb 20, 2020

2020-02-19T21:34:59-1e43298/darwin-arm64-corellium

go build cmd/compile/internal/ssa: /tmp/workdir-host-darwin-arm64-corellium-ios/go/pkg/tool/darwin_arm64/compile: signal: illegal instruction
FAIL	cmd/compile/internal/ssa [build failed]

@markmentovai
Copy link

@markmentovai markmentovai commented Dec 2, 2020

The workaround in #42774 will probably address this, but it relies on sigaltstack, which isn’t working on ios-arm64 until iOS 14.

You ought to be able to use sigaltstack and SA_ONSTACK on iOS when the architecture is x86_64 (meaning the iOS simulator) or arm64 starting with iOS 14. #42774 (comment). If you do that, then the mlock workaround should keep this bug at bay on newer OS versions.

Apple FB8922558 aims to get this fixed in the kernel, because it is a kernel bug.

@eliasnaur
Copy link
Contributor Author

@eliasnaur eliasnaur commented Dec 2, 2020

I've upgraded all builders to iOS 14. I'm not sure what versions are covered by your "on newer OS versions", but I think Go should run on iOS < 14, at least until iOS 15 is out.

@eliasnaur
Copy link
Contributor Author

@eliasnaur eliasnaur commented Dec 2, 2020

Continuing the discussion from #42774 (comment) here.

If there is someway we can reproduce the failure more easily (like running the io benchmarks in this issue), that would be helpful. Then we can play with the workaround and other ideas and see if it is effective.

It may also be helpful if you could get a crash report.

@cherrymui I tried running ./all.bash while running the io benchmark concurrently. all.bash succeeded but the benchmark failed with an memory range error:

 ../bin/go test io -run='^$' -short -bench=BenchmarkCopy -count=1000
goos: ios
goarch: arm64
pkg: io
BenchmarkCopyNSmall-2   	 1000000	      1279 ns/op
BenchmarkCopyNSmall-2   	runtime: memory allocated by OS [0x280000000, 0x2a4000000) not in usable address space: base outside usable address space
fatal error: memory reservation exceeds address space limit

runtime stack:
runtime.throw(0x104135539, 0x2e)
	/var/root/goroot2/src/runtime/panic.go:1112 +0x54
runtime.(*mheap).sysAlloc(0x10417be20, 0x20000000, 0x16bddb368, 0x10404b888)
	/var/root/goroot2/src/runtime/malloc.go:720 +0x630
runtime.(*mheap).grow(0x10417be20, 0x10000, 0x0)
	/var/root/goroot2/src/runtime/mheap.go:1346 +0x74
runtime.(*mheap).allocSpan(0x10417be20, 0x10000, 0x100, 0x105d45690)
	/var/root/goroot2/src/runtime/mheap.go:1173 +0x648
runtime.(*mheap).alloc.func1()
	/var/root/goroot2/src/runtime/mheap.go:910 +0x4c
runtime.systemstack(0x16bddb4a8)
	/var/root/goroot2/src/runtime/asm_arm64.s:248 +0xa0
runtime.mstart()
	/var/root/goroot2/src/runtime/proc.go:1183

goroutine 1021 [running]:
runtime.systemstack_switch()
	/var/root/goroot2/src/runtime/asm_arm64.s:193 +0x8 fp=0x130335b20 sp=0x130335b10 pc=0x10408b008
runtime.(*mheap).alloc(0x10417be20, 0x10000, 0x105d40101, 0x0)
	/var/root/goroot2/src/runtime/mheap.go:904 +0x64 fp=0x130335b70 sp=0x130335b20 pc=0x10404ae64
runtime.(*mcache).allocLarge(0x1044485b8, 0x1ffffe00, 0x104190101, 0x104196b68)
	/var/root/goroot2/src/runtime/mcache.go:224 +0x90 fp=0x130335be0 sp=0x130335b70 pc=0x10403c0a0
runtime.mallocgc(0x1ffffe00, 0x1041ada20, 0x104448501, 0x135)
	/var/root/goroot2/src/runtime/malloc.go:1078 +0x82c fp=0x130335c90 sp=0x130335be0 pc=0x10403392c
runtime.makeslice(0x1041ada20, 0x1ffffe00, 0x1ffffe00, 0x1)
	/var/root/goroot2/src/runtime/slice.go:98 +0x74 fp=0x130335cc0 sp=0x130335c90 pc=0x104071204
bytes.makeSlice(0x1ffffe00, 0x0, 0x0, 0x0)
	/var/root/goroot2/src/bytes/buffer.go:229 +0x5c fp=0x130335d00 sp=0x130335cc0 pc=0x1040daaec
bytes.(*Buffer).grow(0x130092050, 0x200, 0x0)
	/var/root/goroot2/src/bytes/buffer.go:142 +0x130 fp=0x130335d50 sp=0x130335d00 pc=0x1040da570
bytes.(*Buffer).Write(0x130092050, 0x118676000, 0x200, 0x200, 0x0, 0x0, 0x0)
	/var/root/goroot2/src/bytes/buffer.go:172 +0xc4 fp=0x130335d80 sp=0x130335d50 pc=0x1040da7a4
io.copyBuffer(0x1041dcc78, 0x130092050, 0x1041dcb58, 0x118655860, 0x118676000, 0x200, 0x200, 0x200, 0x0, 0x0)
	/var/root/goroot2/src/io/io.go:425 +0x1a8 fp=0x130335e00 sp=0x130335d80 pc=0x1040a43c8
io.Copy(...)
	/var/root/goroot2/src/io/io.go:382
io.CopyN(0x1041dcc78, 0x130092050, 0x1041dca98, 0x1302ec060, 0x200, 0x200, 0x0, 0x0)
	/var/root/goroot2/src/io/io.go:358 +0x84 fp=0x130335e60 sp=0x130335e00 pc=0x1040a40e4
io_test.BenchmarkCopyNSmall(0x13031e480)
	/var/root/goroot2/src/io/io_test.go:181 +0x198 fp=0x130335ee0 sp=0x130335e60 pc=0x10411f368
testing.(*B).runN(0x13031e480, 0x928bf)
	/var/root/goroot2/src/testing/benchmark.go:192 +0xfc fp=0x130335f50 sp=0x130335ee0 pc=0x1040dce4c
testing.(*B).launch(0x13031e480)
	/var/root/goroot2/src/testing/benchmark.go:325 +0xcc fp=0x130335fd0 sp=0x130335f50 pc=0x1040dd4cc
runtime.goexit()
	/var/root/goroot2/src/runtime/asm_arm64.s:1130 +0x4 fp=0x130335fd0 sp=0x130335fd0 pc=0x10408d5c4
created by testing.(*B).doBench
	/var/root/goroot2/src/testing/benchmark.go:280 +0x48

goroutine 1 [chan receive]:
testing.(*B).doBench(0x13031e480, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/var/root/goroot2/src/testing/benchmark.go:281 +0x5c
testing.(*benchContext).processBench(0x130016030, 0x13031e480)
	/var/root/goroot2/src/testing/benchmark.go:580 +0x194
testing.(*B).run(0x13031e240)
	/var/root/goroot2/src/testing/benchmark.go:272 +0x58
testing.(*B).Run(0x13031e000, 0x10412f254, 0x13, 0x1041d9830, 0xbfea243223ce0000)
	/var/root/goroot2/src/testing/benchmark.go:668 +0x330
testing.runBenchmarks.func1(0x13031e000)
	/var/root/goroot2/src/testing/benchmark.go:541 +0x74
testing.(*B).runN(0x13031e000, 0x1)
	/var/root/goroot2/src/testing/benchmark.go:192 +0xfc
testing.runBenchmarks(0x10412c11c, 0x2, 0x130016018, 0x10416d700, 0x2, 0x2, 0x104175720)
	/var/root/goroot2/src/testing/benchmark.go:550 +0x34c
testing.(*M).Run(0x130152000, 0x0)
	/var/root/goroot2/src/testing/testing.go:1425 +0x434
main.main()
	_testmain.go:171 +0x14c
exit status 2
FAIL	io	2.086s
FAIL

Another possibility could be #41702. The workaround (CL https://go-review.googlesource.com/c/go/+/262817 and https://go-review.googlesource.com/c/go/+/262438) only covers GOOS="darwin". Maybe we should extend them ios as well.

This issue being the iOS version of #41702 sounds plausible to me, since both crashes in this issue are from an exec (vet and compile).

@gopherbot
Copy link

@gopherbot gopherbot commented Dec 3, 2020

Change https://golang.org/cl/275293 mentions this issue: runtime: avoid receiving preemotion signal while exec'ing

gopherbot pushed a commit that referenced this issue Dec 4, 2020
The iOS kernel has the same problem as the macOS kernel. Extend
the workaround of #41702 (CL 262438 and CL 262817) to iOS.

Updates #35851.

Change-Id: I7ccec00dc96643c08c5be8b385394856d0fa0f64
Reviewed-on: https://go-review.googlesource.com/c/go/+/275293
Trust: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
@bcmills bcmills changed the title runtime: signal: illegal instruction on darwin/arm64 runtime: signal: illegal instruction on ios-arm64 Dec 11, 2020
@bcmills bcmills changed the title runtime: signal: illegal instruction on ios-arm64 runtime: signal: illegal instruction on ios/arm64 Dec 11, 2020
@cherrymui
Copy link
Contributor

@cherrymui cherrymui commented Dec 11, 2020

Yeah, apparently that was not. I'm planning to do more experiments.

@gopherbot
Copy link

@gopherbot gopherbot commented Dec 21, 2020

Change https://golang.org/cl/279489 mentions this issue: runtime: use sigaltstack+mlock on iOS

@bcmills
Copy link
Member

@bcmills bcmills commented Jun 21, 2021

Still a very frequent failure mode on the ios-arm64-corellium builder: ~250 failures since January.

2021-06-21T14:12:09-1de3329/ios-arm64-corellium
2021-06-21T14:10:22-117ebe0/ios-arm64-corellium
2021-06-18T22:05:09-9401172/ios-arm64-corellium
2021-06-16T14:23:50-a6a853f/ios-arm64-corellium
2021-06-16T04:45:46-a752bc0/ios-arm64-corellium
2021-06-15T18:09:44-723f199/ios-arm64-corellium
2021-06-15T18:05:29-4d2d89f/ios-arm64-corellium
2021-06-15T10:13:08-abc56fd/ios-arm64-corellium
2021-06-14T22:13:47-8a5a6f4/ios-arm64-corellium
2021-05-17T18:03:56-a2c07a9/ios-arm64-corellium
2021-05-15T02:39:08-ce92a20/ios-arm64-corellium
2021-05-14T07:53:46-c925e15/ios-arm64-corellium
2021-05-14T00:56:52-12d383c/ios-arm64-corellium
2021-05-13T18:59:27-7a7624a/ios-arm64-corellium
2021-05-13T14:52:20-2a61b3c/ios-arm64-corellium
2021-05-12T20:20:11-6db7480/ios-arm64-corellium
2021-05-12T15:23:09-0388670/ios-arm64-corellium
2021-05-11T21:22:08-9995c6b/ios-arm64-corellium
2021-05-11T14:21:06-2520e72/ios-arm64-corellium
2021-05-09T17:07:01-bedf2c4/ios-arm64-corellium
2021-05-08T14:59:49-68327e1/ios-arm64-corellium
2021-05-07T20:44:40-d80d142/ios-arm64-corellium
2021-05-07T16:08:24-9bfa6f7/ios-arm64-corellium
2021-05-07T03:55:29-f5423ea/ios-arm64-corellium
2021-05-07T02:17:32-d2b0311/ios-arm64-corellium
2021-05-06T17:09:23-51ff3a6/ios-arm64-corellium
2021-05-06T16:28:56-5f9fe47/ios-arm64-corellium
2021-05-04T17:37:39-da7496d/ios-arm64-corellium
2021-05-04T16:10:58-371ea54/ios-arm64-corellium
2021-05-04T00:15:27-2422c5e/ios-arm64-corellium
2021-05-03T22:59:55-731a015/ios-arm64-corellium
2021-05-03T18:23:49-7918547/ios-arm64-corellium
2021-05-03T18:17:42-8a4b729/ios-arm64-corellium
2021-05-03T17:46:12-90ec257/ios-arm64-corellium
2021-05-03T16:42:22-169155d/ios-arm64-corellium
2021-05-03T16:25:05-d75fbac/ios-arm64-corellium
2021-05-03T15:53:36-8327d21/ios-arm64-corellium
2021-05-03T12:20:09-30674ae/ios-arm64-corellium
2021-05-03T01:35:44-2c9f5a1/ios-arm64-corellium
2021-05-02T21:24:28-7eb2d30/ios-arm64-corellium
2021-05-02T20:38:13-fadad85/ios-arm64-corellium
2021-05-02T18:22:19-0d32d9e/ios-arm64-corellium
2021-05-02T18:13:38-352a322/ios-arm64-corellium
2021-05-01T19:17:47-053fe2f/ios-arm64-corellium
2021-04-30T18:43:09-83ac59b/ios-arm64-corellium
2021-04-30T18:06:26-c05d50f/ios-arm64-corellium
2021-04-29T16:29:52-f7c6f62/ios-arm64-corellium
2021-04-29T15:21:14-f12dfea/ios-arm64-corellium
2021-04-29T04:19:20-42953bc/ios-arm64-corellium
2021-04-29T04:18:39-d099475/ios-arm64-corellium
2021-04-29T01:15:34-756fd56/ios-arm64-corellium
2021-04-28T20:36:50-fa6ed6e/ios-arm64-corellium
2021-04-28T20:21:24-414af50/ios-arm64-corellium
2021-04-28T20:17:20-c96fec9/ios-arm64-corellium
2021-04-28T20:16:36-b36596b/ios-arm64-corellium
2021-04-28T19:52:58-07e006d/ios-arm64-corellium
2021-04-28T19:52:04-168dd4e/ios-arm64-corellium
2021-04-28T19:13:50-ad989c7/ios-arm64-corellium
2021-04-28T18:50:47-42812a2/ios-arm64-corellium
2021-04-28T18:50:41-ea65a12/ios-arm64-corellium
2021-04-28T17:39:34-a547625/ios-arm64-corellium
2021-04-28T17:12:39-22a56b6/ios-arm64-corellium
2021-04-28T15:57:49-92c9f3a/ios-arm64-corellium
2021-04-28T15:55:10-becb9a2/ios-arm64-corellium
2021-04-28T13:49:52-4fe324d/ios-arm64-corellium
2021-04-28T00:49:58-c9f4350/ios-arm64-corellium
2021-04-27T21:44:16-645cb62/ios-arm64-corellium
2021-04-27T19:40:24-b9dfaf7/ios-arm64-corellium
2021-04-27T19:12:41-0c3557e/ios-arm64-corellium
2021-04-27T11:50:41-1eca6aa/ios-arm64-corellium
2021-04-27T01:16:39-40254ec/ios-arm64-corellium
2021-04-26T21:35:26-be28caf/ios-arm64-corellium
2021-04-26T18:42:12-8ff1da0/ios-arm64-corellium
2021-04-26T17:13:36-d5d24db/ios-arm64-corellium
2021-04-26T14:55:26-00d42ff/ios-arm64-corellium
2021-04-24T20:08:46-70deaa3/ios-arm64-corellium
2021-04-23T22:31:20-e7db792/ios-arm64-corellium
2021-04-23T21:43:08-a25d1d4/ios-arm64-corellium
2021-04-23T21:43:06-e5a6c52/ios-arm64-corellium
2021-04-23T21:43:02-59ceb98/ios-arm64-corellium
2021-04-23T21:42:59-41e5ae4/ios-arm64-corellium
2021-04-23T21:42:52-19470dc/ios-arm64-corellium
2021-04-23T20:57:54-691e1b8/ios-arm64-corellium
2021-04-23T19:08:32-c3e2ed7/ios-arm64-corellium
2021-04-23T17:50:33-d2f96f2/ios-arm64-corellium
2021-04-23T14:06:54-d4bfe00/ios-arm64-corellium
2021-04-23T02:29:37-1b0a031/ios-arm64-corellium
2021-04-23T01:16:19-cfac62a/ios-arm64-corellium
2021-04-23T00:40:48-050b408/ios-arm64-corellium
2021-04-22T22:01:47-7405968/ios-arm64-corellium
2021-04-22T21:23:59-f7afdfd/ios-arm64-corellium
2021-04-22T18:07:38-1a56655/ios-arm64-corellium
2021-04-22T17:02:01-d4aa720/ios-arm64-corellium
2021-04-22T16:38:37-e8666ab/ios-arm64-corellium
2021-04-22T16:15:44-d3853fb/ios-arm64-corellium
2021-04-22T14:40:40-d5b2d80/ios-arm64-corellium
2021-04-22T04:07:38-14a18b7/ios-arm64-corellium
2021-04-22T04:07:32-a71528a/ios-arm64-corellium
2021-04-22T03:03:48-617a83e/ios-arm64-corellium
2021-04-22T03:03:41-f0a8101/ios-arm64-corellium
2021-04-22T01:40:02-5daefc5/ios-arm64-corellium
2021-04-22T01:29:53-0636d88/ios-arm64-corellium
2021-04-22T00:34:25-02a8e83/ios-arm64-corellium
2021-04-21T23:21:55-7bedd47/ios-arm64-corellium
2021-04-21T21:25:26-7e97e4e/ios-arm64-corellium
2021-04-21T20:44:00-122fca4/ios-arm64-corellium
2021-04-21T20:36:36-614a9c2/ios-arm64-corellium
2021-04-21T20:24:34-2550563/ios-arm64-corellium
2021-04-21T16:46:52-35806ef/ios-arm64-corellium
2021-04-21T14:15:55-daee726/ios-arm64-corellium
2021-04-21T13:38:59-4d56576/ios-arm64-corellium
2021-04-21T09:07:02-7735ec9/ios-arm64-corellium
2021-04-21T08:46:51-acf1b46/ios-arm64-corellium
2021-04-21T04:27:01-f53c2fa/ios-arm64-corellium
2021-04-21T04:23:52-81fcb18/ios-arm64-corellium
2021-04-20T21:59:20-f448cb8/ios-arm64-corellium
2021-04-20T21:00:46-57b0d83/ios-arm64-corellium
2021-04-19T23:08:09-62cad23/ios-arm64-corellium
2021-04-19T22:25:20-8865548/ios-arm64-corellium
2021-04-19T21:27:43-e97d8eb/ios-arm64-corellium
2021-04-19T16:49:40-5780ab4/ios-arm64-corellium
2021-04-19T04:37:32-a72622d/ios-arm64-corellium
2021-04-18T17:40:12-4efd581/ios-arm64-corellium
2021-04-16T21:20:31-02a2ff4/ios-arm64-corellium
2021-04-16T20:56:09-9fbcba6/ios-arm64-corellium
2021-04-16T03:16:55-d26fc68/ios-arm64-corellium
2021-04-15T19:41:39-7ed6d1f/ios-arm64-corellium
2021-04-15T19:41:38-a63ff39/ios-arm64-corellium
2021-04-15T15:48:19-1d20a36/ios-arm64-corellium
2021-04-15T12:38:13-7ad496b/ios-arm64-corellium
2021-04-15T09:18:03-61a08fc/ios-arm64-corellium
2021-04-15T08:44:21-0184b44/ios-arm64-corellium
2021-04-14T03:15:34-e7ab1a5/ios-arm64-corellium
2021-04-12T22:19:11-841bc14/ios-arm64-corellium
2021-04-12T21:26:59-5c9b6e8/ios-arm64-corellium
2021-04-12T19:22:52-1b736b3/ios-arm64-corellium
2021-04-12T19:22:50-a25a77a/ios-arm64-corellium
2021-04-12T18:08:47-849dba0/ios-arm64-corellium
2021-04-12T15:51:43-51a47b7/ios-arm64-corellium
2021-04-12T05:11:59-e12abe4/ios-arm64-corellium
2021-04-12T00:45:40-424abc8/ios-arm64-corellium
2021-04-10T19:02:05-36c5f90/ios-arm64-corellium
2021-04-10T19:02:03-4638545/ios-arm64-corellium
2021-04-10T03:38:08-6382ec1/ios-arm64-corellium
2021-04-10T01:46:41-52bf14e/ios-arm64-corellium
2021-04-09T23:54:31-554d2c4/ios-arm64-corellium
2021-04-09T23:18:47-5305bde/ios-arm64-corellium
2021-04-09T22:57:48-281d168/ios-arm64-corellium
2021-04-09T19:11:12-756e2b1/ios-arm64-corellium
2021-04-09T18:49:05-c3faff7/ios-arm64-corellium
2021-04-09T18:20:48-814c5ff/ios-arm64-corellium
2021-04-09T18:19:42-952187a/ios-arm64-corellium
2021-04-09T18:11:15-fcf8a66/ios-arm64-corellium
2021-04-09T17:56:50-0ad4688/ios-arm64-corellium
2021-04-09T17:49:01-2698be4/ios-arm64-corellium
2021-04-09T15:01:13-6951da5/ios-arm64-corellium
2021-04-09T14:36:20-77b3269/ios-arm64-corellium
2021-04-09T14:31:01-d138ee2/ios-arm64-corellium
2021-04-09T13:19:10-dcc801e/ios-arm64-corellium
2021-04-09T13:10:36-c432917/ios-arm64-corellium
2021-04-09T12:56:04-519f223/ios-arm64-corellium
2021-04-09T09:01:07-4d7d7a4/ios-arm64-corellium
2021-04-09T04:03:50-8518aac/ios-arm64-corellium
2021-04-08T22:45:16-5811605/ios-arm64-corellium
2021-04-08T21:26:36-ec367e5/ios-arm64-corellium
2021-04-08T19:58:50-bb76193/ios-arm64-corellium
2021-04-08T19:52:03-7938442/ios-arm64-corellium
2021-04-08T19:32:37-46ffbec/ios-arm64-corellium
2021-04-08T17:43:33-d474b6c/ios-arm64-corellium
2021-04-08T17:18:24-23e1d36/ios-arm64-corellium
2021-04-08T16:37:24-31d2556/ios-arm64-corellium
2021-04-08T15:03:31-283b020/ios-arm64-corellium
2021-04-08T14:08:29-1749f39/ios-arm64-corellium
2021-04-08T06:08:49-2123dfb/ios-arm64-corellium
2021-04-08T02:17:19-89ca1ce/ios-arm64-corellium
2021-04-07T18:44:30-e306d06/ios-arm64-corellium
2021-04-07T16:57:40-b3064b6/ios-arm64-corellium
2021-04-07T13:24:10-4520da4/ios-arm64-corellium
2021-04-07T09:59:58-d6aa162/ios-arm64-corellium
2021-04-07T05:19:29-8f1099b/ios-arm64-corellium
2021-04-07T05:19:15-7d5c54e/ios-arm64-corellium
2021-04-07T03:42:11-8462169/ios-arm64-corellium
2021-04-06T19:55:44-bcc4422/ios-arm64-corellium
2021-04-06T19:00:26-d6a90d0/ios-arm64-corellium
2021-04-06T18:59:08-3a30381/ios-arm64-corellium
2021-04-05T20:05:34-254fb85/ios-arm64-corellium
2021-04-05T19:29:17-d5b9dc1/ios-arm64-corellium
2021-04-05T16:22:18-191167c/ios-arm64-corellium
2021-04-05T16:22:12-24dd8cf/ios-arm64-corellium
2021-04-04T20:04:37-4230a6e/ios-arm64-corellium
2021-04-03T20:16:00-dac136f/ios-arm64-corellium
2021-04-03T18:16:29-6986c02/ios-arm64-corellium
2021-04-02T19:27:06-34b87b4/ios-arm64-corellium
2021-04-02T16:30:18-759116b/ios-arm64-corellium
2021-04-02T01:13:58-41e8a9f/ios-arm64-corellium
2021-03-30T17:51:37-c40dc67/ios-arm64-corellium
2021-03-30T16:13:36-89b141c/ios-arm64-corellium
2021-03-30T15:43:47-e4a4161/ios-arm64-corellium
2021-03-30T03:05:45-eeadfa2/ios-arm64-corellium
2021-03-29T16:48:08-2abf280/ios-arm64-corellium
2021-03-26T06:03:20-98a9023/ios-arm64-corellium
2021-03-25T21:35:05-374b190/ios-arm64-corellium
2021-03-25T20:57:58-691db37/ios-arm64-corellium
2021-03-25T19:23:18-ada77d2/ios-arm64-corellium
2021-03-25T04:14:58-ddcdbb4/ios-arm64-corellium
2021-03-25T03:19:56-4889afe/ios-arm64-corellium
2021-03-25T02:50:11-a95e2ae/ios-arm64-corellium
2021-03-24T20:19:44-dade83a/ios-arm64-corellium
2021-03-24T05:15:53-2e94401/ios-arm64-corellium
2021-03-23T18:14:47-c59b17e/ios-arm64-corellium
2021-03-23T03:49:17-b8371d4/ios-arm64-corellium
2021-03-22T16:01:44-bd8b3fe/ios-arm64-corellium
2021-03-02T19:15:52-e9eed78/ios-arm64-corellium
2021-02-26T18:57:20-d8e33d5/ios-arm64-corellium
2021-02-26T18:23:29-2d76081/ios-arm64-corellium
2021-02-26T10:18:26-23943a6/ios-arm64-corellium
2021-02-26T02:11:50-9a555fc/ios-arm64-corellium
2021-02-25T21:52:49-9a7fe19/ios-arm64-corellium
2021-02-25T19:42:00-526ee96/ios-arm64-corellium
2021-02-25T19:34:27-194b636/ios-arm64-corellium
2021-02-25T18:57:20-4ebb6f5/ios-arm64-corellium
2021-02-24T19:26:13-80ddc17/ios-arm64-corellium
2021-02-24T16:38:56-27684ea/ios-arm64-corellium
2021-02-24T15:48:11-b97b145/ios-arm64-corellium
2021-02-24T04:00:46-3780529/ios-arm64-corellium
2021-02-23T22:57:27-6cc8aa7/ios-arm64-corellium
2021-02-23T21:18:30-fa40c02/ios-arm64-corellium
2021-02-23T20:06:07-a4dac8b/ios-arm64-corellium
2021-02-23T17:58:33-f1562c7/ios-arm64-corellium
2021-02-23T15:04:12-ab331c0/ios-arm64-corellium
2021-02-23T15:03:54-91cfbf3/ios-arm64-corellium
2021-02-23T09:16:54-08543f0/ios-arm64-corellium
2021-02-19T16:09:17-02e5a8f/ios-arm64-corellium
2021-02-19T00:02:40-fbe74db/ios-arm64-corellium
2021-02-18T20:47:38-3b7277d/ios-arm64-corellium
2021-02-16T02:07:02-0cb3415/ios-arm64-corellium
2021-02-15T01:54:22-33d72fd/ios-arm64-corellium
2021-02-13T15:15:13-66c2709/ios-arm64-corellium
2021-02-05T21:03:18-4516afe/ios-arm64-corellium
2021-01-27T21:11:22-00f2ff5/ios-arm64-corellium
2021-01-26T00:30:18-1d5e146/ios-arm64-corellium
2021-01-25T20:41:05-3d85c69/ios-arm64-corellium
2021-01-25T15:12:58-ff82cc9/ios-arm64-corellium
2021-01-22T21:16:10-a2cef9b/ios-arm64-corellium
2021-01-22T16:43:04-ec40517/ios-arm64-corellium
2021-01-21T19:08:52-5a8a226/ios-arm64-corellium
2021-01-19T17:33:33-ccb2e90/ios-arm64-corellium
2021-01-14T22:01:23-e125ccd/ios-arm64-corellium
2021-01-14T16:50:30-c73232d/ios-arm64-corellium
2021-01-11T18:19:08-c3b4c70/ios-arm64-corellium
2021-01-04T17:59:30-9eef49c/ios-arm64-corellium

@cherrymui
Copy link
Contributor

@cherrymui cherrymui commented Jun 21, 2021

I tried to understand it better, but with the limited access on the builders I couldn't get anything useful. I'm inclined that this is a kernel bug or simulator bug (it is not a real iOS device, right?).

It might be helpful to get a crash report or a core dump or something.

@bcmills
Copy link
Member

@bcmills bcmills commented Jun 21, 2021

I'm inclined that this is a kernel bug or simulator bug (it is not a real iOS device, right?).

Agreed, a kernel bug seems likely, especially given #42774. (I don't think we have any evidence to suggest a simulator bug at this point, but I can't rule it out either.)

There was some discussion on #42774 (comment) of porting a workaround based on sigaltstack. Does that still seem viable?

@cherrymui
Copy link
Contributor

@cherrymui cherrymui commented Jun 21, 2021

There was some discussion on #42774 (comment) of porting a workaround based on sigaltstack. Does that still seem viable?

I recall I wrote a CL for the workaround and Elias tried but it wasn't helpful...

@eliasnaur
Copy link
Contributor Author

@eliasnaur eliasnaur commented Jun 22, 2021

Let me know if there are more experiments I can do. I believe gomote should work, as long as the Corellium builders are turned on (I suspect they turn them off when upgrading infrastructure).

@eliasnaur
Copy link
Contributor Author

@eliasnaur eliasnaur commented Jun 22, 2021

I found 3 crash reports in the /var/mobile/Library/Logs/CrashReporter directory on iOS builder no. 3, all three dated 2021-06-21:

{"app_name":"compile","timestamp":"2021-06-21 07:14:05.00 -0700","app_version":"","slice_uuid":"466bad2f-2a67-3403-8bd6-112aecef5eac","build_version":"","platform":2,"share_with_app_devs":0,"is_first_party":1,"bug_type":"109","os_version":"iPhone OS 14.2 (18B92)","incident_id":"10DE5A2B-4502-438E-AF58-ED817E305251","name":"compile"} Incident Identifier: 10DE5A2B-4502-438E-AF58-ED817E305251 CrashReporter Key: f09d9091c97b4e4f6bb695146954c56a7f0495a7 Hardware Model: iPhone8,4 Process: compile [77475] Path: /private/var/tmp/workdir-host-ios-arm64-corellium-ios/go/pkg/tool/ios_arm64/compile Identifier: compile Version: ??? Code Type: ARM-64 (Native) Role: Unspecified Parent Process: go_bootstrap [77188] Coalition: org.golang.builder [157]

Date/Time: 2021-06-21 07:14:05.4771 -0700
Launch Time: 2021-06-21 07:13:42.8174 -0700
OS Version: iPhone OS 14.2 (18B92)
Release Type: User
Baseband Version:
Report Version: 104

Exception Type: EXC_CRASH (SIGILL)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Exception Note: EXC_CORPSE_NOTIFY
Termination Signal: Illegal instruction: 4
Termination Reason: Namespace SIGNAL, Code 0x4
Terminating Process: compile [77475]
Triggered by Thread: 0

Thread 0 name: Dispatch queue: com.apple.main-thread
Thread 0 Crashed:
0 libsystem_kernel.dylib 0x00000001b77f961c 0x1b77d2000 + 161308
1 libsystem_pthread.dylib 0x00000001d2e262fc 0x1d2e23000 + 13052
2 compile 0x000000010217ff58 0x102110000 + 458584
3 compile 0x000000010217ecc8 0x102110000 + 453832
4 ??? 0x00000000d65f03c0 0 + 3596551104

Thread 1:
0 libsystem_pthread.dylib 0x00000001d2e3186c 0x1d2e23000 + 59500

Thread 2:
0 libsystem_kernel.dylib 0x00000001b77f9bf0 0x1b77d2000 + 162800
1 libsystem_c.dylib 0x00000001954766d4 0x195406000 + 460500
2 libsystem_c.dylib 0x00000001954765ac 0x195406000 + 460204
3 compile 0x000000010217fbe4 0x102110000 + 457700
4 compile 0x000000010217ecc8 0x102110000 + 453832

Thread 3:
0 compile 0x000000010217f370 0x102110000 + 455536
1 compile 0x0000000102226be4 0x102110000 + 1141732

Thread 4:
0 libsystem_kernel.dylib 0x00000001b77f961c 0x1b77d2000 + 161308
1 libsystem_pthread.dylib 0x00000001d2e262fc 0x1d2e23000 + 13052
2 compile 0x000000010217ff58 0x102110000 + 458584
3 compile 0x000000010217ecc8 0x102110000 + 453832
4 ??? 0x00000000d65f03c0 0 + 3596551104

Thread 5:
0 libsystem_kernel.dylib 0x00000001b77f961c 0x1b77d2000 + 161308
1 libsystem_pthread.dylib 0x00000001d2e262fc 0x1d2e23000 + 13052
2 compile 0x000000010217ff58 0x102110000 + 458584
3 compile 0x000000010217ecc8 0x102110000 + 453832
4 ??? 0x00000000d65f03c0 0 + 3596551104

Thread 0 crashed with ARM Thread State (64-bit):
x0: 0x0000000000000104 x1: 0x0000000000000000 x2: 0x0000000000003700 x3: 0x0000000000000000
x4: 0x0000000000000000 x5: 0x00000000000000a0 x6: 0x0000000000000000 x7: 0x0000000000000000
x8: 0x000000016dceca48 x9: 0x0000000000003701 x10: 0x0000000102b81078 x11: 0x0000010000000102
x12: 0x00000000000020a0 x13: 0x0000000000000100 x14: 0x0000000000000000 x15: 0x0000000000000100
x16: 0x0000000000000131 x17: 0x0000000000000000 x18: 0x0000000000000000 x19: 0x0000000102b81060
x20: 0x0000000102b810a0 x21: 0x000000010352f960 x22: 0x0000000000000000 x23: 0x0000000000000000
x24: 0x0000000000003700 x25: 0x0000000000003701 x26: 0x0000000000003800 x27: 0xffffffffffffffe0
x28: 0x0000000102b804e0 fp: 0x000000016dcecac0 lr: 0x00000001d2e262fc
sp: 0x000000016dceca30 pc: 0x00000001b77f961c cpsr: 0x60000000
esr: 0x56000080 Address size fault

Binary Images:
0x102110000 - 0x102abbfff compile arm64 <466bad2f2a6734038bd6112aecef5eac> /var/tmp/workdir-host-ios-arm64-corellium-ios/go/pkg/tool/ios_arm64/compile
0x103264000 - 0x10326bfff libsubstrate.dylib arm64 /usr/lib/libsubstrate.dylib
0x103274000 - 0x10327bfff pspawn_payload.dylib arm64 <010fa335cb823edda8339536e71ec9d9> /usr/lib/pspawn_payload.dylib
0x103298000 - 0x1032a3fff substitute-loader.dylib arm64 /usr/lib/substitute-loader.dylib
0x1032b4000 - 0x1032cffff libsubstitute.0.dylib arm64 <560ee8b4dbb43c5c98ad5eb56be360b6> /usr/lib/libsubstitute.0.dylib
0x103488000 - 0x1034f3fff dyld arm64 /usr/lib/dyld
0x18c3a5000 - 0x18c423fff libdispatch.dylib arm64 <6c143c51288f3cd084dd17b9a0aaee62> /usr/lib/system/libdispatch.dylib
0x18c424000 - 0x18c45afff libdyld.dylib arm64 /usr/lib/system/libdyld.dylib
0x18c45b000 - 0x18c6b2fff libicucore.A.dylib arm64 <9fc7dac3deed3a479abbe36867045ba5> /usr/lib/libicucore.A.dylib
0x18c6b3000 - 0x18ca5cfff CoreFoundation arm64 <96f8386dd88a3c89a323a17975c3317f> /System/Library/Frameworks/CoreFoundation.framework/CoreFoundation
0x18cc40000 - 0x18ccb8fff SystemConfiguration arm64 <75fec68015053f2e84dc4a0a8586c617> /System/Library/Frameworks/SystemConfiguration.framework/SystemConfiguration
0x18cda4000 - 0x18d225fff CFNetwork arm64 /System/Library/Frameworks/CFNetwork.framework/CFNetwork
0x18d226000 - 0x18d8ebfff libnetwork.dylib arm64 /usr/lib/libnetwork.dylib
0x18d95d000 - 0x18dbf7fff Foundation arm64 /System/Library/Frameworks/Foundation.framework/Foundation
0x193744000 - 0x193881fff Security arm64 <149f8d12822a31a198bf48ce690f678d> /System/Library/Frameworks/Security.framework/Security
0x1953e0000 - 0x195405fff libsystem_info.dylib arm64 <3536d46d36cd3af4b92c67e4bdebd8c6> /usr/lib/system/libsystem_info.dylib
0x195406000 - 0x195482fff libsystem_c.dylib arm64 /usr/lib/system/libsystem_c.dylib
0x196d64000 - 0x196e0afff IOKit arm64 /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit
0x19ad0f000 - 0x19ad6cfff libMobileGestalt.dylib arm64 <448540b0eca03930b98b2d18101114d6> /usr/lib/libMobileGestalt.dylib
0x19b1dc000 - 0x19b1fefff libsystem_malloc.dylib arm64 <9f2d3d4683d9358f972c1cf61c073a31> /usr/lib/system/libsystem_malloc.dylib
0x19d169000 - 0x19d178fff libsystem_networkextension.dylib arm64 <0e55773557ba3611994ce782f3cbfe75> /usr/lib/system/libsystem_networkextension.dylib
0x1a0178000 - 0x1a01adfff libobjc.A.dylib arm64 <7cad222912fa3d6c82005306fb157fa4> /usr/lib/libobjc.A.dylib
0x1a020e000 - 0x1a0267fff libc++.1.dylib arm64 <88496d66c6683f1797c5d0c91e8bee42> /usr/lib/libc++.1.dylib
0x1a0268000 - 0x1a0280fff libc++abi.dylib arm64 <464199d1cdfe3663bedfcbc1b5f19491> /usr/lib/libc++abi.dylib
0x1a03eb000 - 0x1a042bfff CoreAutoLayout arm64 <83a9b00eb095347aa9a2fd147deb1d75> /System/Library/PrivateFrameworks/CoreAutoLayout.framework/CoreAutoLayout
0x1a1dc6000 - 0x1a1ddcfff libsystem_trace.dylib arm64 <4ed5b6b480e9322784b590bc92570931> /usr/lib/system/libsystem_trace.dylib
0x1a4d5d000 - 0x1a4edffff libsqlite3.dylib arm64 /usr/lib/libsqlite3.dylib
0x1a5a73000 - 0x1a5a7dfff libsystem_notify.dylib arm64 <4dbd2e3e6bc43d1388d0e01ba0cc4e07> /usr/lib/system/libsystem_notify.dylib
0x1a5b28000 - 0x1a5b97fff libcorecrypto.dylib arm64 <071f7a201cf6324783d7d4e309a64c16> /usr/lib/system/libcorecrypto.dylib
0x1a5c8b000 - 0x1a5ca1fff libsystem_asl.dylib arm64 <441adb81097f31d6a85d380a72993d66> /usr/lib/system/libsystem_asl.dylib
0x1a987e000 - 0x1a9888fff IOMobileFramebuffer arm64 <674f742e890231f3b8999fac2c9eb8fa> /System/Library/PrivateFrameworks/IOMobileFramebuffer.framework/IOMobileFramebuffer
0x1ac45b000 - 0x1ac46cfff IOSurface arm64 <19dd2ad445e03197a5c453b357f361a0> /System/Library/Frameworks/IOSurface.framework/IOSurface
0x1aca1b000 - 0x1aca22fff libsystem_symptoms.dylib arm64 /usr/lib/system/libsystem_symptoms.dylib
0x1b77d2000 - 0x1b7802fff libsystem_kernel.dylib arm64 <8bcb6ff951623657a2e25ce5704353e6> /usr/lib/system/libsystem_kernel.dylib
0x1ba248000 - 0x1ba249fff libSystem.B.dylib arm64 <5f81896adca13ef7999f6c5da89a750b> /usr/lib/libSystem.B.dylib
0x1ba68a000 - 0x1ba6f8fff libarchive.2.dylib arm64 /usr/lib/libarchive.2.dylib
0x1cae1e000 - 0x1cae1efff SoftLinking arm64 <2a4d987221d13057a2e4287595be79b5> /System/Library/PrivateFrameworks/SoftLinking.framework/SoftLinking
0x1d1705000 - 0x1d1737fff libCRFSuite.dylib arm64 /usr/lib/libCRFSuite.dylib
0x1d1deb000 - 0x1d1e01fff libapple_nghttp2.dylib arm64 <741cbe1e00f93a20b845d77885621920> /usr/lib/libapple_nghttp2.dylib
0x1d1f23000 - 0x1d1f33fff libbsm.0.dylib arm64 <62262c68bf203584b2ea3889b8647e16> /usr/lib/libbsm.0.dylib
0x1d1f34000 - 0x1d1f40fff libbz2.1.0.dylib arm64 /usr/lib/libbz2.1.0.dylib
0x1d1f41000 - 0x1d1f41fff libcharset.1.dylib arm64 <72b0c36f725f38fea3579a515aaf64b9> /usr/lib/libcharset.1.dylib
0x1d1f54000 - 0x1d1f6bfff libcompression.dylib arm64 <36dd56ca87a33b8bbc2e9ee086e82284> /usr/lib/libcompression.dylib
0x1d1f6c000 - 0x1d1f81fff libcoretls.dylib arm64 <31987a4e4ec23c89952efad70a0d71d5> /usr/lib/libcoretls.dylib
0x1d1f82000 - 0x1d1f83fff libcoretls_cfhelpers.dylib arm64 <56fa9fe8ebb83d17872877e4c5cd0848> /usr/lib/libcoretls_cfhelpers.dylib
0x1d1fae000 - 0x1d1fb5fff libdns_services.dylib arm64 <8d3832488b4333b492251a634982e109> /usr/lib/libdns_services.dylib
0x1d1fd3000 - 0x1d1fd3fff libenergytrace.dylib arm64 /usr/lib/libenergytrace.dylib
0x1d2022000 - 0x1d2113fff libiconv.2.dylib arm64 <6bf1c6e1e4b93cc3928eaa07c4ecdfc5> /usr/lib/libiconv.2.dylib
0x1d2131000 - 0x1d2132fff liblangid.dylib arm64 <982686583ed13ebaa2cc14345035abe9> /usr/lib/liblangid.dylib
0x1d213f000 - 0x1d2157fff liblzma.5.dylib arm64 <60e6e4ad8b823583b81e7e39df26637f> /usr/lib/liblzma.5.dylib
0x1d27cb000 - 0x1d27fefff libpcap.A.dylib arm64 <0d7a2422cdb838ce9604933d6ced3742> /usr/lib/libpcap.A.dylib
0x1d295b000 - 0x1d2a40fff libxml2.2.dylib arm64 <3102a95fbfd13d2292ed7d4f31ce994f> /usr/lib/libxml2.2.dylib
0x1d2a6e000 - 0x1d2a7ffff libz.1.dylib arm64 /usr/lib/libz.1.dylib
0x1d2cac000 - 0x1d2cb1fff libcache.dylib arm64 <93f46de334c83b548e87b685e63149e7> /usr/lib/system/libcache.dylib
0x1d2cb2000 - 0x1d2cbefff libcommonCrypto.dylib arm64 <163c756fa4413bb7b2ca067463d98526> /usr/lib/system/libcommonCrypto.dylib
0x1d2cbf000 - 0x1d2cc3fff libcompiler_rt.dylib arm64 <6749df438089311b939a87f9ca479efd> /usr/lib/system/libcompiler_rt.dylib
0x1d2cc4000 - 0x1d2cccfff libcopyfile.dylib arm64 <5ea5d77aebc930459a3ccf1838e73c6b> /usr/lib/system/libcopyfile.dylib
0x1d2daa000 - 0x1d2daafff liblaunch.dylib arm64 /usr/lib/system/liblaunch.dylib
0x1d2dab000 - 0x1d2db0fff libmacho.dylib arm64 <863174fc64663104aca38d1d4a82d075> /usr/lib/system/libmacho.dylib
0x1d2db1000 - 0x1d2db3fff libremovefile.dylib arm64 /usr/lib/system/libremovefile.dylib
0x1d2db4000 - 0x1d2db5fff libsystem_blocks.dylib arm64 /usr/lib/system/libsystem_blocks.dylib
0x1d2db6000 - 0x1d2db8fff libsystem_collections.dylib arm64 <828126f3e6583da2aceac728678588de> /usr/lib/system/libsystem_collections.dylib
0x1d2db9000 - 0x1d2dbdfff libsystem_configuration.dylib arm64 <27db1d5d607530bf888edca2c13e80ef> /usr/lib/system/libsystem_configuration.dylib
0x1d2dbe000 - 0x1d2dd0fff libsystem_containermanager.dylib arm64 /usr/lib/system/libsystem_containermanager.dylib
0x1d2dd1000 - 0x1d2dd2fff libsystem_coreservices.dylib arm64 /usr/lib/system/libsystem_coreservices.dylib
0x1d2dd3000 - 0x1d2ddcfff libsystem_darwin.dylib arm64 /usr/lib/system/libsystem_darwin.dylib
0x1d2ddd000 - 0x1d2de5fff libsystem_dnssd.dylib arm64 /usr/lib/system/libsystem_dnssd.dylib
0x1d2de6000 - 0x1d2de8fff libsystem_featureflags.dylib arm64 <1e7cd17eb5283b85bf4a27e1aa8dfa3e> /usr/lib/system/libsystem_featureflags.dylib
0x1d2de9000 - 0x1d2e16fff libsystem_m.dylib arm64 /usr/lib/system/libsystem_m.dylib
0x1d2e17000 - 0x1d2e21fff libsystem_platform.dylib arm64 /usr/lib/system/libsystem_platform.dylib
0x1d2e22000 - 0x1d2e22fff libsystem_product_info_filter.dylib arm64 <61b5d8563fb33fe9b30540a45f659732> /usr/lib/system/libsystem_product_info_filter.dylib
0x1d2e23000 - 0x1d2e33fff libsystem_pthread.dylib arm64 <93c7f0dc89f13162a1196fc88028d027> /usr/lib/system/libsystem_pthread.dylib
0x1d2e34000 - 0x1d2e37fff libsystem_sandbox.dylib arm64 <176b1087aaad36258e7ffc497673706c> /usr/lib/system/libsystem_sandbox.dylib
0x1d2e38000 - 0x1d2e41fff libunwind.dylib arm64 /usr/lib/system/libunwind.dylib
0x1d2e42000 - 0x1d2e74fff libxpc.dylib arm64 /usr/lib/system/libxpc.dylib

EOF

{"app_name":"compile","timestamp":"2021-06-21 09:57:23.00 -0700","app_version":"","slice_uuid":"709e8d8b-fdc9-330c-bc07-60b0cd693652","build_version":"","platform":2,"share_with_app_devs":0,"is_first_party":1,"bug_type":"109","os_version":"iPhone OS 14.2 (18B92)","incident_id":"AA3C332D-EA5C-42CD-A6F0-56BF34DBC86F","name":"compile"} Incident Identifier: AA3C332D-EA5C-42CD-A6F0-56BF34DBC86F CrashReporter Key: f09d9091c97b4e4f6bb695146954c56a7f0495a7 Hardware Model: iPhone8,4 Process: compile [90439] Path: /private/var/tmp/workdir-host-ios-arm64-corellium-ios/go/pkg/tool/ios_arm64/compile Identifier: compile Version: ??? Code Type: ARM-64 (Native) Role: Unspecified Parent Process: go_bootstrap [90151] Coalition: org.golang.builder [157]

Date/Time: 2021-06-21 09:57:23.5054 -0700
Launch Time: 2021-06-21 09:56:55.8589 -0700
OS Version: iPhone OS 14.2 (18B92)
Release Type: User
Baseband Version:
Report Version: 104

Exception Type: EXC_CRASH (SIGILL)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Exception Note: EXC_CORPSE_NOTIFY
Termination Signal: Illegal instruction: 4
Termination Reason: Namespace SIGNAL, Code 0x4
Terminating Process: compile [90439]
Triggered by Thread: 0

Thread 0 name: Dispatch queue: com.apple.main-thread
Thread 0 Crashed:
0 compile 0x0000000100558898 0x100428000 + 1247384
1 compile 0x000000010054d5ec 0x100428000 + 1201644

Thread 1:
0 libsystem_pthread.dylib 0x00000001d2e3186c 0x1d2e23000 + 59500

Thread 2:
0 libsystem_kernel.dylib 0x00000001b77f9bf0 0x1b77d2000 + 162800
1 libsystem_c.dylib 0x00000001954766d4 0x195406000 + 460500
2 libsystem_c.dylib 0x00000001954765ac 0x195406000 + 460204
3 compile 0x0000000100491544 0x100428000 + 431428
4 compile 0x00000001004906f8 0x100428000 + 427768

Thread 3:
0 compile 0x000000010044a824 0x100428000 + 141348
1 compile 0x000000010044a004 0x100428000 + 139268

Thread 4:
0 libsystem_kernel.dylib 0x00000001b77f961c 0x1b77d2000 + 161308
1 libsystem_pthread.dylib 0x00000001d2e262fc 0x1d2e23000 + 13052
2 compile 0x00000001004918b8 0x100428000 + 432312
3 compile 0x00000001004906f8 0x100428000 + 427768
4 ??? 0x00000000d65f03c0 0 + 3596551104

Thread 5:
0 libsystem_kernel.dylib 0x00000001b77f961c 0x1b77d2000 + 161308
1 libsystem_pthread.dylib 0x00000001d2e262fc 0x1d2e23000 + 13052
2 compile 0x00000001004918b8 0x100428000 + 432312
3 compile 0x00000001004906f8 0x100428000 + 427768
4 ??? 0x00000000d65f03c0 0 + 3596551104

Thread 0 crashed with ARM Thread State (64-bit):
x0: 0x00000001303a2000 x1: 0x000000015ed0f7e0 x2: 0x000000015a5e9880 x3: 0x000000007ffffffd
x4: 0x0000000000000382 x5: 0x0000000000000f99 x6: 0x0000000000000fb4 x7: 0x00000001606d0000
x8: 0x0000000000000381 x9: 0x00000000000000f1 x10: 0x00000001607b93ff x11: 0x0000000000000058
x12: 0x0000000000000001 x13: 0x0000000100ab9cf2 x14: 0x0000000000000000 x15: 0x00000000000000f1
x16: 0x000000015ed0f948 x17: 0x0000000000000000 x18: 0x0000000000000000 x19: 0x0000000000000010
x20: 0x0000000000000008 x21: 0x0000000000000003 x22: 0x000000000000000a x23: 0x0000000000000008
x24: 0x0000000000000000 x25: 0x0000000000000018 x26: 0x000000015ed0f7f8 x27: 0x0000000000000000
x28: 0x0000000130000180 fp: 0x000000015ed0f618 lr: 0x000000010054d5ec
sp: 0x000000015ed0f620 pc: 0x0000000100558898 cpsr: 0x20000000
esr: 0x9200004f (Data Abort) byte write Permission fault

Binary Images:
0x100428000 - 0x100ac7fff compile arm64 <709e8d8bfdc9330cbc0760b0cd693652> /var/tmp/workdir-host-ios-arm64-corellium-ios/go/pkg/tool/ios_arm64/compile
0x101144000 - 0x10114bfff libsubstrate.dylib arm64 /usr/lib/libsubstrate.dylib
0x101154000 - 0x10115bfff pspawn_payload.dylib arm64 <010fa335cb823edda8339536e71ec9d9> /usr/lib/pspawn_payload.dylib
0x101170000 - 0x10117bfff substitute-loader.dylib arm64 /usr/lib/substitute-loader.dylib
0x10118c000 - 0x1011a7fff libsubstitute.0.dylib arm64 <560ee8b4dbb43c5c98ad5eb56be360b6> /usr/lib/libsubstitute.0.dylib
0x101468000 - 0x1014d3fff dyld arm64 /usr/lib/dyld
0x18c3a5000 - 0x18c423fff libdispatch.dylib arm64 <6c143c51288f3cd084dd17b9a0aaee62> /usr/lib/system/libdispatch.dylib
0x18c424000 - 0x18c45afff libdyld.dylib arm64 /usr/lib/system/libdyld.dylib
0x18c45b000 - 0x18c6b2fff libicucore.A.dylib arm64 <9fc7dac3deed3a479abbe36867045ba5> /usr/lib/libicucore.A.dylib
0x18c6b3000 - 0x18ca5cfff CoreFoundation arm64 <96f8386dd88a3c89a323a17975c3317f> /System/Library/Frameworks/CoreFoundation.framework/CoreFoundation
0x18cc40000 - 0x18ccb8fff SystemConfiguration arm64 <75fec68015053f2e84dc4a0a8586c617> /System/Library/Frameworks/SystemConfiguration.framework/SystemConfiguration
0x18cda4000 - 0x18d225fff CFNetwork arm64 /System/Library/Frameworks/CFNetwork.framework/CFNetwork
0x18d226000 - 0x18d8ebfff libnetwork.dylib arm64 /usr/lib/libnetwork.dylib
0x18d95d000 - 0x18dbf7fff Foundation arm64 /System/Library/Frameworks/Foundation.framework/Foundation
0x193744000 - 0x193881fff Security arm64 <149f8d12822a31a198bf48ce690f678d> /System/Library/Frameworks/Security.framework/Security
0x1953e0000 - 0x195405fff libsystem_info.dylib arm64 <3536d46d36cd3af4b92c67e4bdebd8c6> /usr/lib/system/libsystem_info.dylib
0x195406000 - 0x195482fff libsystem_c.dylib arm64 /usr/lib/system/libsystem_c.dylib
0x196d64000 - 0x196e0afff IOKit arm64 /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit
0x19ad0f000 - 0x19ad6cfff libMobileGestalt.dylib arm64 <448540b0eca03930b98b2d18101114d6> /usr/lib/libMobileGestalt.dylib
0x19b1dc000 - 0x19b1fefff libsystem_malloc.dylib arm64 <9f2d3d4683d9358f972c1cf61c073a31> /usr/lib/system/libsystem_malloc.dylib
0x19d169000 - 0x19d178fff libsystem_networkextension.dylib arm64 <0e55773557ba3611994ce782f3cbfe75> /usr/lib/system/libsystem_networkextension.dylib
0x1a0178000 - 0x1a01adfff libobjc.A.dylib arm64 <7cad222912fa3d6c82005306fb157fa4> /usr/lib/libobjc.A.dylib
0x1a020e000 - 0x1a0267fff libc++.1.dylib arm64 <88496d66c6683f1797c5d0c91e8bee42> /usr/lib/libc++.1.dylib
0x1a0268000 - 0x1a0280fff libc++abi.dylib arm64 <464199d1cdfe3663bedfcbc1b5f19491> /usr/lib/libc++abi.dylib
0x1a03eb000 - 0x1a042bfff CoreAutoLayout arm64 <83a9b00eb095347aa9a2fd147deb1d75> /System/Library/PrivateFrameworks/CoreAutoLayout.framework/CoreAutoLayout
0x1a1dc6000 - 0x1a1ddcfff libsystem_trace.dylib arm64 <4ed5b6b480e9322784b590bc92570931> /usr/lib/system/libsystem_trace.dylib
0x1a4d5d000 - 0x1a4edffff libsqlite3.dylib arm64 /usr/lib/libsqlite3.dylib
0x1a5a73000 - 0x1a5a7dfff libsystem_notify.dylib arm64 <4dbd2e3e6bc43d1388d0e01ba0cc4e07> /usr/lib/system/libsystem_notify.dylib
0x1a5b28000 - 0x1a5b97fff libcorecrypto.dylib arm64 <071f7a201cf6324783d7d4e309a64c16> /usr/lib/system/libcorecrypto.dylib
0x1a5c8b000 - 0x1a5ca1fff libsystem_asl.dylib arm64 <441adb81097f31d6a85d380a72993d66> /usr/lib/system/libsystem_asl.dylib
0x1a987e000 - 0x1a9888fff IOMobileFramebuffer arm64 <674f742e890231f3b8999fac2c9eb8fa> /System/Library/PrivateFrameworks/IOMobileFramebuffer.framework/IOMobileFramebuffer
0x1ac45b000 - 0x1ac46cfff IOSurface arm64 <19dd2ad445e03197a5c453b357f361a0> /System/Library/Frameworks/IOSurface.framework/IOSurface
0x1aca1b000 - 0x1aca22fff libsystem_symptoms.dylib arm64 /usr/lib/system/libsystem_symptoms.dylib
0x1b77d2000 - 0x1b7802fff libsystem_kernel.dylib arm64 <8bcb6ff951623657a2e25ce5704353e6> /usr/lib/system/libsystem_kernel.dylib
0x1ba248000 - 0x1ba249fff libSystem.B.dylib arm64 <5f81896adca13ef7999f6c5da89a750b> /usr/lib/libSystem.B.dylib
0x1ba68a000 - 0x1ba6f8fff libarchive.2.dylib arm64 /usr/lib/libarchive.2.dylib
0x1cae1e000 - 0x1cae1efff SoftLinking arm64 <2a4d987221d13057a2e4287595be79b5> /System/Library/PrivateFrameworks/SoftLinking.framework/SoftLinking
0x1d1705000 - 0x1d1737fff libCRFSuite.dylib arm64 /usr/lib/libCRFSuite.dylib
0x1d1deb000 - 0x1d1e01fff libapple_nghttp2.dylib arm64 <741cbe1e00f93a20b845d77885621920> /usr/lib/libapple_nghttp2.dylib
0x1d1f23000 - 0x1d1f33fff libbsm.0.dylib arm64 <62262c68bf203584b2ea3889b8647e16> /usr/lib/libbsm.0.dylib
0x1d1f34000 - 0x1d1f40fff libbz2.1.0.dylib arm64 /usr/lib/libbz2.1.0.dylib
0x1d1f41000 - 0x1d1f41fff libcharset.1.dylib arm64 <72b0c36f725f38fea3579a515aaf64b9> /usr/lib/libcharset.1.dylib
0x1d1f54000 - 0x1d1f6bfff libcompression.dylib arm64 <36dd56ca87a33b8bbc2e9ee086e82284> /usr/lib/libcompression.dylib
0x1d1f6c000 - 0x1d1f81fff libcoretls.dylib arm64 <31987a4e4ec23c89952efad70a0d71d5> /usr/lib/libcoretls.dylib
0x1d1f82000 - 0x1d1f83fff libcoretls_cfhelpers.dylib arm64 <56fa9fe8ebb83d17872877e4c5cd0848> /usr/lib/libcoretls_cfhelpers.dylib
0x1d1fae000 - 0x1d1fb5fff libdns_services.dylib arm64 <8d3832488b4333b492251a634982e109> /usr/lib/libdns_services.dylib
0x1d1fd3000 - 0x1d1fd3fff libenergytrace.dylib arm64 /usr/lib/libenergytrace.dylib
0x1d2022000 - 0x1d2113fff libiconv.2.dylib arm64 <6bf1c6e1e4b93cc3928eaa07c4ecdfc5> /usr/lib/libiconv.2.dylib
0x1d2131000 - 0x1d2132fff liblangid.dylib arm64 <982686583ed13ebaa2cc14345035abe9> /usr/lib/liblangid.dylib
0x1d213f000 - 0x1d2157fff liblzma.5.dylib arm64 <60e6e4ad8b823583b81e7e39df26637f> /usr/lib/liblzma.5.dylib
0x1d27cb000 - 0x1d27fefff libpcap.A.dylib arm64 <0d7a2422cdb838ce9604933d6ced3742> /usr/lib/libpcap.A.dylib
0x1d295b000 - 0x1d2a40fff libxml2.2.dylib arm64 <3102a95fbfd13d2292ed7d4f31ce994f> /usr/lib/libxml2.2.dylib
0x1d2a6e000 - 0x1d2a7ffff libz.1.dylib arm64 /usr/lib/libz.1.dylib
0x1d2cac000 - 0x1d2cb1fff libcache.dylib arm64 <93f46de334c83b548e87b685e63149e7> /usr/lib/system/libcache.dylib
0x1d2cb2000 - 0x1d2cbefff libcommonCrypto.dylib arm64 <163c756fa4413bb7b2ca067463d98526> /usr/lib/system/libcommonCrypto.dylib
0x1d2cbf000 - 0x1d2cc3fff libcompiler_rt.dylib arm64 <6749df438089311b939a87f9ca479efd> /usr/lib/system/libcompiler_rt.dylib
0x1d2cc4000 - 0x1d2cccfff libcopyfile.dylib arm64 <5ea5d77aebc930459a3ccf1838e73c6b> /usr/lib/system/libcopyfile.dylib
0x1d2daa000 - 0x1d2daafff liblaunch.dylib arm64 /usr/lib/system/liblaunch.dylib
0x1d2dab000 - 0x1d2db0fff libmacho.dylib arm64 <863174fc64663104aca38d1d4a82d075> /usr/lib/system/libmacho.dylib
0x1d2db1000 - 0x1d2db3fff libremovefile.dylib arm64 /usr/lib/system/libremovefile.dylib
0x1d2db4000 - 0x1d2db5fff libsystem_blocks.dylib arm64 /usr/lib/system/libsystem_blocks.dylib
0x1d2db6000 - 0x1d2db8fff libsystem_collections.dylib arm64 <828126f3e6583da2aceac728678588de> /usr/lib/system/libsystem_collections.dylib
0x1d2db9000 - 0x1d2dbdfff libsystem_configuration.dylib arm64 <27db1d5d607530bf888edca2c13e80ef> /usr/lib/system/libsystem_configuration.dylib
0x1d2dbe000 - 0x1d2dd0fff libsystem_containermanager.dylib arm64 /usr/lib/system/libsystem_containermanager.dylib
0x1d2dd1000 - 0x1d2dd2fff libsystem_coreservices.dylib arm64 /usr/lib/system/libsystem_coreservices.dylib
0x1d2dd3000 - 0x1d2ddcfff libsystem_darwin.dylib arm64 /usr/lib/system/libsystem_darwin.dylib
0x1d2ddd000 - 0x1d2de5fff libsystem_dnssd.dylib arm64 /usr/lib/system/libsystem_dnssd.dylib
0x1d2de6000 - 0x1d2de8fff libsystem_featureflags.dylib arm64 <1e7cd17eb5283b85bf4a27e1aa8dfa3e> /usr/lib/system/libsystem_featureflags.dylib
0x1d2de9000 - 0x1d2e16fff libsystem_m.dylib arm64 /usr/lib/system/libsystem_m.dylib
0x1d2e17000 - 0x1d2e21fff libsystem_platform.dylib arm64 /usr/lib/system/libsystem_platform.dylib
0x1d2e22000 - 0x1d2e22fff libsystem_product_info_filter.dylib arm64 <61b5d8563fb33fe9b30540a45f659732> /usr/lib/system/libsystem_product_info_filter.dylib
0x1d2e23000 - 0x1d2e33fff libsystem_pthread.dylib arm64 <93c7f0dc89f13162a1196fc88028d027> /usr/lib/system/libsystem_pthread.dylib
0x1d2e34000 - 0x1d2e37fff libsystem_sandbox.dylib arm64 <176b1087aaad36258e7ffc497673706c> /usr/lib/system/libsystem_sandbox.dylib
0x1d2e38000 - 0x1d2e41fff libunwind.dylib arm64 /usr/lib/system/libunwind.dylib
0x1d2e42000 - 0x1d2e74fff libxpc.dylib arm64 /usr/lib/system/libxpc.dylib

EOF

{"app_name":"compile","timestamp":"2021-06-21 10:41:21.00 -0700","app_version":"","slice_uuid":"4e440b1b-ddbb-3cb9-9f4d-384ffadbf0bc","build_version":"","platform":2,"share_with_app_devs":0,"is_first_party":1,"bug_type":"109","os_version":"iPhone OS 14.2 (18B92)","incident_id":"8C91A684-E77A-4476-8930-BC84B04EEDB9","name":"compile"} Incident Identifier: 8C91A684-E77A-4476-8930-BC84B04EEDB9 CrashReporter Key: f09d9091c97b4e4f6bb695146954c56a7f0495a7 Hardware Model: iPhone8,4 Process: compile [91896] Path: /private/var/tmp/workdir-host-ios-arm64-corellium-ios/go/pkg/tool/ios_arm64/compile Identifier: compile Version: ??? Code Type: ARM-64 (Native) Role: Unspecified Parent Process: go_bootstrap [91607] Coalition: org.golang.builder [157]

Date/Time: 2021-06-21 10:41:21.7868 -0700
Launch Time: 2021-06-21 10:41:05.0928 -0700
OS Version: iPhone OS 14.2 (18B92)
Release Type: User
Baseband Version:
Report Version: 104

Exception Type: EXC_CRASH (SIGILL)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Exception Note: EXC_CORPSE_NOTIFY
Termination Signal: Illegal instruction: 4
Termination Reason: Namespace SIGNAL, Code 0x4
Terminating Process: compile [91896]
Triggered by Thread: 0

Thread 0 name: Dispatch queue: com.apple.main-thread
Thread 0 Crashed:
0 libsystem_kernel.dylib 0x00000001b77f961c 0x1b77d2000 + 161308
1 libsystem_pthread.dylib 0x00000001d2e262fc 0x1d2e23000 + 13052
2 compile 0x0000000100e8bf48 0x100e1c000 + 458568
3 compile 0x0000000100e8acb8 0x100e1c000 + 453816
4 ??? 0x00000000d65f03c0 0 + 3596551104

Thread 1:
0 libsystem_pthread.dylib 0x00000001d2e3186c 0x1d2e23000 + 59500

Thread 2:
0 libsystem_kernel.dylib 0x00000001b77f9bf0 0x1b77d2000 + 162800
1 libsystem_c.dylib 0x00000001954766d4 0x195406000 + 460500
2 libsystem_c.dylib 0x00000001954765ac 0x195406000 + 460204
3 compile 0x0000000100e8bbd4 0x100e1c000 + 457684
4 compile 0x0000000100e8acb8 0x100e1c000 + 453816

Thread 3:
0 compile 0x000000010117b614 0x100e1c000 + 3536404
1 compile 0x000000010117a928 0x100e1c000 + 3533096

Thread 4:
0 libsystem_kernel.dylib 0x00000001b77f961c 0x1b77d2000 + 161308
1 libsystem_pthread.dylib 0x00000001d2e262fc 0x1d2e23000 + 13052
2 compile 0x0000000100e8bf48 0x100e1c000 + 458568
3 compile 0x0000000100e8acb8 0x100e1c000 + 453816
4 ??? 0x00000000d65f03c0 0 + 3596551104

Thread 5:
0 libsystem_kernel.dylib 0x00000001b77f961c 0x1b77d2000 + 161308
1 libsystem_pthread.dylib 0x00000001d2e262fc 0x1d2e23000 + 13052
2 compile 0x0000000100e8bf48 0x100e1c000 + 458568
3 compile 0x0000000100e8acb8 0x100e1c000 + 453816
4 ??? 0x00000000d65f03c0 0 + 3596551104

Thread 6:
0 libsystem_kernel.dylib 0x00000001b77f961c 0x1b77d2000 + 161308
1 libsystem_pthread.dylib 0x00000001d2e262fc 0x1d2e23000 + 13052
2 compile 0x0000000100e8bf48 0x100e1c000 + 458568
3 compile 0x0000000100e8acb8 0x100e1c000 + 453816
4 ??? 0x00000000d65f03c0 0 + 3596551104

Thread 0 crashed with ARM Thread State (64-bit):
x0: 0x0000000000000104 x1: 0x0000000000000000 x2: 0x0000000000000200 x3: 0x0000000000000000
x4: 0x0000000000000000 x5: 0x00000000000000a0 x6: 0x0000000000000000 x7: 0x0000000000000000
x8: 0x000000016efe2c38 x9: 0x0000000000000201 x10: 0x000000010188d078 x11: 0x0000000000000002
x12: 0x00000000000020a0 x13: 0x0000000000000000 x14: 0x0000000000000000 x15: 0x0000000000000000
x16: 0x0000000000000131 x17: 0x0000000000000000 x18: 0x0000000000000000 x19: 0x000000010188d060
x20: 0x000000010188d0a0 x21: 0x000000010232f960 x22: 0x0000000000000000 x23: 0x0000000000000000
x24: 0x0000000000000200 x25: 0x0000000000000201 x26: 0x0000000000000300 x27: 0xffffffffffffffe0
x28: 0x000000010188c4e0 fp: 0x000000016efe2cb0 lr: 0x00000001d2e262fc
sp: 0x000000016efe2c20 pc: 0x00000001b77f961c cpsr: 0x60000000
esr: 0x56000080 Address size fault

Binary Images:
0x100e1c000 - 0x1017c7fff compile arm64 <4e440b1bddbb3cb99f4d384ffadbf0bc> /var/tmp/workdir-host-ios-arm64-corellium-ios/go/pkg/tool/ios_arm64/compile
0x101f74000 - 0x101f7bfff pspawn_payload.dylib arm64 <010fa335cb823edda8339536e71ec9d9> /usr/lib/pspawn_payload.dylib
0x101f90000 - 0x101f9bfff substitute-loader.dylib arm64 /usr/lib/substitute-loader.dylib
0x101fac000 - 0x101fb3fff libsubstrate.dylib arm64 /usr/lib/libsubstrate.dylib
0x101fbc000 - 0x101fd7fff libsubstitute.0.dylib arm64 <560ee8b4dbb43c5c98ad5eb56be360b6> /usr/lib/libsubstitute.0.dylib
0x102288000 - 0x1022f3fff dyld arm64 /usr/lib/dyld
0x18c3a5000 - 0x18c423fff libdispatch.dylib arm64 <6c143c51288f3cd084dd17b9a0aaee62> /usr/lib/system/libdispatch.dylib
0x18c424000 - 0x18c45afff libdyld.dylib arm64 /usr/lib/system/libdyld.dylib
0x18c45b000 - 0x18c6b2fff libicucore.A.dylib arm64 <9fc7dac3deed3a479abbe36867045ba5> /usr/lib/libicucore.A.dylib
0x18c6b3000 - 0x18ca5cfff CoreFoundation arm64 <96f8386dd88a3c89a323a17975c3317f> /System/Library/Frameworks/CoreFoundation.framework/CoreFoundation
0x18cc40000 - 0x18ccb8fff SystemConfiguration arm64 <75fec68015053f2e84dc4a0a8586c617> /System/Library/Frameworks/SystemConfiguration.framework/SystemConfiguration
0x18cda4000 - 0x18d225fff CFNetwork arm64 /System/Library/Frameworks/CFNetwork.framework/CFNetwork
0x18d226000 - 0x18d8ebfff libnetwork.dylib arm64 /usr/lib/libnetwork.dylib
0x18d95d000 - 0x18dbf7fff Foundation arm64 /System/Library/Frameworks/Foundation.framework/Foundation
0x193744000 - 0x193881fff Security arm64 <149f8d12822a31a198bf48ce690f678d> /System/Library/Frameworks/Security.framework/Security
0x1953e0000 - 0x195405fff libsystem_info.dylib arm64 <3536d46d36cd3af4b92c67e4bdebd8c6> /usr/lib/system/libsystem_info.dylib
0x195406000 - 0x195482fff libsystem_c.dylib arm64 /usr/lib/system/libsystem_c.dylib
0x196d64000 - 0x196e0afff IOKit arm64 /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit
0x19ad0f000 - 0x19ad6cfff libMobileGestalt.dylib arm64 <448540b0eca03930b98b2d18101114d6> /usr/lib/libMobileGestalt.dylib
0x19b1dc000 - 0x19b1fefff libsystem_malloc.dylib arm64 <9f2d3d4683d9358f972c1cf61c073a31> /usr/lib/system/libsystem_malloc.dylib
0x19d169000 - 0x19d178fff libsystem_networkextension.dylib arm64 <0e55773557ba3611994ce782f3cbfe75> /usr/lib/system/libsystem_networkextension.dylib
0x1a0178000 - 0x1a01adfff libobjc.A.dylib arm64 <7cad222912fa3d6c82005306fb157fa4> /usr/lib/libobjc.A.dylib
0x1a020e000 - 0x1a0267fff libc++.1.dylib arm64 <88496d66c6683f1797c5d0c91e8bee42> /usr/lib/libc++.1.dylib
0x1a0268000 - 0x1a0280fff libc++abi.dylib arm64 <464199d1cdfe3663bedfcbc1b5f19491> /usr/lib/libc++abi.dylib
0x1a03eb000 - 0x1a042bfff CoreAutoLayout arm64 <83a9b00eb095347aa9a2fd147deb1d75> /System/Library/PrivateFrameworks/CoreAutoLayout.framework/CoreAutoLayout
0x1a1dc6000 - 0x1a1ddcfff libsystem_trace.dylib arm64 <4ed5b6b480e9322784b590bc92570931> /usr/lib/system/libsystem_trace.dylib
0x1a4d5d000 - 0x1a4edffff libsqlite3.dylib arm64 /usr/lib/libsqlite3.dylib
0x1a5a73000 - 0x1a5a7dfff libsystem_notify.dylib arm64 <4dbd2e3e6bc43d1388d0e01ba0cc4e07> /usr/lib/system/libsystem_notify.dylib
0x1a5b28000 - 0x1a5b97fff libcorecrypto.dylib arm64 <071f7a201cf6324783d7d4e309a64c16> /usr/lib/system/libcorecrypto.dylib
0x1a5c8b000 - 0x1a5ca1fff libsystem_asl.dylib arm64 <441adb81097f31d6a85d380a72993d66> /usr/lib/system/libsystem_asl.dylib
0x1a987e000 - 0x1a9888fff IOMobileFramebuffer arm64 <674f742e890231f3b8999fac2c9eb8fa> /System/Library/PrivateFrameworks/IOMobileFramebuffer.framework/IOMobileFramebuffer
0x1ac45b000 - 0x1ac46cfff IOSurface arm64 <19dd2ad445e03197a5c453b357f361a0> /System/Library/Frameworks/IOSurface.framework/IOSurface
0x1aca1b000 - 0x1aca22fff libsystem_symptoms.dylib arm64 /usr/lib/system/libsystem_symptoms.dylib
0x1b77d2000 - 0x1b7802fff libsystem_kernel.dylib arm64 <8bcb6ff951623657a2e25ce5704353e6> /usr/lib/system/libsystem_kernel.dylib
0x1ba248000 - 0x1ba249fff libSystem.B.dylib arm64 <5f81896adca13ef7999f6c5da89a750b> /usr/lib/libSystem.B.dylib
0x1ba68a000 - 0x1ba6f8fff libarchive.2.dylib arm64 /usr/lib/libarchive.2.dylib
0x1cae1e000 - 0x1cae1efff SoftLinking arm64 <2a4d987221d13057a2e4287595be79b5> /System/Library/PrivateFrameworks/SoftLinking.framework/SoftLinking
0x1d1705000 - 0x1d1737fff libCRFSuite.dylib arm64 /usr/lib/libCRFSuite.dylib
0x1d1deb000 - 0x1d1e01fff libapple_nghttp2.dylib arm64 <741cbe1e00f93a20b845d77885621920> /usr/lib/libapple_nghttp2.dylib
0x1d1f23000 - 0x1d1f33fff libbsm.0.dylib arm64 <62262c68bf203584b2ea3889b8647e16> /usr/lib/libbsm.0.dylib
0x1d1f34000 - 0x1d1f40fff libbz2.1.0.dylib arm64 /usr/lib/libbz2.1.0.dylib
0x1d1f41000 - 0x1d1f41fff libcharset.1.dylib arm64 <72b0c36f725f38fea3579a515aaf64b9> /usr/lib/libcharset.1.dylib
0x1d1f54000 - 0x1d1f6bfff libcompression.dylib arm64 <36dd56ca87a33b8bbc2e9ee086e82284> /usr/lib/libcompression.dylib
0x1d1f6c000 - 0x1d1f81fff libcoretls.dylib arm64 <31987a4e4ec23c89952efad70a0d71d5> /usr/lib/libcoretls.dylib
0x1d1f82000 - 0x1d1f83fff libcoretls_cfhelpers.dylib arm64 <56fa9fe8ebb83d17872877e4c5cd0848> /usr/lib/libcoretls_cfhelpers.dylib
0x1d1fae000 - 0x1d1fb5fff libdns_services.dylib arm64 <8d3832488b4333b492251a634982e109> /usr/lib/libdns_services.dylib
0x1d1fd3000 - 0x1d1fd3fff libenergytrace.dylib arm64 /usr/lib/libenergytrace.dylib
0x1d2022000 - 0x1d2113fff libiconv.2.dylib arm64 <6bf1c6e1e4b93cc3928eaa07c4ecdfc5> /usr/lib/libiconv.2.dylib
0x1d2131000 - 0x1d2132fff liblangid.dylib arm64 <982686583ed13ebaa2cc14345035abe9> /usr/lib/liblangid.dylib
0x1d213f000 - 0x1d2157fff liblzma.5.dylib arm64 <60e6e4ad8b823583b81e7e39df26637f> /usr/lib/liblzma.5.dylib
0x1d27cb000 - 0x1d27fefff libpcap.A.dylib arm64 <0d7a2422cdb838ce9604933d6ced3742> /usr/lib/libpcap.A.dylib
0x1d295b000 - 0x1d2a40fff libxml2.2.dylib arm64 <3102a95fbfd13d2292ed7d4f31ce994f> /usr/lib/libxml2.2.dylib
0x1d2a6e000 - 0x1d2a7ffff libz.1.dylib arm64 /usr/lib/libz.1.dylib
0x1d2cac000 - 0x1d2cb1fff libcache.dylib arm64 <93f46de334c83b548e87b685e63149e7> /usr/lib/system/libcache.dylib
0x1d2cb2000 - 0x1d2cbefff libcommonCrypto.dylib arm64 <163c756fa4413bb7b2ca067463d98526> /usr/lib/system/libcommonCrypto.dylib
0x1d2cbf000 - 0x1d2cc3fff libcompiler_rt.dylib arm64 <6749df438089311b939a87f9ca479efd> /usr/lib/system/libcompiler_rt.dylib
0x1d2cc4000 - 0x1d2cccfff libcopyfile.dylib arm64 <5ea5d77aebc930459a3ccf1838e73c6b> /usr/lib/system/libcopyfile.dylib
0x1d2daa000 - 0x1d2daafff liblaunch.dylib arm64 /usr/lib/system/liblaunch.dylib
0x1d2dab000 - 0x1d2db0fff libmacho.dylib arm64 <863174fc64663104aca38d1d4a82d075> /usr/lib/system/libmacho.dylib
0x1d2db1000 - 0x1d2db3fff libremovefile.dylib arm64 /usr/lib/system/libremovefile.dylib
0x1d2db4000 - 0x1d2db5fff libsystem_blocks.dylib arm64 /usr/lib/system/libsystem_blocks.dylib
0x1d2db6000 - 0x1d2db8fff libsystem_collections.dylib arm64 <828126f3e6583da2aceac728678588de> /usr/lib/system/libsystem_collections.dylib
0x1d2db9000 - 0x1d2dbdfff libsystem_configuration.dylib arm64 <27db1d5d607530bf888edca2c13e80ef> /usr/lib/system/libsystem_configuration.dylib
0x1d2dbe000 - 0x1d2dd0fff libsystem_containermanager.dylib arm64 /usr/lib/system/libsystem_containermanager.dylib
0x1d2dd1000 - 0x1d2dd2fff libsystem_coreservices.dylib arm64 /usr/lib/system/libsystem_coreservices.dylib
0x1d2dd3000 - 0x1d2ddcfff libsystem_darwin.dylib arm64 /usr/lib/system/libsystem_darwin.dylib
0x1d2ddd000 - 0x1d2de5fff libsystem_dnssd.dylib arm64 /usr/lib/system/libsystem_dnssd.dylib
0x1d2de6000 - 0x1d2de8fff libsystem_featureflags.dylib arm64 <1e7cd17eb5283b85bf4a27e1aa8dfa3e> /usr/lib/system/libsystem_featureflags.dylib
0x1d2de9000 - 0x1d2e16fff libsystem_m.dylib arm64 /usr/lib/system/libsystem_m.dylib
0x1d2e17000 - 0x1d2e21fff libsystem_platform.dylib arm64 /usr/lib/system/libsystem_platform.dylib
0x1d2e22000 - 0x1d2e22fff libsystem_product_info_filter.dylib arm64 <61b5d8563fb33fe9b30540a45f659732> /usr/lib/system/libsystem_product_info_filter.dylib
0x1d2e23000 - 0x1d2e33fff libsystem_pthread.dylib arm64 <93c7f0dc89f13162a1196fc88028d027> /usr/lib/system/libsystem_pthread.dylib
0x1d2e34000 - 0x1d2e37fff libsystem_sandbox.dylib arm64 <176b1087aaad36258e7ffc497673706c> /usr/lib/system/libsystem_sandbox.dylib
0x1d2e38000 - 0x1d2e41fff libunwind.dylib arm64 /usr/lib/system/libunwind.dylib
0x1d2e42000 - 0x1d2e74fff libxpc.dylib arm64 /usr/lib/system/libxpc.dylib

EOF

@cherrymui
Copy link
Contributor

@cherrymui cherrymui commented Jun 24, 2021

Thanks. The crash reports look a lot like the ones from #42774. Elias, do you confirm that CL https://go-review.googlesource.com/c/go/+/279489 doesn't help? Maybe there is other possibility for the kernel to fail to deliver signal...

What is the builder setup? I think it is not a real iOS device, but some kind of VM? Does the SIGILL happen on real iOS devices? Thanks.

@eliasnaur
Copy link
Contributor Author

@eliasnaur eliasnaur commented Jun 25, 2021

I tried again to be sure:

git log -2
commit e50583caf50b25f9f07ce9cc67671a877963ad5b (HEAD -> master)
Author: Cherry Zhang <cherryyz@google.com>
Date:   Mon Dec 21 17:46:16 2020 -0500

    runtime: use sigaltstack+mlock on iOS

    See if locking the signal stack helps reducing SIGILLs.

    DO NOT SUBMIT. This is an experiment. This will break older
    version of iOS (which doesn't support sigaltstack).

    For #35851.

    Change-Id: I4ac32923632f581e453d4761876fc757fb6ffe29

commit 37f9a8f69d6299783eac8848d87e27eb563500ac (origin/master, origin/HEAD)
Author: Rob Findley <rfindley@google.com>
Date:   Thu Jun 24 11:01:49 2021 -0400

    go/types: fix a bug in package qualification logic

    CL 313035 had a bug, initializing pkgPathMap by walking the imported
    package being considered rather than check.pkg.

    Fix this, and enhance our tests to exercise this bug as well as other
    edge cases.

    Also fix error assertions in issues.src to not use quotation marks
    inside the error regexp. The check tests only matched the error regexp
    up to the first quotation mark.

    Fixes #46905

    Change-Id: I6aa8eae4bec6495006a5c03fc063db0d66b44cd6
    Reviewed-on: https://go-review.googlesource.com/c/go/+/330629
    Trust: Robert Findley <rfindley@google.com>
    Trust: Robert Griesemer <gri@golang.org>
    Run-TryBot: Robert Findley <rfindley@google.com>
    TryBot-Result: Go Bot <gobot@golang.org>
    Reviewed-by: Robert Griesemer <gri@golang.org>
$ while true; do CC=$HOME/bin/clangwrap PATH=$PATH:$HOME/bin ./all.bash ; sleep 1; done;
Building Go cmd/dist using /var/root/go-ios-arm64-bootstrap. (devel +694025e74f Tue Oct 6 01:14:39 2020 +0000 ios/arm64)
Building Go toolchain1 using /var/root/go-ios-arm64-bootstrap.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
warning: unable to find runtime/cgo.a
Building Go toolchain2 using go_bootstrap and Go toolchain1.
go build cmd/compile/internal/ssa: /var/root/goroot/pkg/tool/ios_arm64/compile: signal: illegal instruction
go tool dist: FAILED: /var/root/goroot/pkg/tool/ios_arm64/go_bootstrap install -gcflags=all= -ldflags=all= -i cmd/asm cmd/cgo cmd/compile cmd/link: exit status 1
Building Go cmd/dist using /var/root/go-ios-arm64-bootstrap. (devel +694025e74f Tue Oct 6 01:14:39 2020 +0000 ios/arm64)
Building Go toolchain1 using /var/root/go-ios-arm64-bootstrap.

Crash report:

$ cat /var/mobile/Library/Logs/CrashReporter/compile-2021-06-25-060613.ips
{"app_name":"compile","timestamp":"2021-06-25 06:06:13.00 -0700","app_version":"","slice_uuid":"cd51514b-ed86-31d8-aa75-c4b00aa94c5d","build_version":"","platform":2,"share_with_app_devs":0,"is_first_party":1,"bug_type":"109","os_version":"iPhone OS 14.2 (18B92)","incident_id":"75BDAC6F-8ADD-4DFF-B13C-48283FCAF48E","name":"compile"}
Incident Identifier: 75BDAC6F-8ADD-4DFF-B13C-48283FCAF48E
CrashReporter Key:   f09d9091c97b4e4f6bb695146954c56a7f0495a7
Hardware Model:      iPhone8,4
Process:             compile [6249]
Path:                /private/var/root/goroot/pkg/tool/ios_arm64/compile
Identifier:          compile
Version:             ???
Code Type:           ARM-64 (Native)
Role:                Unspecified
Parent Process:      go_bootstrap [5667]
Coalition:           com.openssh.sshd.12FDA541-4545-4AF8-976B-7B801CB9D777 [455]


Date/Time:           2021-06-25 06:06:13.6773 -0700
Launch Time:         2021-06-25 06:05:39.0166 -0700
OS Version:          iPhone OS 14.2 (18B92)
Release Type:        User
Baseband Version:
Report Version:      104

Exception Type:  EXC_CRASH (SIGILL)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Exception Note:  EXC_CORPSE_NOTIFY
Termination Signal: Illegal instruction: 4
Termination Reason: Namespace SIGNAL, Code 0x4
Terminating Process: compile [6249]
Triggered by Thread:  0

Thread 0 name:  Dispatch queue: com.apple.main-thread
Thread 0 Crashed:
0   libsystem_kernel.dylib        	0x00000001bac8d61c 0x1bac66000 + 161308
1   libsystem_pthread.dylib       	0x00000001d62ba2fc 0x1d62b7000 + 13052
2   compile                       	0x0000000104b79888 0x104b10000 + 432264
3   compile                       	0x0000000104b786c8 0x104b10000 + 427720
4   ???                           	0x00000000d65f03c0 0 + 3596551104

Thread 1:
0   libsystem_pthread.dylib       	0x00000001d62c586c 0x1d62b7000 + 59500

Thread 2:
0   libsystem_kernel.dylib        	0x00000001bac8dbf0 0x1bac66000 + 162800
1   libsystem_c.dylib             	0x000000019890a6d4 0x19889a000 + 460500
2   libsystem_c.dylib             	0x000000019890a5ac 0x19889a000 + 460204
3   compile                       	0x0000000104b79514 0x104b10000 + 431380
4   compile                       	0x0000000104b786c8 0x104b10000 + 427720

Thread 3:
0   compile                       	0x0000000104b3559c 0x104b10000 + 152988
1   compile                       	0x0000000104b3558c 0x104b10000 + 152972

Thread 4:
0   libsystem_kernel.dylib        	0x00000001bac8d61c 0x1bac66000 + 161308
1   libsystem_pthread.dylib       	0x00000001d62ba2fc 0x1d62b7000 + 13052
2   compile                       	0x0000000104b79888 0x104b10000 + 432264
3   compile                       	0x0000000104b786c8 0x104b10000 + 427720
4   ???                           	0x00000000d65f03c0 0 + 3596551104

Thread 5:
0   libsystem_kernel.dylib        	0x00000001bac8d61c 0x1bac66000 + 161308
1   libsystem_pthread.dylib       	0x00000001d62ba2fc 0x1d62b7000 + 13052
2   compile                       	0x0000000104b79888 0x104b10000 + 432264
3   compile                       	0x0000000104b786c8 0x104b10000 + 427720
4   ???                           	0x00000000d65f03c0 0 + 3596551104

Thread 6:
0   compile                       	0x0000000104c9fb78 0x104b10000 + 1637240
1   compile                       	0x0000000104cbb250 0x104b10000 + 1749584

Thread 0 crashed with ARM Thread State (64-bit):
    x0: 0x0000000000000104   x1: 0x0000000000000000   x2: 0x0000000000017600   x3: 0x0000000000000000
    x4: 0x0000000000000000   x5: 0x00000000000000a0   x6: 0x0000000000000000   x7: 0x0000000000000000
    x8: 0x000000016b2ed8e8   x9: 0x0000000000017601  x10: 0x00000001052734d8  x11: 0x0000000000000002
   x12: 0x00000000000020a0  x13: 0x0000000000000000  x14: 0x0000000000000000  x15: 0x0000000000000000
   x16: 0x0000000000000131  x17: 0x0000000000000000  x18: 0x0000000000000000  x19: 0x00000001052734c0
   x20: 0x0000000105273500  x21: 0x0000000105917960  x22: 0x0000000000000000  x23: 0x0000000000000000
   x24: 0x0000000000017600  x25: 0x0000000000017601  x26: 0x0000000000017700  x27: 0x0000000000000040
   x28: 0x0000000105272d40   fp: 0x000000016b2ed960   lr: 0x00000001d62ba2fc
    sp: 0x000000016b2ed8d0   pc: 0x00000001bac8d61c cpsr: 0x60000000
   esr: 0x56000080  Address size fault

Binary Images:
0x104b10000 - 0x1051affff compile arm64  <cd51514bed8631d8aa75c4b00aa94c5d> /var/root/goroot/pkg/tool/ios_arm64/compile
0x105870000 - 0x1058dbfff dyld arm64  <f756a8d55a803b4d996236fe0e7638c2> /usr/lib/dyld
0x18f839000 - 0x18f8b7fff libdispatch.dylib arm64  <6c143c51288f3cd084dd17b9a0aaee62> /usr/lib/system/libdispatch.dylib
0x18f8b8000 - 0x18f8eefff libdyld.dylib arm64  <e31c0887c99930479b80203248ec2396> /usr/lib/system/libdyld.dylib
0x18f8ef000 - 0x18fb46fff libicucore.A.dylib arm64  <9fc7dac3deed3a479abbe36867045ba5> /usr/lib/libicucore.A.dylib
0x18fb47000 - 0x18fef0fff CoreFoundation arm64  <96f8386dd88a3c89a323a17975c3317f> /System/Library/Frameworks/CoreFoundation.framework/CoreFoundation
0x198874000 - 0x198899fff libsystem_info.dylib arm64  <3536d46d36cd3af4b92c67e4bdebd8c6> /usr/lib/system/libsystem_info.dylib
0x19889a000 - 0x198916fff libsystem_c.dylib arm64  <edf3e843a62a3dd092ccebe94d5a83e3> /usr/lib/system/libsystem_c.dylib
0x19e670000 - 0x19e692fff libsystem_malloc.dylib arm64  <9f2d3d4683d9358f972c1cf61c073a31> /usr/lib/system/libsystem_malloc.dylib
0x1a05fd000 - 0x1a060cfff libsystem_networkextension.dylib arm64  <0e55773557ba3611994ce782f3cbfe75> /usr/lib/system/libsystem_networkextension.dylib
0x1a360c000 - 0x1a3641fff libobjc.A.dylib arm64  <7cad222912fa3d6c82005306fb157fa4> /usr/lib/libobjc.A.dylib
0x1a36a2000 - 0x1a36fbfff libc++.1.dylib arm64  <88496d66c6683f1797c5d0c91e8bee42> /usr/lib/libc++.1.dylib
0x1a36fc000 - 0x1a3714fff libc++abi.dylib arm64  <464199d1cdfe3663bedfcbc1b5f19491> /usr/lib/libc++abi.dylib
0x1a525a000 - 0x1a5270fff libsystem_trace.dylib arm64  <4ed5b6b480e9322784b590bc92570931> /usr/lib/system/libsystem_trace.dylib
0x1a8f07000 - 0x1a8f11fff libsystem_notify.dylib arm64  <4dbd2e3e6bc43d1388d0e01ba0cc4e07> /usr/lib/system/libsystem_notify.dylib
0x1a8fbc000 - 0x1a902bfff libcorecrypto.dylib arm64  <071f7a201cf6324783d7d4e309a64c16> /usr/lib/system/libcorecrypto.dylib
0x1a911f000 - 0x1a9135fff libsystem_asl.dylib arm64  <441adb81097f31d6a85d380a72993d66> /usr/lib/system/libsystem_asl.dylib
0x1afeaf000 - 0x1afeb6fff libsystem_symptoms.dylib arm64  <cf41ba3e1ea634e58692d79f7c1b90b8> /usr/lib/system/libsystem_symptoms.dylib
0x1bac66000 - 0x1bac96fff libsystem_kernel.dylib arm64  <8bcb6ff951623657a2e25ce5704353e6> /usr/lib/system/libsystem_kernel.dylib
0x1bd6dc000 - 0x1bd6ddfff libSystem.B.dylib arm64  <5f81896adca13ef7999f6c5da89a750b> /usr/lib/libSystem.B.dylib
0x1d6140000 - 0x1d6145fff libcache.dylib arm64  <93f46de334c83b548e87b685e63149e7> /usr/lib/system/libcache.dylib
0x1d6146000 - 0x1d6152fff libcommonCrypto.dylib arm64  <163c756fa4413bb7b2ca067463d98526> /usr/lib/system/libcommonCrypto.dylib
0x1d6153000 - 0x1d6157fff libcompiler_rt.dylib arm64  <6749df438089311b939a87f9ca479efd> /usr/lib/system/libcompiler_rt.dylib
0x1d6158000 - 0x1d6160fff libcopyfile.dylib arm64  <5ea5d77aebc930459a3ccf1838e73c6b> /usr/lib/system/libcopyfile.dylib
0x1d623e000 - 0x1d623efff liblaunch.dylib arm64  <a0b8f064a9ee310e8b976e27f49a05f2> /usr/lib/system/liblaunch.dylib
0x1d623f000 - 0x1d6244fff libmacho.dylib arm64  <863174fc64663104aca38d1d4a82d075> /usr/lib/system/libmacho.dylib
0x1d6245000 - 0x1d6247fff libremovefile.dylib arm64  <fc34eec2b9013ebbbc31e73f38842b70> /usr/lib/system/libremovefile.dylib
0x1d6248000 - 0x1d6249fff libsystem_blocks.dylib arm64  <bc8afa7d401e35f789e89e7a59aef82b> /usr/lib/system/libsystem_blocks.dylib
0x1d624a000 - 0x1d624cfff libsystem_collections.dylib arm64  <828126f3e6583da2aceac728678588de> /usr/lib/system/libsystem_collections.dylib
0x1d624d000 - 0x1d6251fff libsystem_configuration.dylib arm64  <27db1d5d607530bf888edca2c13e80ef> /usr/lib/system/libsystem_configuration.dylib
0x1d6252000 - 0x1d6264fff libsystem_containermanager.dylib arm64  <cb5192ce591f3bb8bc455646bb298477> /usr/lib/system/libsystem_containermanager.dylib
0x1d6265000 - 0x1d6266fff libsystem_coreservices.dylib arm64  <b3a52ad613a83868ab485b08b9c8fa23> /usr/lib/system/libsystem_coreservices.dylib
0x1d6267000 - 0x1d6270fff libsystem_darwin.dylib arm64  <e5bec594b4643527b60ef752cb6e3386> /usr/lib/system/libsystem_darwin.dylib
0x1d6271000 - 0x1d6279fff libsystem_dnssd.dylib arm64  <e4268fdd91b5313994546631865c32ca> /usr/lib/system/libsystem_dnssd.dylib
0x1d627a000 - 0x1d627cfff libsystem_featureflags.dylib arm64  <1e7cd17eb5283b85bf4a27e1aa8dfa3e> /usr/lib/system/libsystem_featureflags.dylib
0x1d627d000 - 0x1d62aafff libsystem_m.dylib arm64  <dedeec341f9e3964a5ff79cddc6a811e> /usr/lib/system/libsystem_m.dylib
0x1d62ab000 - 0x1d62b5fff libsystem_platform.dylib arm64  <abb790d318753996aaee06e7568ea1c9> /usr/lib/system/libsystem_platform.dylib
0x1d62b6000 - 0x1d62b6fff libsystem_product_info_filter.dylib arm64  <61b5d8563fb33fe9b30540a45f659732> /usr/lib/system/libsystem_product_info_filter.dylib
0x1d62b7000 - 0x1d62c7fff libsystem_pthread.dylib arm64  <93c7f0dc89f13162a1196fc88028d027> /usr/lib/system/libsystem_pthread.dylib
0x1d62c8000 - 0x1d62cbfff libsystem_sandbox.dylib arm64  <176b1087aaad36258e7ffc497673706c> /usr/lib/system/libsystem_sandbox.dylib
0x1d62cc000 - 0x1d62d5fff libunwind.dylib arm64  <b7cd41ad513335ac84299bc0dcc6511b> /usr/lib/system/libunwind.dylib
0x1d62d6000 - 0x1d6308fff libxpc.dylib arm64  <e3552d23adfb36feb7d0b23090553c49> /usr/lib/system/libxpc.dylib

EOF

@markmentovai
Copy link

@markmentovai markmentovai commented Jun 25, 2021

Let me symbolize that for you (below). The crash is in __psynch_cvwait. The crash is structured as a SIGILL originating from software instead of a hardware trap. This is the same as #42774 (comment). FB8922558 is on file with Apple for this. The most recent update is that Apple believes it’s fixed in macOS 11.3 (2021-04-26), which has iOS equivalent 14.5). The crash report indicates that the test was performed on iOS 14.2 (2020-11-05), which is actually quite old. Have you tried reproducing on a newer iOS version? iOS 14.6 (2021-05-24) is current.

The SIGILL I identified was coming from 10.15.6 xnu-6153.141.1/bsd/dev/arm/unix_signal.c sendsig. I don’t see any diffs in that file between 11.0.1 xnu-7195.50.7.100.1 and 11.3 xnu-7195.101.1, and I haven’t spotted any diffs elsewhere in xnu that look like a fix for the problem. I don’t currently have anything running macOS 11.3 or 11.4 to verify, but I do have something running macOS 12.0db1 21A5248p (2021-06-07), xnu-7938.0.0.111.2, and I can’t reproduce the spurious signals there.

WARNING: crash2: received RPC error DATA_NOT_FOUND, details: Couldn't find symbol data for module dyld|F756A8D55A803B4D996236FE0E7638C20
WARNING: crash2: received RPC error DATA_NOT_FOUND, details: Couldn't find symbol data for module compile|CD51514BED8631D8AA75C4B00AA94C5D0

Incident Identifier: 75BDAC6F-8ADD-4DFF-B13C-48283FCAF48E
CrashReporter Key:   f09d9091c97b4e4f6bb695146954c56a7f0495a7
Hardware Model:      iPhone8,4
Process:             compile [6249]
Path:                /private/var/root/goroot/pkg/tool/ios_arm64/compile
Identifier:          compile
Version:             ???
Code Type:           ARM-64 (Native)
Role:                Unspecified
Parent Process:      go_bootstrap [5667]
Coalition:           com.openssh.sshd.12FDA541-4545-4AF8-976B-7B801CB9D777 [455]


Date/Time:           2021-06-25 06:06:13.6773 -0700
Launch Time:         2021-06-25 06:05:39.0166 -0700
OS Version:          iPhone OS 14.2 (18B92)
Release Type:        User
Baseband Version:
Report Version:      104

Exception Type:  EXC_CRASH (SIGILL)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Exception Note:  EXC_CORPSE_NOTIFY
Termination Signal: Illegal instruction: 4
Termination Reason: Namespace SIGNAL, Code 0x4
Terminating Process: compile [6249]
Triggered by Thread:  0

Thread 0 name:  Dispatch queue: com.apple.main-thread
Thread 0 Crashed:
0   libsystem_kernel.dylib        	0x00000001bac8d61c __psynch_cvwait + 
1   libsystem_pthread.dylib       	0x00000001d62ba2fc _pthread_cond_wait$VARIANT$mp + 
2   compile                       	0x0000000104b79888 0x104b10000 + 432264
3   compile                       	0x0000000104b786c8 0x104b10000 + 427720
4   ???                           	0x00000000d65f03c0 0 + 3596551104

Thread 1:
0   libsystem_pthread.dylib       	0x00000001d62c586c start_wqthread + 

Thread 2:
0   libsystem_kernel.dylib        	0x00000001bac8dbf0 __semwait_signal + 
1   libsystem_c.dylib             	0x000000019890a6d4 nanosleep + 
2   libsystem_c.dylib             	0x000000019890a5ac usleep + 
3   compile                       	0x0000000104b79514 0x104b10000 + 431380
4   compile                       	0x0000000104b786c8 0x104b10000 + 427720

Thread 3:
0   compile                       	0x0000000104b3559c 0x104b10000 + 152988
1   compile                       	0x0000000104b3558c 0x104b10000 + 152972

Thread 4:
0   libsystem_kernel.dylib        	0x00000001bac8d61c __psynch_cvwait + 
1   libsystem_pthread.dylib       	0x00000001d62ba2fc _pthread_cond_wait$VARIANT$mp + 
2   compile                       	0x0000000104b79888 0x104b10000 + 432264
3   compile                       	0x0000000104b786c8 0x104b10000 + 427720
4   ???                           	0x00000000d65f03c0 0 + 3596551104

Thread 5:
0   libsystem_kernel.dylib        	0x00000001bac8d61c __psynch_cvwait + 
1   libsystem_pthread.dylib       	0x00000001d62ba2fc _pthread_cond_wait$VARIANT$mp + 
2   compile                       	0x0000000104b79888 0x104b10000 + 432264
3   compile                       	0x0000000104b786c8 0x104b10000 + 427720
4   ???                           	0x00000000d65f03c0 0 + 3596551104

Thread 6:
0   compile                       	0x0000000104c9fb78 0x104b10000 + 1637240
1   compile                       	0x0000000104cbb250 0x104b10000 + 1749584

Thread 0 crashed with ARM Thread State (64-bit):
    x0: 0x0000000000000104   x1: 0x0000000000000000   x2: 0x0000000000017600   x3: 0x0000000000000000
    x4: 0x0000000000000000   x5: 0x00000000000000a0   x6: 0x0000000000000000   x7: 0x0000000000000000
    x8: 0x000000016b2ed8e8   x9: 0x0000000000017601  x10: 0x00000001052734d8  x11: 0x0000000000000002
   x12: 0x00000000000020a0  x13: 0x0000000000000000  x14: 0x0000000000000000  x15: 0x0000000000000000
   x16: 0x0000000000000131  x17: 0x0000000000000000  x18: 0x0000000000000000  x19: 0x00000001052734c0
   x20: 0x0000000105273500  x21: 0x0000000105917960  x22: 0x0000000000000000  x23: 0x0000000000000000
   x24: 0x0000000000017600  x25: 0x0000000000017601  x26: 0x0000000000017700  x27: 0x0000000000000040
   x28: 0x0000000105272d40   fp: 0x000000016b2ed960   lr: 0x00000001d62ba2fc
    sp: 0x000000016b2ed8d0   pc: 0x00000001bac8d61c cpsr: 0x60000000
   esr: 0x56000080  Address size fault

Binary Images:
0x104b10000 - 0x1051affff compile arm64  <cd51514bed8631d8aa75c4b00aa94c5d> /var/root/goroot/pkg/tool/ios_arm64/compile
0x105870000 - 0x1058dbfff dyld arm64  <f756a8d55a803b4d996236fe0e7638c2> /usr/lib/dyld
0x18f839000 - 0x18f8b7fff libdispatch.dylib arm64  <6c143c51288f3cd084dd17b9a0aaee62> /usr/lib/system/libdispatch.dylib
0x18f8b8000 - 0x18f8eefff libdyld.dylib arm64  <e31c0887c99930479b80203248ec2396> /usr/lib/system/libdyld.dylib
0x18f8ef000 - 0x18fb46fff libicucore.A.dylib arm64  <9fc7dac3deed3a479abbe36867045ba5> /usr/lib/libicucore.A.dylib
0x18fb47000 - 0x18fef0fff CoreFoundation arm64  <96f8386dd88a3c89a323a17975c3317f> /System/Library/Frameworks/CoreFoundation.framework/CoreFoundation
0x198874000 - 0x198899fff libsystem_info.dylib arm64  <3536d46d36cd3af4b92c67e4bdebd8c6> /usr/lib/system/libsystem_info.dylib
0x19889a000 - 0x198916fff libsystem_c.dylib arm64  <edf3e843a62a3dd092ccebe94d5a83e3> /usr/lib/system/libsystem_c.dylib
0x19e670000 - 0x19e692fff libsystem_malloc.dylib arm64  <9f2d3d4683d9358f972c1cf61c073a31> /usr/lib/system/libsystem_malloc.dylib
0x1a05fd000 - 0x1a060cfff libsystem_networkextension.dylib arm64  <0e55773557ba3611994ce782f3cbfe75> /usr/lib/system/libsystem_networkextension.dylib
0x1a360c000 - 0x1a3641fff libobjc.A.dylib arm64  <7cad222912fa3d6c82005306fb157fa4> /usr/lib/libobjc.A.dylib
0x1a36a2000 - 0x1a36fbfff libc++.1.dylib arm64  <88496d66c6683f1797c5d0c91e8bee42> /usr/lib/libc++.1.dylib
0x1a36fc000 - 0x1a3714fff libc++abi.dylib arm64  <464199d1cdfe3663bedfcbc1b5f19491> /usr/lib/libc++abi.dylib
0x1a525a000 - 0x1a5270fff libsystem_trace.dylib arm64  <4ed5b6b480e9322784b590bc92570931> /usr/lib/system/libsystem_trace.dylib
0x1a8f07000 - 0x1a8f11fff libsystem_notify.dylib arm64  <4dbd2e3e6bc43d1388d0e01ba0cc4e07> /usr/lib/system/libsystem_notify.dylib
0x1a8fbc000 - 0x1a902bfff libcorecrypto.dylib arm64  <071f7a201cf6324783d7d4e309a64c16> /usr/lib/system/libcorecrypto.dylib
0x1a911f000 - 0x1a9135fff libsystem_asl.dylib arm64  <441adb81097f31d6a85d380a72993d66> /usr/lib/system/libsystem_asl.dylib
0x1afeaf000 - 0x1afeb6fff libsystem_symptoms.dylib arm64  <cf41ba3e1ea634e58692d79f7c1b90b8> /usr/lib/system/libsystem_symptoms.dylib
0x1bac66000 - 0x1bac96fff libsystem_kernel.dylib arm64  <8bcb6ff951623657a2e25ce5704353e6> /usr/lib/system/libsystem_kernel.dylib
0x1bd6dc000 - 0x1bd6ddfff libSystem.B.dylib arm64  <5f81896adca13ef7999f6c5da89a750b> /usr/lib/libSystem.B.dylib
0x1d6140000 - 0x1d6145fff libcache.dylib arm64  <93f46de334c83b548e87b685e63149e7> /usr/lib/system/libcache.dylib
0x1d6146000 - 0x1d6152fff libcommonCrypto.dylib arm64  <163c756fa4413bb7b2ca067463d98526> /usr/lib/system/libcommonCrypto.dylib
0x1d6153000 - 0x1d6157fff libcompiler_rt.dylib arm64  <6749df438089311b939a87f9ca479efd> /usr/lib/system/libcompiler_rt.dylib
0x1d6158000 - 0x1d6160fff libcopyfile.dylib arm64  <5ea5d77aebc930459a3ccf1838e73c6b> /usr/lib/system/libcopyfile.dylib
0x1d623e000 - 0x1d623efff liblaunch.dylib arm64  <a0b8f064a9ee310e8b976e27f49a05f2> /usr/lib/system/liblaunch.dylib
0x1d623f000 - 0x1d6244fff libmacho.dylib arm64  <863174fc64663104aca38d1d4a82d075> /usr/lib/system/libmacho.dylib
0x1d6245000 - 0x1d6247fff libremovefile.dylib arm64  <fc34eec2b9013ebbbc31e73f38842b70> /usr/lib/system/libremovefile.dylib
0x1d6248000 - 0x1d6249fff libsystem_blocks.dylib arm64  <bc8afa7d401e35f789e89e7a59aef82b> /usr/lib/system/libsystem_blocks.dylib
0x1d624a000 - 0x1d624cfff libsystem_collections.dylib arm64  <828126f3e6583da2aceac728678588de> /usr/lib/system/libsystem_collections.dylib
0x1d624d000 - 0x1d6251fff libsystem_configuration.dylib arm64  <27db1d5d607530bf888edca2c13e80ef> /usr/lib/system/libsystem_configuration.dylib
0x1d6252000 - 0x1d6264fff libsystem_containermanager.dylib arm64  <cb5192ce591f3bb8bc455646bb298477> /usr/lib/system/libsystem_containermanager.dylib
0x1d6265000 - 0x1d6266fff libsystem_coreservices.dylib arm64  <b3a52ad613a83868ab485b08b9c8fa23> /usr/lib/system/libsystem_coreservices.dylib
0x1d6267000 - 0x1d6270fff libsystem_darwin.dylib arm64  <e5bec594b4643527b60ef752cb6e3386> /usr/lib/system/libsystem_darwin.dylib
0x1d6271000 - 0x1d6279fff libsystem_dnssd.dylib arm64  <e4268fdd91b5313994546631865c32ca> /usr/lib/system/libsystem_dnssd.dylib
0x1d627a000 - 0x1d627cfff libsystem_featureflags.dylib arm64  <1e7cd17eb5283b85bf4a27e1aa8dfa3e> /usr/lib/system/libsystem_featureflags.dylib
0x1d627d000 - 0x1d62aafff libsystem_m.dylib arm64  <dedeec341f9e3964a5ff79cddc6a811e> /usr/lib/system/libsystem_m.dylib
0x1d62ab000 - 0x1d62b5fff libsystem_platform.dylib arm64  <abb790d318753996aaee06e7568ea1c9> /usr/lib/system/libsystem_platform.dylib
0x1d62b6000 - 0x1d62b6fff libsystem_product_info_filter.dylib arm64  <61b5d8563fb33fe9b30540a45f659732> /usr/lib/system/libsystem_product_info_filter.dylib
0x1d62b7000 - 0x1d62c7fff libsystem_pthread.dylib arm64  <93c7f0dc89f13162a1196fc88028d027> /usr/lib/system/libsystem_pthread.dylib
0x1d62c8000 - 0x1d62cbfff libsystem_sandbox.dylib arm64  <176b1087aaad36258e7ffc497673706c> /usr/lib/system/libsystem_sandbox.dylib
0x1d62cc000 - 0x1d62d5fff libunwind.dylib arm64  <b7cd41ad513335ac84299bc0dcc6511b> /usr/lib/system/libunwind.dylib
0x1d62d6000 - 0x1d6308fff libxpc.dylib arm64  <e3552d23adfb36feb7d0b23090553c49> /usr/lib/system/libxpc.dylib

@eliasnaur
Copy link
Contributor Author

@eliasnaur eliasnaur commented Jun 25, 2021

Thank you for your analysis, @markmentovai. I don't mind upgrading the builders to a newer iOS, but I don't want to paper over the issue for devices running older iOS versions. If the issue is the same as #42774, why don't our macOS workarounds work?

@markmentovai
Copy link

@markmentovai markmentovai commented Jun 25, 2021

@eliasnaur I left my thoughts on the mlock workaround at #42774 (comment). Is RLIMIT_MEMLOCK enforced on iOS? It’s not by default on macOS. (This is ulimit -l, not that the command would help much on iOS.)

I don't mind upgrading the builders to a newer iOS, but I don't want to paper over the issue for devices running older iOS versions.

Spurious SIGILL is already a problem on iOS (that’s what this bug is about) and is not solvable (and not even really readily able to be worked around) without kernel fixes. We thought that we could work around it for iOS ≥ 14 with sigaltstack and mlock. If it turns out that we can’t actually work around effectively until iOS 14.5 (sigaltstack, with or without mlock), it’s still a win. But note that this may be fixed on iOS 14.5 even without sigaltstack if the kernel fix was correct and comprehensive.

@eliasnaur
Copy link
Contributor Author

@eliasnaur eliasnaur commented Jun 25, 2021

@eliasnaur I left my thoughts on the mlock workaround at #42774 (comment). Is RLIMIT_MEMLOCK enforced on iOS? It’s not by default on macOS. (This is ulimit -l, not that the command would help much on iOS.)

ulimit -l returns unlimited on the virtual Corellium builders where the crashes appear.

I don't mind upgrading the builders to a newer iOS, but I don't want to paper over the issue for devices running older iOS versions.

Spurious SIGILL is already a problem on iOS (that’s what this bug is about) and is not solvable (and not even really readily able to be worked around) without kernel fixes. We thought that we could work around it for iOS ≥ 14 with sigaltstack and mlock. If it turns out that we can’t actually work around effectively until iOS 14.5 (sigaltstack, with or without mlock), it’s still a win. But note that this may be fixed on iOS 14.5 even without sigaltstack if the kernel fix was correct and comprehensive.

I'd still be interested in knowing why @cherrymui's CL doesn't work around the problem on 14.2 like a similar CL does on macOS. But I see your point now: a workaround based on sigaltstack will only cover [14.0;14.4] which is probably not worth the trouble.

I'll upgrade the builders and report back.

@markmentovai
Copy link

@markmentovai markmentovai commented Jun 25, 2021

ulimit -l returns unlimited on the virtual Corellium builders where the crashes appear.

But that’s the builder and not the device that you’re running tests on?

@eliasnaur
Copy link
Contributor Author

@eliasnaur eliasnaur commented Jun 25, 2021

ulimit -l returns unlimited on the virtual Corellium builders where the crashes appear.

But that’s the builder and not the device that you’re running tests on?

I only test on the Corellium builders. Running all.bash on devices is too painful for me.

@eliasnaur
Copy link
Contributor Author

@eliasnaur eliasnaur commented Jun 25, 2021

I've upgraded all builders now. I haven't reproduced the SIGILL crashes yet, but get these weird errors now:

/var/root/goroot/pkg/tool/ios_arm64/link: running /var/root/bin/clangwrap failed: exit status 1
ld: building for iOS, but linking in object file built for macOS, file '/tmp/go-link-218947704/000000.o' for architecture arm64
Not signing file
clang-5.0: error: linker command failed with exit code 1 (use -v to see invocation)
FAIL	runtime/race [build failed]

I've filed #46931.

@cherrymui
Copy link
Contributor

@cherrymui cherrymui commented Jun 25, 2021

Thanks @eliasnaur and @markmentovai for the update!

Regarding the workaround, I noticed that the SIGILL occurs very early, during bootstrapping (this is also the case you commented on CL 279489). It is possible that the workaround is actually effective, but the bootstrap toolchain, which does not include the workaround, may still crash.

That said, as you said earlier the workaround only helps 14.0 - 14.4, so probably don't bother.

Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants