runtime: use register instead of TLS for G pointer for darwin/arm and darwin/arm64

[randall77]

Need to decide how to handle TLS on darwin/arm. On x86, TLS was moved from a pthread_get/setspecific implementation to a reserved (by Apple) slot in the TLS (see #23617). That's not possible for arm/arm64 so we still use the old method. Since these archs have more registers, we should probably just reserve a register for the G pointer.

Reserving a register might invalidate some existing assembly code. Kind of a wrench in the plan, need to investigate how bad it might be.

Or maybe someone can think of a better solution.

We might need a solution before we can do #17490 for arm.

[cherrymui]

I think we already do reserve a G register on ARM and ARM64, regardless of OS. This is R10 on ARM, and R28 on ARM64. TLS is only used when crossing Go-C boundary, where we save G register to a TLS slot when calling to C, and reload it when entering Go code, as C code may clobber our G register.

[randall77]

Maybe this is unnecessary then.

[eliasnaur]

The pthread_get/set magic is still present in runtime/cgo/gcc_darwin_arm*.c so I wouldn't qualify that as unnecessary.

@cherrymui given that we only use the TLS slot for Cgo, could we use a callee-saved register for G and thus avoid the TLS slot (and the save/restore) altogether?

[cherrymui]

I haven't looked into the code in detail, but I'm not sure callee-saved register is enough. C code can still temporarily use the register, and a signal can happen at this point. The signal handler, runtime.sigtramp needs to look at the G register to tell whether/which Go stack we are on.

[bcmills]

Backing up one level: why does runtime.sigtramp need to know which goroutine is running if we're in a C call?

[ianlancetaylor]

I'm not sure I completely understand the question, but see the sigtramp code in runtime/sys_darwin_arm.s and runtime/sys_darwin_arm64.s. It does today examine the g register, which it loads out of the TLS slot.

[cherrymui]

Besides signal handler, another place where we need G is when Go calls C which calls back to Go. There we want to use the same G, I think.

[aclements]

What's the status of this? Is there actually a bug here?

[eliasnaur]

#17490 is fixed, even for iOS, so the remaining bug is that we're relying on an undocumented assumption that a TLS slot allocated by pthread_key_create is located at some positive offset from the TLS base. See

go/src/runtime/cgo/gcc_darwin_arm64.c

Lines 31 to 45 in 59ea685

	err = pthread_key_create(&k, nil);
	if(err != 0) {
	fprintf(stderr, "runtime/cgo: pthread_key_create failed: %d\n", err);
	abort();
	}
	//fprintf(stderr, "runtime/cgo: k = %d, tlsbase = %p\n", (int)k, tlsbase); // debug
	pthread_setspecific(k, (void*)magic);
	// The first key should be at 257.
	for (i=0; i<PTHREAD_KEYS_MAX; i++) {
	if (*(tlsbase+i) == (void*)magic) {
	*tlsg = (void*)(i*sizeof(void *));
	pthread_setspecific(k, 0);
	return;
	}
	}

We recently had to remove a similar assumption for Android (#29674).

Since we use a register for g on arm and arm64 the appropriate fix for this issue is probably to call pthread_getspecific and pthread_setspecific when entering or exiting C code.