cmd/asm: R15 gets clobbered in plugin mode

[DankFC]

What version of Go are you using (go version)?

go version : 1.15.7 ( However go version 1.14 and 1.16beta1 also show same error behaviour.)

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

OS: linux x86_64

What did you do?

A bug which occurs only in plugins with assembly code.
The package consists of a bn256 crypto implementation which has amd64 assembly code and also a generic implementation.
The assembly implementation is default and generic implementation can be enabled using the "generic" tag.
The plugin when compiled as a binary runs OK with both assembly and go implementation.
The plugin when compiled as a plugin FAILS with assembly code but runs OK with generic implementation.

TESTING:

#Run plugin directly to see correct result.
go run plugin/plugin.go

#Compile as plugin.
go build -buildmode=plugin -o /tmp/plugin.so ../compiler_bug/plugin/

#Running plugin will trigger error
go run runplugin/main.go

However, if the plugin is compiled with generic tag (To disable asm code, the bug doesn't occur), Some register corruption occurs due to wrong code being generated.
go build -buildmode=plugin -tags generic -o /tmp/plugin.so ../compiler_bug/plugin/

Running plugin will not cause an error.
go run runplugin/main.go

Repo to reproduce

https://github.com/DankFC/compile-bug

What did you expect to see?

./demo.sh 
Actual a01f9bcc1208dee302769931ad378a4c0c4b2c21b0cfb3e752607e12d2b6fa6425 
Expected a01f9bcc1208dee302769931ad378a4c0c4b2c21b0cfb3e752607e12d2b6fa6425 
running plugin 
Actual a01f9bcc1208dee302769931ad378a4c0c4b2c21b0cfb3e752607e12d2b6fa6425 
Expected a01f9bcc1208dee302769931ad378a4c0c4b2c21b0cfb3e752607e12d2b6fa6425 

What did you see instead?

./demo.sh 
Actual a01f9bcc1208dee302769931ad378a4c0c4b2c21b0cfb3e752607e12d2b6fa6425 
Expected a01f9bcc1208dee302769931ad378a4c0c4b2c21b0cfb3e752607e12d2b6fa6425 
running plugin 
panic: Decode point err bn256: malformed point err 
goroutine 1 [running]: 
_/tmp/compiler_bug/plugin.Export() 
/tmp/compiler_bug/plugin/plugin.go:23 +0x2d7 
main.main() 
/tmp/compiler_bug/runplugin/main.go:24 +0x107 
exit status 2 

demo.sh is included in the repo to recreate the bug easily.

[randall77]

Try adding NOSPLIT to your assembly code. For example,

TEXT ·gfpNeg(SB),0,$0-16

TEXT ·gfpNeg(SB),NOSPLIT,$0-16

You probably also need

#include "textflag.h"

What I suspect is happening is that depending on stack size (which is why plugin may change things) a stack growth is occurring on entry to your assembly. The stack gets copied (or possibly, a GC gets run), but the arguments to your assembly don't get adjusted (or, if it is the GC, its referents marked).

You need either NOSPLIT on your assembly, so a stack copy can't happen on entry, or provide a proper argument map for that assembly so the stack copier knows where the pointers are in its arguments. I believe the NO_LOCAL_POINTERS macro (see runtime/asm_amd64.s in the stdlib) will do that for you, but I haven't done that in a while. NOSPLIT is simpler.

[randall77]

Acutally, maybe not. I think you should get pointer maps automatically added, see the docs for GO_ARGS in runtime/funcdata.h.
Worth trying NOSPLIT just to see if it helps, though.

[DankFC]

Acutally, maybe not. I think you should get pointer maps automatically added, see the docs for GO_ARGS in runtime/funcdata.h.
Worth trying NOSPLIT just to see if it helps, though.

Thanks for the input, but there was no change using NOSPLIT.

[randall77]

New theory: you can't use R15 in plugin assembly.
It's used for accessing the GOT. See rewriteToUseGot in cmd/internal/obj/x86/obj6.go.
Or at least, you can't use R15 and access a global variable in the same assembly. Accessing a global variable clobbers R15.

Replacing R15 with CX in your assembly fixes it.

Let me know if you can get it to work as well.
We really need to either not clobber R15, or document it better, or have the toolchain complain when using R15 in GOT assembly, or something.

[randall77]

Our compiler avoids R15 when compiling for dynamic linking (cmd/compile/internal/ssa/regalloc.go):

	if s.f.Config.ctxt.Flag_dynlink {
		switch s.f.Config.arch {
		case "amd64":
			s.allocatable &^= 1 << 15 // R15

[randall77]

It's been like this for a while. See #18820.

I've got a CL that errors out in the assembler when it detects a bad case. I think waiting until 1.17 is the right plan.