cmd/compile: many new no-ops as a result of mid-stack inlining

[laboger]

What version of Go are you using (go version)?

latest

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

ppc64le and ppc64

What did you do?

I was inspecting code for another reason and I saw a lot of new no-ops in unexpected places. For example in runtime.cgoCheckTypedBlock:

       return mheap_.arenas[ai.l1()][ai.l2()].spans[(p/pageSize)%pagesPerArena]
  0x13d08               3fe0002b                ADDIS $0,$43,R31        
  0x13d0c               e8ff3728                MOVD 14120(R31),R7      
  0x13d10               8be70000                MOVBZ 0(R7),R31         
        s := spanOfUnchecked(uintptr(src))
  0x13d14               7c000378                OR R0,R0,R0             
  0x13d18               7c000378                OR R0,R0,R0             
  0x13d1c               7dc87378                OR R14,R14,R8           
        ai := arenaIndex(p)
  0x13d20               7c000378                OR R0,R0,R0             
  0x13d24               7c000378                OR R0,R0,R0             
        return mheap_.arenas[ai.l1()][ai.l2()].spans[(p/pageSize)%pagesPerArena]
  0x13d28               7c000378                OR R0,R0,R0             
  0x13d2c               7c000378                OR R0,R0,R0             
  0x13d30               7c000378                OR R0,R0,R0             
  0x13d34               7c000378                OR R0,R0,R0             
        return arenaIdx((p + arenaBaseOffset) / heapArenaBytes)
  0x13d38               79c93682                RLDICL R14,$38,$26,R9   

What did you expect to see?

I should only see no-ops where they are expected.

What did you see instead?

Lots of no-ops in unexpected places. In the test for bytes there are almost 9000 extra no-ops with this commit.

cd src/bytes
go test -c
objdump -D bytes.test | grep mr | grep r0,r0 | wc -l
Using go version devel +c043fc4 Fri Dec 28 04:17:55 2018 +0000 linux/ppc64le
347
Using go version devel +69c2c56 Fri Dec 28 20:55:36 2018 +0000 linux/ppc64le
9193

I have not found runtime breakage because of this but performance could be affected depending where the unnecessary no-ops occur.

[randall77]

There's generally one nop per inlining that the compiler does. These are the InlMark opcodes that represent the callsite of an inlined function. They are required to get stack tracebacks correct when mid-stack inlining is enabled.

There's two ways to reduce them, both of which I was going to leave for 1.13 (but let me know if this is painful enough to try to get in for 1.12):

1. Use an existing instruction instead of adding a nop. All we need is an instruction with the right file/line number to use as the inline mark. Existing instructions would work fine (if there is one, which there probably usually is).
2. Drop nops if no safepoint references them. This will typically happen for nops inserted for leaf inlinings. Note that "safepoints" here includes instructions that might panic (like loads), so it might not remove all that many nops. And we're moving towards having safepoints everwhere™ which would probably make this case moot, except for inlined bodies which were completely optimized away.

I checked one of the cases you reported and the number of nops looks correct. For s := spanOfUnchecked(uintptr(src)) I expect to see 4 nops, one for the inlining of spanOfUnchecked and one each for the calls to arenaIndex, l1, and l2 called by spanOfUnchecked.

[laboger]

For now it's only painful to me because of how it looks. I will do some testing to see if it affects performance in any meaningful way. Otherwise your fixes for 1.13 sound OK.

1. seems more promising, because isn't there always at least 1 instruction from the inlined code?

[randall77]

1. seems more promising, because isn't there always at least 1 instruction from the inlined code?

Not always. func f() { g() } generates no code for f. But usually there will be.

[laboger]

I'm OK with leaving this as is for 1.12 if there will be improvements in 1.13.
Do you want to keep this issue for that work or do you have another?

[randall77]

Keeping this issue is fine.

[gopherbot]

Change https://golang.org/cl/158021 mentions this issue: cmd/compile: use existing instructions instead of nops for inline marks

[gopherbot]

Change https://golang.org/cl/170445 mentions this issue: cmd/compile: get rid of unnecessary inline marks

[randall77]

I don't plan on doing anything more on this for 1.13.
Punting to 1.14.

Any further work here I think involves either associating multiple source lines with an instruction, and/or with encoding that fact in dwarf.