cmd/compile: arm64 rematerializes multi-instruction 64-bit constants inside loops #81755

Go version

go version go1.28-devel_651ec67c3b Fri Sep 25 09:02:19 2026 -0700 (master), building for GOOS=linux GOARCH=arm64. go1.27.1 generates the same code, so this isn't a regression.

Output of go env in your module/workspace:

GOOS=linux
GOARCH=arm64
GOARM64=v8.0
CGO_ENABLED=0
GOEXPERIMENT=

Cross-compiled from linux/amd64 and benchmarked on linux/arm64 (Neoverse N1).

What did you do?

A loop multiplying by a 64-bit constant that doesn't fit an arm64 immediate, compared with the same loop loading the constant from a variable:

package repro

const prime = 0xcf1bbcdcb7a56463

//go:noinline
func Hash(dst []uint32, src []uint64) {
	for i, v := range src {
		dst[i] = uint32((v * prime) >> 45)
	}
}

var primeVar uint64 = prime

//go:noinline
func HashVar(dst []uint32, src []uint64) {
	p := primeVar
	for i, v := range src {
		dst[i] = uint32((v * p) >> 45)
	}
}

What did you see happen?

Hash rebuilds the constant with MOVD + 3×MOVK on every iteration, even though the loop uses only a handful of registers. (From go tool objdump, with branch targets relabelled for readability.)

loop:
	LSR	$45, R5, R5
	MOVW	R5, (R0)(R2<<2)
	ADD	$1, R2, R2
	CMP	R2, R4
	BLE	done
	MOVD	(R3)(R2<<3), R5
	MOVD	$25699, R6
	MOVK	$(47013<<16), R6
	MOVK	$(48348<<32), R6
	MOVK	$(53019<<48), R6
	MUL	R6, R5, R5
	CMP	R2, R1
	BHI	loop

HashVar loads it once before the loop and keeps it in R2:

	ADRP	..., R27
	MOVD	1232(R27), R2
loop:
	LSR	$45, R6, R6
	MOVW	R6, (R0)(R5<<2)
	ADD	$1, R5, R5
	CMP	R5, R4
	BLE	done
	MOVD	(R3)(R5<<3), R6
	MUL	R2, R6, R6
	CMP	R5, R1
	BHI	loop

On a Neoverse N1 (4096 elements, one pinned core, -count 10), HashVar is 20% faster:

     │   Hash    │              HashVar               │
     │  sec/op   │   sec/op     vs base               │
Hash   5.161µ ± 0%   4.128µ ± 0%  -20.02% (p=0.000 n=10)

For comparison, amd64 makes the same choice, rebuilding the constant inside the loop, but there it is a single MOVQ $imm64 with no dependency chain:

loop:
	SHRQ	$0x2d, DX
	MOVL	DX, 0(AX)(CX*4)
	INCQ	CX
	CMPQ	SI, CX
	JLE	done
	MOVQ	0(DI)(CX*8), DX
	MOVQ	$0xcf1bbcdcb7a56463, R8
	IMULQ	R8, DX
	CMPQ	BX, CX
	JA	loop

so the same comparison on a Ryzen 5 5600X is much smaller:

     │   Hash    │              HashVar              │
     │  sec/op   │   sec/op     vs base              │
Hash   2.037µ ± 7%   1.956µ ± 2%  -3.98% (p=0.000 n=10)

What did you expect to see?

The constant built once outside the loop and kept in a register, as with HashVar.

As far as I can tell, this is because MOVDconst is marked rematerializeable: true (ARM64Ops.go#L413), and (*Value).Rematerializeable only looks at the opcode. So the register allocator treats a constant that takes 4 instructions to build the same as one that takes 1. That's the right call on amd64, where MOVQ $imm64 is always one instruction, but not on arm64. The MOVDconst ops for riscv64, ppc64, mips64 and loong64 carry the same flag. I haven't checked their code generation.

Motivation: this showed up in the zstd encoder's match-finder hashing in klauspost/compress. klauspost/compress#1230 works around it by loading the hash primes from a package-level var on arm64 only (hashPrimesInRegs = runtime.GOARCH == "arm64"). That accounts for most of the PR's +6–8% encode throughput on Graviton2/3/4. On amd64 the same workaround is slower, because rebuilding the constant costs a single MOVQ and holding it adds spills in a loop that already has high register pressure. So the workaround has to be per-architecture, and it would be better fixed in the compiler.

Loop-invariant code motion (#63670) wouldn't help on its own: SSA constants already live in the entry block, outside the loop, and the register allocator re-emits them at each use.

🤖 Generated with Claude Code