licm should follow last instcombine

[chriselrod]

Godbolt
LLVM Bugzilla

julia> using BenchmarkTools

julia> x = rand(128); y = rand(128);

julia> @noinline function vdiv!(x, y, a)
           inva = inv(a)
           @inbounds for i ∈ eachindex(x, y)
               x[i] = y[i] * inva
           end
           x
       end
vdiv! (generic function with 1 method)

julia> @noinline function vdiv_fast!(x, y, a)
           @fastmath inva = 1 / a
           @inbounds for i ∈ eachindex(x, y)
               @fastmath x[i] = y[i] * inva
           end
           x
       end
vdiv_fast! (generic function with 1 method)

julia> @btime vdiv_fast!($x, $y, 1/1.4);
  71.293 ns (0 allocations: 0 bytes)

julia> @btime vdiv!($x, $y, 1/1.4);
  11.967 ns (0 allocations: 0 bytes)

The LICM pass moves fdiv out of loops, while instcombine moves it back in by combining the multiplication and division.

Clang does not have any instcombines following the last licm, while Julia does.
I don't know if fdiv is the only instruction with this adversarial licm-instcombine behavior, but if it isn't, it seems (based on Clang) that the LLVM passes assume licm will be ordered last.

[laborg]

I lack the confidence regarding LLVM to close this one, but apparently on master its the same speed:

julia> @btime vdiv_fast!($x, $y, 1/1.4);
  31.550 ns (0 allocations: 0 bytes)

julia> @btime vdiv!($x, $y, 1/1.4);
  33.171 ns (0 allocations: 0 bytes)

I don't have

[chriselrod]

I can confirm:

julia> @btime vdiv_fast!($x, $y, 1/1.4);
  10.291 ns (0 allocations: 0 bytes)

julia> @btime vdiv!($x, $y, 1/1.4);
  9.776 ns (0 allocations: 0 bytes)

I also double checked the assembly. They are not generating the same code, but both are using vmulpd instead of vdivpd, @fastmath:

.LBB0_8:                                # %vector.body
                                        # =>This Inner Loop Header: Depth=1
	vmulpd	zmm3, zmm2, zmmword ptr [r10 + 8*rbx]
	vmovupd	zmmword ptr [rcx + 8*rbx], zmm3
	vmulpd	zmm3, zmm2, zmmword ptr [r10 + 8*rbx + 64]
	vmovupd	zmmword ptr [rcx + 8*rbx + 64], zmm3
	vmulpd	zmm3, zmm2, zmmword ptr [r10 + 8*rbx + 128]
	vmovupd	zmmword ptr [rcx + 8*rbx + 128], zmm3
	vmulpd	zmm3, zmm2, zmmword ptr [r10 + 8*rbx + 192]
	vmovupd	zmmword ptr [rcx + 8*rbx + 192], zmm3
	add	rbx, 32
	add	rsi, -4
	jne	.LBB0_8

Without @fastmath, LLVM unrolls more aggressively and interleaves, which it doesn't do with @fastmath:

.LBB0_8:                                # %vector.body
                                        # =>This Inner Loop Header: Depth=1
	vmulpd	zmm2, zmm1, zmmword ptr [rax + 8*rsi]
	vmulpd	zmm3, zmm1, zmmword ptr [rax + 8*rsi + 64]
	vmulpd	zmm4, zmm1, zmmword ptr [rax + 8*rsi + 128]
	vmulpd	zmm5, zmm1, zmmword ptr [rax + 8*rsi + 192]
	vmovupd	zmmword ptr [rcx + 8*rsi], zmm2
	vmovupd	zmmword ptr [rcx + 8*rsi + 64], zmm3
	vmovupd	zmmword ptr [rcx + 8*rsi + 128], zmm4
	vmovupd	zmmword ptr [rcx + 8*rsi + 192], zmm5
	vmulpd	zmm2, zmm1, zmmword ptr [rax + 8*rsi + 256]
	vmulpd	zmm3, zmm1, zmmword ptr [rax + 8*rsi + 320]
	vmulpd	zmm4, zmm1, zmmword ptr [rax + 8*rsi + 384]
	vmulpd	zmm5, zmm1, zmmword ptr [rax + 8*rsi + 448]
	vmovupd	zmmword ptr [rcx + 8*rsi + 256], zmm2
	vmovupd	zmmword ptr [rcx + 8*rsi + 320], zmm3
	vmovupd	zmmword ptr [rcx + 8*rsi + 384], zmm4
	vmovupd	zmmword ptr [rcx + 8*rsi + 448], zmm5
	add	rsi, 64
	add	rbx, -2
	jne	.LBB0_8

Not sure why it does that, but my issue was about using vdivpd instead of vmulpd.