Inlining arithmetic can produce slowdowns

[timholy]

I've got an application where I'm seeing a big performance difference between

cm*vm + c*v + cp*vp

and

(cm*vm + c*v) + cp*vp

This is related to previous issues #6193 (comment) #5011 #7075 (comment) #10278. However, they're all closed, so I thought it would be better to start fresh.

This demo relies on ForwardDiff.jl, and uses the code in this gist.

Timing results:

julia> include("perf_parens.jl")
Warmup @time
  0.000001 seconds (3 allocations: 144 bytes)
Without parentheses:
  0.350804 seconds (16.00 M allocations: 488.282 MB, 19.45% gc time)
With parentheses:
  0.085496 seconds (15 allocations: 624 bytes)

Considerably more discussion can be found here.

Anything to do here? Or just tweak Interpolations.jl (which is where those functions get @generated) to insert the parentheses?

[jrevels]

To add a little context, GradientNumber is the number type with forcibly inlined arithmetic. With that in mind, I think the most useful comparisons here might be found by calling the below code (after loading the gist linked by @timholy above):

G = ForwardDiff.GradientNumber{1,Float64,Tuple{Float64}}
gdx, gdy = rand(G), rand(G)

@code_llvm mygetindex_slow(A, gdx, gdy)
@code_llvm mygetindex_fast(A, gdx, gdy)

In Tim's actual gist, ForwardDiff uses G = ForwardDiff.GradientNumber{2,Float64,Tuple{Float64, Float64}}, but the single-component case shown above is more minimal for the sake of demonstrating the performance differences.

[timholy]

Actually, @jrevels, since this affects even the 1d case, we can make this even easier. It really is just the two arithmetic operations at the top. Would you be able to put together a demonstration in "raw" julia, without using ForwardDiff at all? Might help debug this.

[jrevels]

Would you be able to put together a demonstration in "raw" julia, without using ForwardDiff at all?

I wasn't able to reproduce this when the involved number types are Base number types; or do you mean an example with a simpler wrapper type than GradientNumber, on which some inlined arithmetic functions are defined?

[timholy]

Yes, or just a version for ForwardDiff itself that strips the operations down to the essentials. If it can be reduced to ~20 loc or something, that might make it a lot easier for folks. (I frequently do that when I report bugs, but in this case you understand ForwardDiff far better than I.)

[timholy]

Ah, but see #13355!

[JeffBezanson]

I am now seeing much worse performance here, and it does not seem to be fixed by #13355.

[pao]

Reopening based on @JeffBezanson's comment from yesterday--this got autoclosed by #13355.

[yuyichao]

Is this a dup of #12219 ?

[simonster]

The original issue was kind of the opposite of #12219: we had type information, but we weren't inlining aggressively enough to avoid calling * with a suboptimal calling convention. It was basically the same issue as #5011, and my fix was to fix the heuristic that @JeffBezanson introduced in 1316f81, which had been broken by inference refactoring in #11274. But I think there was a (different?) regression on master, which I haven't had time to take a look at.

[simonster]

The new regression that @JeffBezanson saw may be #14294.

[vtjnash]

confirmed the new issue was #14294, not the original report here