sum(a) is now 30% slower than NumPy

[stevengj]

I was updating my performance-optimization lecture notes from last year to Julia 1.0, which start with a comparison of C, Python, and Julia sum functions, and I noticed something odd:

Both the Julia sum(::Vector{Float64}) function and the NumPy sum function are faster than last year (yay for compiler improvements?). Last year, Julia and NumPy sum had almost identical speed, but now the NumPy sum function is now about 30% faster than Julia.

I'm running a 2016 Intel core i7, the same as last year. So apparently the NumPy sum function has gotten some new optimization that we don't have? (I did switch from Python 2 to Python 3; I'm using the Conda Python.) Some kind of missing SIMD optimization?

I'm not so concerned about sum per se, but this is a pretty basic function — if we are leaving 30% on the table here, then we might be missing performance opportunities in many other places too.

[andreasnoack]

I (and the audience) noticed this last week when I used the notebook in a presentation. Looks like Intel has been looking into this since I suspect this could be numpy/numpy#10251.

[KristofferC]

This PR numpy/numpy#11113 removes all the intrinsic stuff and says the compiler can do it by itself with the same performance. So not sure why we get less with LLVM then.

[stevengj]

Note that numpy11113 does not seem to specifically benchmark sum, so it could be that the intrinsics are still better than LLVM there? Or maybe Conda's numpy is using a different compiler (icc?) or different options?

[StefanKarpinski]

I've been seeing the same thing on a 2018 2.7 GHz Intel Core i7, so this is something that's widespread across a pretty wide range of Intel CPUs. Makes the "variations on sum" / "Julia is fast" summation notebook a bit of a sad trumpet demo since NumPy wins.

[JeffreySarnoff]

Are the summed values equal?

[SyxP]

Are the summed values equal?

Yes. They are within error for floating points, but not exactly equal as floats.

[andreasnoack]

Just an update. I don't think this is caused by the PR I linked to. I tried out older versions of Numpy and they are all (back to and including 1.11) as fast.

[tkf]

FYI, if you are using conda to install Numpy 1.11, you may be installing a very new build https://anaconda.org/anaconda/numpy/files?version=1.11.3 (says 6 days old). Anaconda could be using a newer/better compiler?

[andreasnoack]

I built the Numpy version from source with the default gcc (Apple's LLVM based version). The odd thing is that Numpy is also significantly faster than the C version regardless of flags I try to use.

[JeffreySarnoff]

Can you see the code generated by Numpy?
Annotating a Julia sum function's loop with @simd gives that ~30% speedup on my machine (until the number of summands > ~35,000, when the speedup starts to decrease).

[KristofferC]

If you look in the notebook you can see @simd is (of course) used.

[JeffreySarnoff]

I meant "do we know the instructions that Numpy generates to perform the summation and how those instructions (emitted assembler that is the summation loop) differ from what Julia via LLVM executes?"

[AzamatB]

Would be interesting to see whether updating Julia's LLVM to the latest version would solve the issue.

[c42f]

Oddly enough, the julia builtin sum doesn't vectorize on my machine. See also #30320 where there is some discussion of vectorization in the current mapreduce implementation.

[c42f]

sum doesn't vectorize on my machine

Sorry, ignore that — I misread the assembly.

[hycakir]

On my notebook (i7-6700HQ, not AVX-512), using conda distributed Numpy sum does not give any performance benefit.

I believe this is an AVX-512 optimization issue. Perhaps conda Numpy distributions are compiled with icc (although there is also an intel numpy ) which presumably optimizes better for AVX-512 targets.

Looking at the output from gcc 8.2 vs icc 19.0 following the link below, icc generates instructions using 512-bit registers (zmms) while gcc or clang does not.

https://godbolt.org/z/we75gC

[chriselrod]

If you use `-mprefer-vector-width=512` and `-ffast-math` then clang 7.0 and gcc 8.2 will use zmm registers. https://godbolt.org/z/r8lQTH gcc's code looks awkward, while clang has a rather extreme amount of unrolling (by a factor of 128 doubles!), then another body that unrolls by a factor of 32. `ssh` is failing me right now, but I could benchmark on a skylake-x architecture in a couple days when I get home. Julia's `@simd` code tends to match Clang's. I would guess that this wins for long loops by a small margin (icc close behind), while icc is much faster for small loops. And that gcc tends slower than both.

…

On Mon, Dec 31, 2018 at 4:23 PM Hamza Yusuf Çakır ***@***.***> wrote: On my notebook (i7-6700HQ), using conda distributed Numpy sum does not give any performance benefit. Seems to be that conda Numpy distributions are compiled with icc. I believe this is a AVX-512 optimization issue. Looking at the output from gcc 8.2 vs icc 19.0 following the link below, icc uses 512-bit registers while gcc or clang does not. https://godbolt.org/z/we75gC — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#30290 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHq8U_YNQ5D7AKwG9srMt-td9ofYZAupks5u-o5fgaJpZM4ZGktk> .

-- https://github.com/chriselrod?tab=repositories https://www.linkedin.com/in/chris-elrod-9720391a/

[hycakir]

Ah, I see. icc's default being -fp-model fast=1 misled me. Julia also makes use of AVX-512 registers with the same sum implementation with @simd annotation.

[nlw0]

Would be interesting to see whether updating Julia's LLVM to the latest version would solve the issue.

I have seen other vectorization issues where clang 6.0 produces vectorized code and Julia doesn't, for instance #29445 and #31442 .

[vchuravy]

One idea might be to check whether this happens with numpy from Conda or numpy from pip. The one from Conda has Intel specific code and uses VML

[carstenbauer]

I just noticed this as well on 1.5.3 (~25% faster numpy). On julia#master (i.e. 1.7-dev) I find that numpy is "only" ~12% faster. Still unfortunate that we are slower here (in particular for demonstration/selling purposes).

[StefanKarpinski]

I'm not sure if there's much value to keeping this open: this is generally a symptom of LLVM not generating as well optimized simd code for newer hardware as numpy's hand coded kernels. We generally catch up as soon as llvm learns how to do as well as the hand written code but there's always newer hardware. Not sure what is actionable.

[vtjnash]

I love closing issues

[chriselrod]

I'm not sure if there's much value to keeping this open: this is generally a symptom of LLVM not generating as well optimized simd code for newer hardware as numpy's hand coded kernels.

Hand coded kernels of course adds a lot of maintenance burden. LLVM is normally way (months or years) ahead of OpenBLAS in supporting new architectures.
For example, it wasn't until last year that OpenBLAS really started supporting AVX512, but my laptop (which has AVX512) still isn't supported in the latest release and still uses Nehalem kernels.

So just stating the obvious of why I don't think hand coded kernels are the best idea.