Squeeze More Performance from Intel MKL

[RoyiAvital]

Hello,
I don't see the way MKL is built here (I see DLL Open is called, could it be no compilation is done only calling a DLL?).

But it would be great if it is built and compiled into Julia utilizing more features of Intel MKL to improve performance:

I think the Packed API and Compact API are more tricky but if they will be exposed it will be great.

MKL JIT Feature

MKL also has a new JIT feature which I think could be great addition - Intel® Math Kernel Library Improved Small Matrix Performance Using Just-in-Time (JIT) Code Generation for Matrix Multiplication (GEMM).

Remark
It seem you use Intel MKL 2019.0. while Intel MKL 2019.4 is out (OpenMP is even from 2018 release).

Update (14/10/2019)
I listed what I wanted in Benchmark MATLAB & Julia for Matrix Operations - Message 145.

[andreasnoack]

This is not currently on our to do. It's a relatively low priority since for smaller problems you'd generally like to use StaticArrays which would probably be competitive to what MKL can offer for small matrices. Please let us know if you have evidence otherwise.

[RoyiAvital]

@andreasnoack ,
Even if StaticArrays is competitive, why not enable MKL_DIRECT_CALL?
It is only a small compilation flag change.

The other features do something far beyond what you do with StaticArrays.
It uses the structure of a small problem (Say multiplying by the same matrix over and over) to get the gains of large size problem.

[RoyiAvital]

Another way to achieve (Better?) compareable performance to StaticArrays would be using MKL JIT Feature - Intel® Math Kernel Library Improved Small Matrix Performance Using Just-in-Time (JIT) Code Generation for Matrix Multiplication (GEMM).

Performance look pretty impressive:

Pay attention one could use it in 2 manners:

1. Enable JIT and let the MKL Engine do decisions.
2. Ask MKL to export a pointer to a JITted variation of a function and use it.

[RoyiAvital]

By the way, there is a simple trick to squeeze better performance from MKL on AMD CPU's.
This is done by replacing the CPU Dispatch function as guided by Agner Fog.

An example for that is given in:

https://github.com/fo40225/Anaconda-Windows-AMD

[ViralBShah]

I believe much of the discussion here is out of scope for the current MKL.jl package, and might perhaps find a bit more traction on discourse.

[RoyiAvital]

@ViralBShah , What do you mean?
Those features are about integrating MKL into a project.

[ViralBShah]

This package is mainly about providing MKL as a replacement for OpenBLAS. For further functionality, other packages can be created that leverage the presence of MKL. For now, I'm going to close this issue.

[RoyiAvital]

@ViralBShah , I think you are missing the point.
All the features above must be turned on in the integration of MKL instead of OpenBLAS. They can't be done in a different place.

For instance, the flag for direct call can't be done in later place only on the integration.