Shipping MKL with julia

[ViralBShah]

Intel (India) has graciously given us MKL licenses for Mac, Windows, and Linux. Although we are unable to do much in the 0.2 release, we should try and include MKL in the nightlies post 0.2, and in future releases.

[StefanKarpinski]

That is great. How do they give us licenses? Do we get some kind of certificate? An email that says it's ok for us to ship with MKL?

[mlubin]

How will this work with linux package managers? @sebastien-villemot

[kmsquire]

I don't think that "given us MKL licenses" translates in to "letting us distributed MKL for the whole world to use" for software that has a $500US licensing fee.

@ViralBShah, I'm assuming you really meant for testing and development use, and not for distribution?

[kmsquire]

Or, it could be the case that distributing it is fine, but the user has to have a license key on her system in order to use it?

That would certainly not work for Debian, but I don't think anyone is suggesting permanently replace OpenBLAS with MKL.

[ViralBShah]

The way MKL licenses work (discussed in another thread too) is that you need to buy licenses for all the developers, but MKL can then be shipped by those developers - even for open source projects. They gave similar licenses to scipy guys too.

This would certainly not work for debian, and we need to continue supporting openblas for other architectures such as AMD.

[kmsquire]

So to clarify, you're saying that end users would be able to use julia built with MKL without purchasing a license? And that would be binary only--no source distribution of MKL, of course?

Do you have a pointer to the other thread?

[blakejohnson]

@StefanKarpinski presumably, you will receive a license file for each platform.

@kmsquire Intel doesn't provide the source for MKL, all you ever get are binaries (and header files).

[kmsquire]

Okay, makes sense now, thanks.

[ViralBShah]

I requested licenses for myself, Jameson, Keno and Elliot - people who are currently packaging binaries. For those who need licenses for development, there are free licenses for personal use available on linux anyways.

[svillemot]

@mlubin Since MKL is non-free, I don't feel very much involved in this discussion :)

[vtjnash]

I thought there was already an issue open for this.

[stevengj]

How does the performance of MKL compare to OpenBLAS these days?

[blakejohnson]

@stevengj See #3965 for some recent benchmarks.

[lindahua]

What about we use OpenBLAS as the default backend, and have MKL as an available option -- people can turn it on when they prefer MKL?

[staticfloat]

In Linux-land, it should be theoretically possible to switch BLAS backends using symlinks; as long as ARPACK, SuiteSparse, and Julia are all expecting JULIA_HOME/lib/libblas.so to provide everything, you can just swap out where that symlink points to in order to load OpenBLAS, ATLAS, etc.... In fact, that's how debian's alternatives tool works.

With our libgfortblas, we might be able to do the same on OSX, but it's not immediately obvious to me that something won't break. It's something that might be a good project for someone looking to do something relatively simple; I'll open an "up for grabs" issue.

EDIT: Issue opened.

[ViralBShah]

@lindahua Yes, we should certainly plan on shipping both.

[StefanKarpinski]

Ideally, we would ship with both OpenBLAS and MKL and allow choosing via a command-line flag to julia and/or an environment variable. The relevant tests should then be run using both BLASes when doing make testall.

[vtjnash]

The problem with this idea comes from the fact that OpenBLAS inherits a different calling convention from gfortran than either MKL or Accelerate (which seem to share the same calling convention) -- e.g. different from Apple or Intel. Julia inherited the gfortran calling convention, for use with, well, gfortran, so it only works with OpenBLAS currently.

[simonbyrne]

As I understand it, wouldn't packaging with MKL conflict with GPL (since julia as a package becomes GPL once we include GPL dependencies)?

[StefanKarpinski]

That may be an issue. We might have to ship a non-GPL MKL bundle or something like that. We should probably ask someone with more legal expertise about this.

[StefanKarpinski]

Alternatively once we get rid of Rmath and readline, I believe we will be non-GPL entirely and then we will be free to ship with MKL. Since both of those things are likely to happen soon, maybe we can just wait.

[johnmyleswhite]

I thought FFTW was GPL. We really need a list of all GPL components.

[StefanKarpinski]

Ah, yes, FFTW. We may need to move FFTW into a package for this reason and make it possible to drop in Intel's fft libraries with the same interface.

[StefanKarpinski]

We could also probably get a license to ship FFTW, it's just that others would not be allowed to ship Julia under that same arrangement.

[ViralBShah]

The SuiteSparse solvers - UMFPACK and parts of CHOLMOD (the supernodal stuff) are GPL. The rest of CHOLMOD is LGPL, and we have some translated code from CSparse that is also LGPL.

[StefanKarpinski]

So it seems like we may not be able to ship with MKL until we get rid of readline and Rmath, separate FFTW into its own package or get a different license for it, and either replace the GPL parts of SuiteSparse with something else or move it into an add-on package. LGPL stuff is fine.

[ViralBShah]

Yup. We could have two distributions - GPL and non-GPL.

[simonbyrne]

Might have to be careful with translated LGPL stuff as well: that becomes a derivative work, and might no longer be covered by the linking exception.

[blakejohnson]

We have an existence proof that all this should be possible: MATLAB. It contains MKL, FFTW, and SuiteSparse, and is released under a proprietary license. I guess we should contact the FFTW and SuiteSparse folks to see if they will extend us an LGPL license.

[ViralBShah]

The Mathworks has purchased licenses from Intel for MKL, from MIT for FFTW (non-gpl), and from UFL for SuiteSparse (non-gpl). The universities are unlikely to let go of the revenues. UMFPACK used to be LGPL until a few months back. CHOLMOD is LGPL largely, but important parts such as the supernodal solver, IIRC, are GPL. In theory, we could purchase these licenses so that downloads from the julialang.org website could ship all these packages for user convenience - however they would not be re-distributable, etc.

[StefanKarpinski]

I think we should purchase those licenses so that we can do this and make it all just work. However, we will also need a redistributable version as well and label them very clearly.

[ViralBShah]

I personally feel that we would better off putting all our efforts into making OpenBLAS better. @blakejohnson 's performance comparison already showed us that MKL is not strictly better than openblas in all cases. I am going to close this issue.

[ViralBShah]

We certainly will continue to ensuring that julia can be built with MKL - just not distributing it.

[jtravs]

It should also be noted that Intel have a history of crippling MKL on non-Intel CPU's. In general I think the idea of a two tier Julia where only some people can compile binaries with extra performance through proprietary libraries would be a big negative compared to optimising for the best all round performance with open source tools. One of the great aspects of Julia versus Matlab is its openness which makes it perfect for open computational science (i.e. where the full code is auditable and available for reproduction). Just an outside perspective (from someone who runs Julia on thousands of cpus for science and has to compile form source).

[ViralBShah]

I would love it if papers start publishing their codes as Julia packages - that would be pretty transparent and reproducible. Pretty cool that you are running julia on thousands of cores. Have you published anything about it?

[StefanKarpinski]

@jtravs: That was our conclusion too – it's just not worth it. If MKL was universally better than OpenBLAS it might make more sense, but it isn't – sometimes it's better, sometimes it's worse. Better to spend more time and effort on making OpenBLAS as great as possible.

[ViralBShah]

Not to mention that openblas is soon going to have ARM support.

[blakejohnson]

@jtravs Intel actually lost a lawsuit related to that a year or two ago. As a result, the latest version of their compiler and MKL at least try to use optimized codes on non-Intel platforms. Although the docs clearly indicated that YMMV.

More generally, I think the main advantage of MKL over OpenBLAS is that It Just Works™ when using multiple cores and arbitrary problem size. I think it is definitely worth it to try to improve OpenBLAS in this regard.

[ViralBShah]

I do think that it is not too difficult to improve openblas thread performance, and we should just focus on making that happen.

[StefanKarpinski]

We're not going to stop supporting the use of MKL, we just won't ship with it. It's just too difficult legally.

[simonbyrne]

Interestingly, Revolution R Open is distributed with MKL:
http://mran.revolutionanalytics.com/open/

[StefanKarpinski]

@simonbyrne, I'm pretty sure that's illegal and violates the GPL. Then again, Revolution has a reputation for doing exactly that.

[dmbates]

There are some interesting restrictions on the use of the MKL libraries shipped with Revolution R Open. @StefanKarpinski "illegal" with regard to R and the GPL or with regard to the conditions of MKL licenses? IIRC Intel's investment arm provided seed money for Revolution so it is likely that things are kosher on the MKL front.

[StefanKarpinski]

It's completely fine on the MKL side – if you have a license for MKL, you can ship it, end of story. You cannot, however, ship a work derived from GPL code without making all of the source code of that derived work available under the GPL. Their product seems to be a derived work based on R, which is GPL. Yet Revolution is not shipping the source of MKL or their enhancements to R. Thus, they are in violation of the GPL and have no valid license for any of the GPL code they have derived their product from and could therefore be sued for copyright violation by any of the copyright holders of that GPL code. Moreover, anyone using their product is also in violation of the GPL and could be sued by the same copyright holders. The counterargument to this would seem to be claiming that their product is not a derived work, but rather multiple works, some of which are GPL and some of which are not. That seems pretty unlikely to fly in court, however. The main reason it seems that Revolution has been getting away with this is that none of the GPL copyright holders are interested in suing Revolution or their customers.

[ivarne]

none of the GPL copyright holders are interested in suing

Did you mean

none of the copyright holders of R, whom has licenced R to the world under the GPL, are interested in suing

[StefanKarpinski]

Yes, of R and any other GPL code that goes into the Revolution product. Copyright holders of other GPL software cannot sue as they have no copyright claim here.