Beta build for FlavorJulia to help resolve OpenBLAS issues

[hiccup7]

Please create a beta build for FlavorJulia to help resolve OpenBLAS issues. Build Julia like WinPython-64bit-3.4.3.2FlavorJulia, except:
a) add MAX_STACK_ALLOC=2048 to the make options for OpenBLAS, and
b) use the develop branch of OpenBLAS

This build will have these benefits:

1. Potentially fixes this bug in WinPython: OpenBLAS 6 times slower than MKL on DGEMV() OpenMathLib/OpenBLAS#532
2. Supports me testing this fix: OpenBLAS 4 times slower than MKL on DDOT() OpenMathLib/OpenBLAS#530
3. Provides an improved OpenBLAS to be used in WinPython 2015-04

By the way, please keep MKL BLAS for NumPy and SciPy in the near future, as OpenBLAS currently has a number of performance problems compared to MKL BLAS.

[stonebig]

If you are speaking of the OpenBlas as implemented by Julia, it's a Julia's team question.
I can / you can only try:

• to open julia console,
• to refresh packages (and hopefully OpenBlas of julia) by
  Pkg.update()

I don't know how you can introduce a beta OpenBlas in Julia for windows otherwise

[stonebig]

Besides that:

• Julia 0.3.x had some performance weak points, that may be resolved in Julia 0.4,
• maybe Julia team won't put an OpenBlas beta in anything else than Julia0.4dev,
• your performance issue seems "detectable" in a "pure" Julia environnement.

==> Wouldn't it be simpler:

• to ask julia team if they can put the OpenBlas beta on their Julia0.4 repos,
• install a julia0.4dev directly downloaded from julialang.org to get this beta and do the performance test you need ?

[stonebig]

(and the download count of FlavorJulia is disappointing, so I think I won't do another one before Julia 0.4beta3)

[hiccup7]

In the Julia console, execute Pkg.status(). Notice that there is no separate package for OpenBLAS or BLAS. Doesn't this mean that Julia must be rebuilt from source in order to update OpenBLAS?

Do you build Julia using the Windows Self-Extracting Archive (.exe) from here?
http://julialang.org/downloads/

My performance issues are indeed in a "pure" Julia environment, but my code calls Python modules too. So WinPython is a great environment to provide a happy marriage of Julia and Python.

[stonebig]

Well, you're right. Eating my own dog food, I don't see Julia fetching refreshed DLLs.

[stonebig]

and maybe:

using WinRPM
WinRPM.update()

but I'm not sure what it downloads.... https://groups.google.com/forum/#!forum/julia-users may be the best way to go for advice.

[stonebig]

I see Julia team picked up your request: JuliaLang/julia#10780

[hiccup7]

@stonebig , Soon, I will be able to get the OpenBLAS fixes in the Julia v0.4-pre nightly build.

When you build WinPython FlavorJulia, do you build Julia using the latest released version of the Windows Self-Extracting Archive (.exe) from here?
http://julialang.org/downloads/

[stonebig]

Yes, I do roughly the below procedure in an existing winpython (without julia, or with existing settings\.julia and tools\julia removed):
http://nbviewer.ipython.org/github/winpython/winpython_afterdoc/blob/master/examples/installing_julia_and_ijulia.ipynb

then, replace .settings\ipython\kernels\julia\kernel.json by this (to have a movable winpython+julia):

{
"argv": [
"[WINPYDIR]\\..\\tools\\julia\\bin\\julia",
"-i",
"-F",
"[WINPYDIR]\\..\\settings\\.julia\\v0.3\\IJulia\\src\\kernel.jl",
"{connection_file}"
],
"display_name": "Julia",
"language": "julia"
} 

[hiccup7]

Thanks for the details!