Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Removing Caveats #17

Merged
merged 1 commit into from
Nov 28, 2019
Merged

Removing Caveats #17

merged 1 commit into from
Nov 28, 2019

Conversation

aminya
Copy link
Contributor

@aminya aminya commented Oct 11, 2019

  • Downstream binary libraries that depend on BLAS such as SuiteSparse (solving sparse linear systems) and ARPACK (for large scale eigenvalue computations) will currently not work once MKL.jl has been installed. We are working on removing these limitations.

Tested on #14 and there are no issues.

julia> using LinearAlgebra; BLAS.vendor()
:mkl

julia> using Arpack, SparseArrays

julia> (A, B) = kron(A, B);

julia> function Hub_ED(U, μ, L,)
           id = [1 0; 0 1]
           c = [0 0; 1 0]

           c_up = c  id
           c_dn = id  c
           id² = id  id

           n_up = c_up' * c_up
           n_dn = c_dn' * c_dn

           Û =  U*(n_up * n_dn) - μ*(n_up + n_dn)

           c_dg_up(i) = foldl(, sparse.([i==j ? c_up' : id² for j in 1:L]))
           cup(i)     = foldl(, sparse.([i==j ? c_up  : id² for j in 1:L]))
           c_dg_dn(i) = foldl(, sparse.([i==j ? c_dn' : id² for j in 1:L]))
           cdn(i)     = foldl(, sparse.([i==j ? c_dn  : id² for j in 1:L]))
           Ûf(i)      = foldl(, sparse.([i==j ? Û     : id² for j in 1:L]))

           function c_dg_c(i)
               out = c_dg_up(i)*cup(i+1) + c_dg_dn(i)*cdn(i+1)
               out + out'
           end

           H = -sum(c_dg_c, 1:(L-1)) + sum(Ûf, 1:L)
           @show typeof(H)
           (λ, ϕ), t = @timed Arpack.eigs(H, nev=1, which=:SR)
           λ, ϕ, t
       end
Hub_ED (generic function with 1 method)

julia> Hub_ED(1, 1, 8)
typeof(H) = SparseMatrixCSC{Int64,Int64}
([-15.813000555255263], [-1.949487752371787e-18; -2.0668833887408264e-18;  ; 2.035974575802704e-18; 1.216026630767711e-18], 0.340870967)
  • It is not possible to revert the effect of install MKL.jl. To return to OpenBLAS, it is necessary to reinstall Julia.

Enabling openBlas is possible using MKL.enable_openblas_startup()

Tested on Windows:

julia> using MKL

julia> MKL.enable_openblas_startup()
[ Info: Replacing libblas_name in C:\Julia-1.3.0-rc3\bin\..\share\julia\base\build_h.jl
┌ Info: Building `inference.o`:
└ `'C:\Julia-1.3.0-rc3\bin\julia.exe' -C native --output-ji 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\basecompiler.ji' --output-o 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\basecompiler.o' compiler/compiler.jl`
essentials.jl
ctypes.jl
generator.jl
reflection.jl
options.jl
promotion.jl
tuple.jl
pair.jl
traits.jl
range.jl
expr.jl
error.jl
bool.jl
number.jl
int.jl
operators.jl
pointer.jl
refvalue.jl
indices.jl
array.jl
abstractarray.jl
bitarray.jl
bitset.jl
abstractdict.jl
abstractset.jl
iterators.jl
namedtuple.jl
docs/core.jl
ordering.jl
sort.jl
compiler/utilities.jl
compiler/validation.jl
compiler/inferenceresult.jl
compiler/params.jl
compiler/inferencestate.jl
compiler/typeutils.jl
compiler/typelimits.jl
compiler/typelattice.jl
compiler/tfuncs.jl
compiler/abstractinterpretation.jl
compiler/typeinfer.jl
compiler/optimize.jl
compiler/ssair/driver.jl
compiler/ssair/ir.jl
compiler/ssair/domtree.jl
compiler/ssair/slot2ssa.jl
compiler/ssair/queries.jl
compiler/ssair/passes.jl
compiler/ssair/inlining.jl
compiler/ssair/verify.jl
compiler/ssair/legacy.jl
compiler/bootstrap.jl
┌ Info: Building `sys.o`:
└ `'C:\Julia-1.3.0-rc3\bin\julia.exe' -C native --output-ji 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\sys.ji' --output-o 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\sys.o' -J 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\basecompiler.ji' --startup-file=no sysimg.jl`
coreio.jl
exports.jl
essentials.jl
ctypes.jl
gcutils.jl
generator.jl
reflection.jl
options.jl
promotion.jl
tuple.jl
expr.jl
pair.jl
traits.jl
range.jl
error.jl
bool.jl
number.jl
int.jl
operators.jl
pointer.jl
refvalue.jl
refpointer.jl
checked.jl
indices.jl
array.jl
abstractarray.jl
subarray.jl
views.jl
baseext.jl
ntuple.jl
abstractdict.jl
iterators.jl
namedtuple.jl
hashing.jl
rounding.jl
float.jl
twiceprecision.jl
complex.jl
rational.jl
multinverses.jl
abstractarraymath.jl
arraymath.jl
simdloop.jl
reduce.jl
reshapedarray.jl
reinterpretarray.jl
bitarray.jl
bitset.jl
multimedia.jl
some.jl
dict.jl
abstractset.jl
set.jl
char.jl
strings/basic.jl
strings/string.jl
strings/substring.jl
build_h.jl
version_git.jl
osutils.jl
c.jl
io.jl
iobuffer.jl
intfuncs.jl
strings/strings.jl
strings/search.jl
strings/unicode.jl
strings/util.jl
strings/io.jl
parse.jl
shell.jl
regex.jl
pcre.jl
show.jl
arrayshow.jl
methodshow.jl
cartesian.jl
multidimensional.jl
permuteddimsarray.jl
broadcast.jl
missing.jl
version.jl
sysinfo.jl
libc.jl
env.jl
linked_list.jl
condition.jl
threads.jl
lock.jl
task.jl
weakkeydict.jl
logging.jl
libuv.jl
uv_constants.jl
asyncevent.jl
iostream.jl
stream.jl
filesystem.jl
cmd.jl
process.jl
grisu/grisu.jl
secretbuffer.jl
floatfuncs.jl
math.jl
reducedim.jl
accumulate.jl
ordering.jl
sort.jl
fastmath.jl
Enums.jl
gmp.jl
mpfr.jl
combinatorics.jl
hashing2.jl
irrationals.jl
mathconstants.jl
printf.jl
meta.jl
channels.jl
deepcopy.jl
download.jl
summarysize.jl
errorshow.jl
stacktraces.jl
initdefs.jl
threadcall.jl
uuid.jl
loading.jl
util.jl
asyncmap.jl
experimental.jl
deprecated.jl
docs\basedocs.jl
client.jl
docs\Docs.jl
Base  ─────────── 25.654957 seconds
Base64  ─────────  3.794240 seconds
CRC32c  ─────────  0.010188 seconds
SHA  ────────────  0.181228 seconds
FileWatching  ───  0.091911 seconds
Unicode  ────────  0.010193 seconds
Mmap  ───────────  0.074751 seconds
Serialization  ──  0.862091 seconds
Libdl  ──────────  0.030937 seconds
Markdown  ───────  1.054587 seconds
LibGit2  ────────  2.649440 seconds
Logging  ────────  0.506780 seconds
Sockets  ────────  1.877789 seconds
Printf  ─────────  0.012369 seconds
Profile  ────────  0.153111 seconds
Dates  ──────────  2.098834 seconds
DelimitedFiles  ─  0.104160 seconds
Random  ─────────  0.525311 seconds
UUIDs  ──────────  0.019996 seconds
Future  ─────────  0.011412 seconds
LinearAlgebra  ── 12.891522 seconds
SparseArrays  ───  6.296322 seconds
SuiteSparse  ────  1.802302 seconds
Distributed  ────  6.485953 seconds
SharedArrays  ───  0.222248 seconds
Pkg  ──────────── 13.223301 seconds
Test  ───────────  0.961266 seconds
REPL  ───────────  1.154462 seconds
Statistics  ─────  0.162380 seconds
Stdlibs total  ── 57.315055 seconds
Sysimage built. Summary:
Total ───────  82.973635 seconds
Base: ───────  25.654957 seconds 30.9194%
Stdlibs: ────  57.315055 seconds 69.0762%
Build shared library "C:\\Users\\yahyaaba\\.julia\\packages\\PackageCompiler\\CJQcs\\sysimg\\backup\\native\\sys.dll":
  `'C:\Users\yahyaaba\.julia\packages\WinRPM\Y9QdZ\deps\usr\x86_64-w64-mingw32\sys-root\mingw\bin\gcc.exe' --sysroot 'C:\Users\yahyaaba\.julia\packages\WinRPM\Y9QdZ\deps\usr\x86_64-w64-mingw32\sys-root' -shared '-DJULIAC_PROGRAM_LIBNAME="C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\sys.dll"' -o 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\sys.dll' -Wl,--whole-archive 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\sys.o' -Wl,--no-whole-archive -std=gnu99 '-IC:\Julia-1.3.0-rc3\include\julia' -DJULIA_ENABLE_THREADING=1 '-LC:\Julia-1.3.0-rc3\bin' -Wl,--stack,8388608 -ljulia -lopenlibm -m64 -Wl,--export-all-symbols`
[ Info: Copying system image: C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\sys.dll to C:\Julia-1.3.0-rc3\lib\julia\sys.dll

After restart:

julia> using LinearAlgebra

julia> BLAS.vendor()
:openblas64
  • The current version of MKL.jl doesn't have access to the same precompilation information as the official binaries and source builds so the REPL will have more latency after MKL.jl has been installed. It should be possible to fix this once Julia 1.1 has been released. See https://github.com/JuliaComputing/MKL.jl/issues/1.

Tried on Windows and many different installations, only the first boot may take a little longer (less than 20 seconds). The next boots are just the same as normal Julia (less than 5 seconds).

- Downstream binary libraries that depend on BLAS such as SuiteSparse (solving sparse linear systems) and ARPACK (for large scale eigenvalue computations) will currently not work once MKL.jl has been installed. We are working on removing these limitations.



Tested on JuliaLinearAlgebra#14 and there are no issues.
```julia
julia> using LinearAlgebra; BLAS.vendor()
:mkl

julia> using Arpack, SparseArrays

julia> ⊗(A, B) = kron(A, B);

julia> function Hub_ED(U, μ, L,)
           id = [1 0; 0 1]
           c = [0 0; 1 0]

           c_up = c ⊗ id
           c_dn = id ⊗ c
           id² = id ⊗ id

           n_up = c_up' * c_up
           n_dn = c_dn' * c_dn

           Û =  U*(n_up * n_dn) - μ*(n_up + n_dn)

           c_dg_up(i) = foldl(⊗, sparse.([i==j ? c_up' : id² for j in 1:L]))
           cup(i)     = foldl(⊗, sparse.([i==j ? c_up  : id² for j in 1:L]))
           c_dg_dn(i) = foldl(⊗, sparse.([i==j ? c_dn' : id² for j in 1:L]))
           cdn(i)     = foldl(⊗, sparse.([i==j ? c_dn  : id² for j in 1:L]))
           Ûf(i)      = foldl(⊗, sparse.([i==j ? Û     : id² for j in 1:L]))

           function c_dg_c(i)
               out = c_dg_up(i)*cup(i+1) + c_dg_dn(i)*cdn(i+1)
               out + out'
           end

           H = -sum(c_dg_c, 1:(L-1)) + sum(Ûf, 1:L)
           @show typeof(H)
           (λ, ϕ), t = @timed Arpack.eigs(H, nev=1, which=:SR)
           λ, ϕ, t
       end
Hub_ED (generic function with 1 method)

julia> Hub_ED(1, 1, 8)
typeof(H) = SparseMatrixCSC{Int64,Int64}
([-15.813000555255263], [-1.949487752371787e-18; -2.0668833887408264e-18; … ; 2.035974575802704e-18; 1.216026630767711e-18], 0.340870967)
```



- It is not possible to revert the effect of install MKL.jl. To return to OpenBLAS, it is necessary to reinstall Julia.
Enabling openBlas is possible using `MKL.enable_openblas_startup()`


Tested on Windows:
```julia
julia> using MKL

julia> MKL.enable_openblas_startup()
[ Info: Replacing libblas_name in C:\Julia-1.3.0-rc3\bin\..\share\julia\base\build_h.jl
┌ Info: Building `inference.o`:
└ `'C:\Julia-1.3.0-rc3\bin\julia.exe' -C native --output-ji 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\basecompiler.ji' --output-o 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\basecompiler.o' compiler/compiler.jl`
essentials.jl
ctypes.jl
generator.jl
reflection.jl
options.jl
promotion.jl
tuple.jl
pair.jl
traits.jl
range.jl
expr.jl
error.jl
bool.jl
number.jl
int.jl
operators.jl
pointer.jl
refvalue.jl
indices.jl
array.jl
abstractarray.jl
bitarray.jl
bitset.jl
abstractdict.jl
abstractset.jl
iterators.jl
namedtuple.jl
docs/core.jl
ordering.jl
sort.jl
compiler/utilities.jl
compiler/validation.jl
compiler/inferenceresult.jl
compiler/params.jl
compiler/inferencestate.jl
compiler/typeutils.jl
compiler/typelimits.jl
compiler/typelattice.jl
compiler/tfuncs.jl
compiler/abstractinterpretation.jl
compiler/typeinfer.jl
compiler/optimize.jl
compiler/ssair/driver.jl
compiler/ssair/ir.jl
compiler/ssair/domtree.jl
compiler/ssair/slot2ssa.jl
compiler/ssair/queries.jl
compiler/ssair/passes.jl
compiler/ssair/inlining.jl
compiler/ssair/verify.jl
compiler/ssair/legacy.jl
compiler/bootstrap.jl
┌ Info: Building `sys.o`:
└ `'C:\Julia-1.3.0-rc3\bin\julia.exe' -C native --output-ji 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\sys.ji' --output-o 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\sys.o' -J 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\basecompiler.ji' --startup-file=no sysimg.jl`
coreio.jl
exports.jl
essentials.jl
ctypes.jl
gcutils.jl
generator.jl
reflection.jl
options.jl
promotion.jl
tuple.jl
expr.jl
pair.jl
traits.jl
range.jl
error.jl
bool.jl
number.jl
int.jl
operators.jl
pointer.jl
refvalue.jl
refpointer.jl
checked.jl
indices.jl
array.jl
abstractarray.jl
subarray.jl
views.jl
baseext.jl
ntuple.jl
abstractdict.jl
iterators.jl
namedtuple.jl
hashing.jl
rounding.jl
float.jl
twiceprecision.jl
complex.jl
rational.jl
multinverses.jl
abstractarraymath.jl
arraymath.jl
simdloop.jl
reduce.jl
reshapedarray.jl
reinterpretarray.jl
bitarray.jl
bitset.jl
multimedia.jl
some.jl
dict.jl
abstractset.jl
set.jl
char.jl
strings/basic.jl
strings/string.jl
strings/substring.jl
build_h.jl
version_git.jl
osutils.jl
c.jl
io.jl
iobuffer.jl
intfuncs.jl
strings/strings.jl
strings/search.jl
strings/unicode.jl
strings/util.jl
strings/io.jl
parse.jl
shell.jl
regex.jl
pcre.jl
show.jl
arrayshow.jl
methodshow.jl
cartesian.jl
multidimensional.jl
permuteddimsarray.jl
broadcast.jl
missing.jl
version.jl
sysinfo.jl
libc.jl
env.jl
linked_list.jl
condition.jl
threads.jl
lock.jl
task.jl
weakkeydict.jl
logging.jl
libuv.jl
uv_constants.jl
asyncevent.jl
iostream.jl
stream.jl
filesystem.jl
cmd.jl
process.jl
grisu/grisu.jl
secretbuffer.jl
floatfuncs.jl
math.jl
reducedim.jl
accumulate.jl
ordering.jl
sort.jl
fastmath.jl
Enums.jl
gmp.jl
mpfr.jl
combinatorics.jl
hashing2.jl
irrationals.jl
mathconstants.jl
printf.jl
meta.jl
channels.jl
deepcopy.jl
download.jl
summarysize.jl
errorshow.jl
stacktraces.jl
initdefs.jl
threadcall.jl
uuid.jl
loading.jl
util.jl
asyncmap.jl
experimental.jl
deprecated.jl
docs\basedocs.jl
client.jl
docs\Docs.jl
Base  ─────────── 25.654957 seconds
Base64  ─────────  3.794240 seconds
CRC32c  ─────────  0.010188 seconds
SHA  ────────────  0.181228 seconds
FileWatching  ───  0.091911 seconds
Unicode  ────────  0.010193 seconds
Mmap  ───────────  0.074751 seconds
Serialization  ──  0.862091 seconds
Libdl  ──────────  0.030937 seconds
Markdown  ───────  1.054587 seconds
LibGit2  ────────  2.649440 seconds
Logging  ────────  0.506780 seconds
Sockets  ────────  1.877789 seconds
Printf  ─────────  0.012369 seconds
Profile  ────────  0.153111 seconds
Dates  ──────────  2.098834 seconds
DelimitedFiles  ─  0.104160 seconds
Random  ─────────  0.525311 seconds
UUIDs  ──────────  0.019996 seconds
Future  ─────────  0.011412 seconds
LinearAlgebra  ── 12.891522 seconds
SparseArrays  ───  6.296322 seconds
SuiteSparse  ────  1.802302 seconds
Distributed  ────  6.485953 seconds
SharedArrays  ───  0.222248 seconds
Pkg  ──────────── 13.223301 seconds
Test  ───────────  0.961266 seconds
REPL  ───────────  1.154462 seconds
Statistics  ─────  0.162380 seconds
Stdlibs total  ── 57.315055 seconds
Sysimage built. Summary:
Total ───────  82.973635 seconds
Base: ───────  25.654957 seconds 30.9194%
Stdlibs: ────  57.315055 seconds 69.0762%
Build shared library "C:\\Users\\yahyaaba\\.julia\\packages\\PackageCompiler\\CJQcs\\sysimg\\backup\\native\\sys.dll":
  `'C:\Users\yahyaaba\.julia\packages\WinRPM\Y9QdZ\deps\usr\x86_64-w64-mingw32\sys-root\mingw\bin\gcc.exe' --sysroot 'C:\Users\yahyaaba\.julia\packages\WinRPM\Y9QdZ\deps\usr\x86_64-w64-mingw32\sys-root' -shared '-DJULIAC_PROGRAM_LIBNAME="C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\sys.dll"' -o 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\sys.dll' -Wl,--whole-archive 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\sys.o' -Wl,--no-whole-archive -std=gnu99 '-IC:\Julia-1.3.0-rc3\include\julia' -DJULIA_ENABLE_THREADING=1 '-LC:\Julia-1.3.0-rc3\bin' -Wl,--stack,8388608 -ljulia -lopenlibm -m64 -Wl,--export-all-symbols`
[ Info: Copying system image: C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\sys.dll to C:\Julia-1.3.0-rc3\lib\julia\sys.dll
```
After restart:
```julia
julia> using LinearAlgebra

julia> BLAS.vendor()
:openblas64
```

- The current version of MKL.jl doesn't have access to the same precompilation information as the official binaries and source builds so the REPL will have more latency after MKL.jl has been installed. It should be possible to fix this once Julia 1.1 has been released. See JuliaLinearAlgebra#1.

Tried on Windows and many different installations, only the first boot may take a little longer (less than 20 seconds). The next boots is just the same as normal Julia (less than 5 seconds).
@KristofferC
Copy link
Contributor

Tried on Windows and many different installations, only the first boot may take a little longer (less than 20 seconds). The next boots are just the same as normal Julia (less than 5 seconds).

This is not really true. Here is without MKL on my linux machine:

➜ time ./julia -e ''
0.07s user 0.05s system 106% cpu 0.108 total

➜  time ./julia -e 'import Pkg; Pkg.status()'
...
0.32s user 0.13s system 232% cpu 0.195 total

And with MKL:

➜  time julia -e ''
0.84s user 0.28s system 180% cpu 0.621 total

➜ time julia -e 'import Pkg; Pkg.status()'
...
3.92s user 0.29s system 113% cpu 3.707 total

@aminya
Copy link
Contributor Author

aminya commented Oct 12, 2019

Tried on Windows and many different installations, only the first boot may take a little longer (less than 20 seconds). The next boots are just the same as normal Julia (less than 5 seconds).

This is not really true. Here is without MKL on my linux machine:

➜ time ./julia -e ''
0.07s user 0.05s system 106% cpu 0.108 total

➜  time ./julia -e 'import Pkg; Pkg.status()'
...
0.32s user 0.13s system 232% cpu 0.195 total

And with MKL:

➜  time julia -e ''
0.84s user 0.28s system 180% cpu 0.621 total

➜ time julia -e 'import Pkg; Pkg.status()'
...
3.92s user 0.29s system 113% cpu 3.707 total

Yeah, I said less than 5 seconds. For me, 3 seconds is really nothing. The way the Readme is written implies that it takes minutes to load the Julia

@aminya
Copy link
Contributor Author

aminya commented Oct 12, 2019

I think we can add another caveat instead of current ones, which is losing Python interoperability and the chance of crashing when calling Python libraries caused by #11, JuliaLang/PackageCompiler.jl#154, and JuliaLang/julia#30828

@andreasnoack andreasnoack merged commit 512b270 into JuliaLinearAlgebra:master Nov 28, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants