Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove suitesparse / arpack caveat? #14

Closed
MasonProtter opened this issue Oct 2, 2019 · 5 comments
Closed

Remove suitesparse / arpack caveat? #14

MasonProtter opened this issue Oct 2, 2019 · 5 comments

Comments

@MasonProtter
Copy link
Member

MasonProtter commented Oct 2, 2019

I just ran a test

julia> using LinearAlgebra; BLAS.vendor()
:mkl

julia> using Arpack, SparseArrays

julia> (A, B) = kron(A, B);

julia> function Hub_ED(U, μ, L,)
           id = [1 0; 0 1]
           c = [0 0; 1 0]

           c_up = c  id
           c_dn = id  c
           id² = id  id

           n_up = c_up' * c_up
           n_dn = c_dn' * c_dn

           Û =  U*(n_up * n_dn) - μ*(n_up + n_dn)

           c_dg_up(i) = foldl(, sparse.([i==j ? c_up' : id² for j in 1:L]))
           cup(i)     = foldl(, sparse.([i==j ? c_up  : id² for j in 1:L]))
           c_dg_dn(i) = foldl(, sparse.([i==j ? c_dn' : id² for j in 1:L]))
           cdn(i)     = foldl(, sparse.([i==j ? c_dn  : id² for j in 1:L]))
           Ûf(i)      = foldl(, sparse.([i==j ? Û     : id² for j in 1:L]))

           function c_dg_c(i)
               out = c_dg_up(i)*cup(i+1) + c_dg_dn(i)*cdn(i+1)
               out + out'
           end

           H = -sum(c_dg_c, 1:(L-1)) + sum(Ûf, 1:L)
           @show typeof(H)
           (λ, ϕ), t = @timed Arpack.eigs(H, nev=1, which=:SR)
           λ, ϕ, t
       end
Hub_ED (generic function with 1 method)

julia> Hub_ED(1, 1, 8)
typeof(H) = SparseMatrixCSC{Int64,Int64}
([-15.813000555255263], [-1.949487752371787e-18; -2.0668833887408264e-18;  ; 2.035974575802704e-18; 1.216026630767711e-18], 0.340870967)

and it looks like things are running fine with SuiteSparse and Arpack.

aminya added a commit to aminya/MKL.jl that referenced this issue Oct 11, 2019
- Downstream binary libraries that depend on BLAS such as SuiteSparse (solving sparse linear systems) and ARPACK (for large scale eigenvalue computations) will currently not work once MKL.jl has been installed. We are working on removing these limitations.



Tested on JuliaLinearAlgebra#14 and there are no issues.
```julia
julia> using LinearAlgebra; BLAS.vendor()
:mkl

julia> using Arpack, SparseArrays

julia> ⊗(A, B) = kron(A, B);

julia> function Hub_ED(U, μ, L,)
           id = [1 0; 0 1]
           c = [0 0; 1 0]

           c_up = c ⊗ id
           c_dn = id ⊗ c
           id² = id ⊗ id

           n_up = c_up' * c_up
           n_dn = c_dn' * c_dn

           Û =  U*(n_up * n_dn) - μ*(n_up + n_dn)

           c_dg_up(i) = foldl(⊗, sparse.([i==j ? c_up' : id² for j in 1:L]))
           cup(i)     = foldl(⊗, sparse.([i==j ? c_up  : id² for j in 1:L]))
           c_dg_dn(i) = foldl(⊗, sparse.([i==j ? c_dn' : id² for j in 1:L]))
           cdn(i)     = foldl(⊗, sparse.([i==j ? c_dn  : id² for j in 1:L]))
           Ûf(i)      = foldl(⊗, sparse.([i==j ? Û     : id² for j in 1:L]))

           function c_dg_c(i)
               out = c_dg_up(i)*cup(i+1) + c_dg_dn(i)*cdn(i+1)
               out + out'
           end

           H = -sum(c_dg_c, 1:(L-1)) + sum(Ûf, 1:L)
           @show typeof(H)
           (λ, ϕ), t = @timed Arpack.eigs(H, nev=1, which=:SR)
           λ, ϕ, t
       end
Hub_ED (generic function with 1 method)

julia> Hub_ED(1, 1, 8)
typeof(H) = SparseMatrixCSC{Int64,Int64}
([-15.813000555255263], [-1.949487752371787e-18; -2.0668833887408264e-18; … ; 2.035974575802704e-18; 1.216026630767711e-18], 0.340870967)
```



- It is not possible to revert the effect of install MKL.jl. To return to OpenBLAS, it is necessary to reinstall Julia.
Enabling openBlas is possible using `MKL.enable_openblas_startup()`


Tested on Windows:
```julia
julia> using MKL

julia> MKL.enable_openblas_startup()
[ Info: Replacing libblas_name in C:\Julia-1.3.0-rc3\bin\..\share\julia\base\build_h.jl
┌ Info: Building `inference.o`:
└ `'C:\Julia-1.3.0-rc3\bin\julia.exe' -C native --output-ji 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\basecompiler.ji' --output-o 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\basecompiler.o' compiler/compiler.jl`
essentials.jl
ctypes.jl
generator.jl
reflection.jl
options.jl
promotion.jl
tuple.jl
pair.jl
traits.jl
range.jl
expr.jl
error.jl
bool.jl
number.jl
int.jl
operators.jl
pointer.jl
refvalue.jl
indices.jl
array.jl
abstractarray.jl
bitarray.jl
bitset.jl
abstractdict.jl
abstractset.jl
iterators.jl
namedtuple.jl
docs/core.jl
ordering.jl
sort.jl
compiler/utilities.jl
compiler/validation.jl
compiler/inferenceresult.jl
compiler/params.jl
compiler/inferencestate.jl
compiler/typeutils.jl
compiler/typelimits.jl
compiler/typelattice.jl
compiler/tfuncs.jl
compiler/abstractinterpretation.jl
compiler/typeinfer.jl
compiler/optimize.jl
compiler/ssair/driver.jl
compiler/ssair/ir.jl
compiler/ssair/domtree.jl
compiler/ssair/slot2ssa.jl
compiler/ssair/queries.jl
compiler/ssair/passes.jl
compiler/ssair/inlining.jl
compiler/ssair/verify.jl
compiler/ssair/legacy.jl
compiler/bootstrap.jl
┌ Info: Building `sys.o`:
└ `'C:\Julia-1.3.0-rc3\bin\julia.exe' -C native --output-ji 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\sys.ji' --output-o 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\sys.o' -J 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\basecompiler.ji' --startup-file=no sysimg.jl`
coreio.jl
exports.jl
essentials.jl
ctypes.jl
gcutils.jl
generator.jl
reflection.jl
options.jl
promotion.jl
tuple.jl
expr.jl
pair.jl
traits.jl
range.jl
error.jl
bool.jl
number.jl
int.jl
operators.jl
pointer.jl
refvalue.jl
refpointer.jl
checked.jl
indices.jl
array.jl
abstractarray.jl
subarray.jl
views.jl
baseext.jl
ntuple.jl
abstractdict.jl
iterators.jl
namedtuple.jl
hashing.jl
rounding.jl
float.jl
twiceprecision.jl
complex.jl
rational.jl
multinverses.jl
abstractarraymath.jl
arraymath.jl
simdloop.jl
reduce.jl
reshapedarray.jl
reinterpretarray.jl
bitarray.jl
bitset.jl
multimedia.jl
some.jl
dict.jl
abstractset.jl
set.jl
char.jl
strings/basic.jl
strings/string.jl
strings/substring.jl
build_h.jl
version_git.jl
osutils.jl
c.jl
io.jl
iobuffer.jl
intfuncs.jl
strings/strings.jl
strings/search.jl
strings/unicode.jl
strings/util.jl
strings/io.jl
parse.jl
shell.jl
regex.jl
pcre.jl
show.jl
arrayshow.jl
methodshow.jl
cartesian.jl
multidimensional.jl
permuteddimsarray.jl
broadcast.jl
missing.jl
version.jl
sysinfo.jl
libc.jl
env.jl
linked_list.jl
condition.jl
threads.jl
lock.jl
task.jl
weakkeydict.jl
logging.jl
libuv.jl
uv_constants.jl
asyncevent.jl
iostream.jl
stream.jl
filesystem.jl
cmd.jl
process.jl
grisu/grisu.jl
secretbuffer.jl
floatfuncs.jl
math.jl
reducedim.jl
accumulate.jl
ordering.jl
sort.jl
fastmath.jl
Enums.jl
gmp.jl
mpfr.jl
combinatorics.jl
hashing2.jl
irrationals.jl
mathconstants.jl
printf.jl
meta.jl
channels.jl
deepcopy.jl
download.jl
summarysize.jl
errorshow.jl
stacktraces.jl
initdefs.jl
threadcall.jl
uuid.jl
loading.jl
util.jl
asyncmap.jl
experimental.jl
deprecated.jl
docs\basedocs.jl
client.jl
docs\Docs.jl
Base  ─────────── 25.654957 seconds
Base64  ─────────  3.794240 seconds
CRC32c  ─────────  0.010188 seconds
SHA  ────────────  0.181228 seconds
FileWatching  ───  0.091911 seconds
Unicode  ────────  0.010193 seconds
Mmap  ───────────  0.074751 seconds
Serialization  ──  0.862091 seconds
Libdl  ──────────  0.030937 seconds
Markdown  ───────  1.054587 seconds
LibGit2  ────────  2.649440 seconds
Logging  ────────  0.506780 seconds
Sockets  ────────  1.877789 seconds
Printf  ─────────  0.012369 seconds
Profile  ────────  0.153111 seconds
Dates  ──────────  2.098834 seconds
DelimitedFiles  ─  0.104160 seconds
Random  ─────────  0.525311 seconds
UUIDs  ──────────  0.019996 seconds
Future  ─────────  0.011412 seconds
LinearAlgebra  ── 12.891522 seconds
SparseArrays  ───  6.296322 seconds
SuiteSparse  ────  1.802302 seconds
Distributed  ────  6.485953 seconds
SharedArrays  ───  0.222248 seconds
Pkg  ──────────── 13.223301 seconds
Test  ───────────  0.961266 seconds
REPL  ───────────  1.154462 seconds
Statistics  ─────  0.162380 seconds
Stdlibs total  ── 57.315055 seconds
Sysimage built. Summary:
Total ───────  82.973635 seconds
Base: ───────  25.654957 seconds 30.9194%
Stdlibs: ────  57.315055 seconds 69.0762%
Build shared library "C:\\Users\\yahyaaba\\.julia\\packages\\PackageCompiler\\CJQcs\\sysimg\\backup\\native\\sys.dll":
  `'C:\Users\yahyaaba\.julia\packages\WinRPM\Y9QdZ\deps\usr\x86_64-w64-mingw32\sys-root\mingw\bin\gcc.exe' --sysroot 'C:\Users\yahyaaba\.julia\packages\WinRPM\Y9QdZ\deps\usr\x86_64-w64-mingw32\sys-root' -shared '-DJULIAC_PROGRAM_LIBNAME="C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\sys.dll"' -o 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\sys.dll' -Wl,--whole-archive 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\sys.o' -Wl,--no-whole-archive -std=gnu99 '-IC:\Julia-1.3.0-rc3\include\julia' -DJULIA_ENABLE_THREADING=1 '-LC:\Julia-1.3.0-rc3\bin' -Wl,--stack,8388608 -ljulia -lopenlibm -m64 -Wl,--export-all-symbols`
[ Info: Copying system image: C:\Users\yahyaaba\.julia\packages\PackageCompiler\CJQcs\sysimg\backup\native\sys.dll to C:\Julia-1.3.0-rc3\lib\julia\sys.dll
```
After restart:
```julia
julia> using LinearAlgebra

julia> BLAS.vendor()
:openblas64
```

- The current version of MKL.jl doesn't have access to the same precompilation information as the official binaries and source builds so the REPL will have more latency after MKL.jl has been installed. It should be possible to fix this once Julia 1.1 has been released. See JuliaLinearAlgebra#1.

Tried on Windows and many different installations, only the first boot may take a little longer (less than 20 seconds). The next boots is just the same as normal Julia (less than 5 seconds).
@aminya aminya mentioned this issue Oct 11, 2019
@ViralBShah
Copy link
Contributor

Do we know if the SuiteSparse and ARPACK in these cases are using MKL or do they end up using openblas?

@andreasnoack
Copy link
Member

We now have https://github.com/JuliaLinearAlgebra/ArpackMKL.jl which is required for Arpack to use MKL. We might still need a solution for SuiteSparse. @arnavs have you looked into SuiteSparse?

@ViralBShah
Copy link
Contributor

Is there a better way to do this? Perhaps we should be using a shim library that allows us to replace the BLAS implementation dynamically? There's a bunch of other JuMP solvers too, for example.

In theory I would have thought that if a library linked against openblas, and if instead of openblas, we LD_PRELOADed MKL, things would keep working. Is that possible? Also asking @staticfloat and @Keno

@aminya
Copy link
Contributor

aminya commented Nov 13, 2019

Is there any update on this?
#17

@andreasnoack
Copy link
Member

See #17

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants