Errors: No BLAS/LAPACK library loaded! #427
Comments
|
I also wanted to chime in and say that I started experiencing a different, but related issue updating today which brought LinearSolve to v2.18.0. I started experience serious performance regressions due to #408. In my case I'm running a parallel workload via Distributed and realized that after updating, suddenly every worker was launching a bunch of threads causing my system to be completely oversubscribed. It turns out, by default local distributed workers have I'll open a separate issue about setting However I'm also often comparing the difference between openblas and mkl for by use cases and hardware and #408 makes it non-obvious how I can control this behavior. Especially since I now see that the first half of my computation uses openblas, then the second half uses mkl since apparently I'm only calling certain functions in Personally I think with how many downstream packages are affected by #408 and given that it seems there's not an official, documented way to switch back to openblas (sees JuliaLinearAlgebra/MKL.jl#90), it may be wise to revert it? |
|
|
Let me start by saying this will be reverted, but for the right reasons, not the wrong reasons. The right reason to revert it is the head case of this issue, which is that MKL.jl is subtly breaking, even though that is not supposed to be intended by MKL.jl and it's not documented. This addressed in JuliaLinearAlgebra/MKL.jl#138 and JuliaLinearAlgebra/MKL.jl#139 and until that is fixed we cannot use MKL.jl. It's unclear to me whether the MKL thread setting is a "breaking change" to LinearSolve.jl, but whether or not it's not factoring into this decision and Distributed.jl should fix its system to be BLAS-independent. That said, we will need to do something about SuiteSparse. While Base Julia builds SuiteSparse with LBT and thus links with OpenBLAS, this is not recommended by the author and notably leads to a 100x slowdown (DrTimothyAldenDavis/SuiteSparse#1). Given the way the SuiteSparse binaries are built, the only way to make it use MKL is to do the global LBT trigger, so that's something that's effectively blocking us here and putting us in a bind JuliaSparse/SparseArrays.jl#453 . As discussed in that issue, we will go with throwing a warning if people do sparse matrix factorizations with OpenBLAS, and hopefully down the line build specific SuiteSparseMKL binaries. |
|
For future measure, the data is that there were about 50 happy responses, and 4 unhappy responses, 2 here and 2 not here. Many of the happy responses were in the 2x-10x range, a few in the 100x range when manually choosing UMFPACK as the factorization. The 2 unhappy responses here were due to things subtly breaking with MKL which were previously not known. The 2 unhappy responses which were not reported here were both that it was tracked down that MKL is slower on AMD Epyc. What was shown to be slower was the matrix multiplications on AMD Epyc with MKL, while LU factorization was only in the 20%-25% slowdown range. Therefore it's very clear that for LU factorizations then, MKL is a major net win. These benchmarks of other folks showed that MKL is at least as fast (if not faster) on all other tested platforms (including other AMD chips), and I want to highlight that because it seems that those who have negative responses are more inclined to share their griefs more publicly, so I want to make it clear that the data shows that This shows that the bound on the place where KLU is faster, i.e. at around the 10,000 x 10,000 mark and used in the defaults here https://github.com/SciML/LinearSolve.jl/blob/main/src/default.jl#L85, is actually just an artifact of OpenBLAS. If KLU is used, UMFPACK is generally much faster, and so around the 500x500 sparse matrix case it can already make sense to use UMPACK over KLU. Thus we should refine that default to take into account whether someone is using OpenBLAS, where the result is effectively "if using MKL, then use a fast algorithm, otherwise use the slow algorithm because it's at least not as slow as calling OpenBLAS" 😅. In the end, I'm glad I gave this a try. If no one ever tries to move away from OpenBLAS, then we'll be 100x slower in sparse matrix operations than we should be indefinitely. However, it's clear we need to do this another way for it to be safer, and it's also clear what the benefits will be. |
|
@ChrisRackauckas thank you for taking the time to explain the situation. I didn't know the context for why these changes were taking place when I suggested reverting them as I didn't come across JuliaSparse/SparseArrays.jl#453 when looking through the relevant commits and issues here. It sounds like a solid reason to default to MKL, but hopefully Distributed can first be fixed to have consistent threading behavior in order to minimize surprises like what I encountered. It's not difficult to work around at all, by either setting the env variable or via Thanks for continuing to push julia to be faster! |
Recently, I tried to prepare a new version of PeriodicSystems by starting with several commits. Running the tests, the following error pops up repeatedly
Error: no BLAS/LAPACK library loaded!(see here)
The cause of these error messages is apparently the new version of
LinearSolve, whereMKLis loaded now by default insteadOpenBLAS(see the discussion here). I implemented several wrappers to routines from the SLICOT library, using the automatically generated SLICOT_jll. Apparently, some calls toBLAS/LAPACKare not covered there byMKL. I assume that other people usingOrdinaryDiffEqmay face the same issue.I would appreciate if you could help me to overcome this situation, by indicating a way to preserve the behaviour before the last update. Many thanks in advance
The text was updated successfully, but these errors were encountered: