Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compiling with pardiso on ubuntu #106

Closed
JackDevine opened this issue Mar 19, 2018 · 24 comments
Closed

Compiling with pardiso on ubuntu #106

JackDevine opened this issue Mar 19, 2018 · 24 comments
Labels
Linear Solver: pardiso Relates to Pardiso

Comments

@JackDevine
Copy link

I am trying to build Ipopt with the pardiso solver by following the instructions on the README. I used the exact same code as the README, except I changed the configure line to be:

./configure --with-pardiso="-qsmp=omp /lib/libpardiso500-GNU461-X86-64.so"

This results in a big long message which ends with

checking whether user-supplied Pardiso library "-qsmp=omp /lib/libpardiso500-GNU461-X86-64.so" works... configure: error: Pardiso library -qsmp=omp /lib/libpardiso500-GNU461-X86-64.so does not seem to work

I have managed to get the library to work with Pardiso.jl, so it is not as though I downloaded a bad library/licence key.

I also opened a discourse thread about this a while ago, but this issue is separate, because now I am trying to custom install the Pardiso solver.

Not sure if this helps, but here is my Julia versioninfo:

julia> versioninfo()
Julia Version 0.6.2
Commit d386e40c17 (2017-12-13 18:08 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, haswell)

And my gfortran version:

$ gfortran --version
GNU Fortran (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609
Copyright (C) 2015 Free Software Foundation, Inc.

GNU Fortran comes with NO WARRANTY, to the extent permitted by law.
You may redistribute copies of GNU Fortran
under the terms of the GNU General Public License.
For more information about these matters, see the file named COPYING

Any help would be greatly appreciated.

@tkelman
Copy link
Contributor

tkelman commented Mar 19, 2018

wherever you're building from, gist the config.log and Ipopt/config.log - one of those will have more details on the actual error

@JackDevine
Copy link
Author

@tkelman
Copy link
Contributor

tkelman commented Mar 19, 2018

Okay nope the top-level config.log wasn't the useful one, Ipopt/config.log should be. Either way this isn't a julia-wrapper-specific issue at this point.

@JackDevine
Copy link
Author

Thanks, I have made a gist for the Ipopt/config.log

https://gist.github.com/JackDevine/e9638cc9836f559e0955f0900c5ce861

Where do you think is the best place to go for help on the Ipopt side?

@tkelman
Copy link
Contributor

tkelman commented Mar 19, 2018

Some combination of here, the Ipopt mailing list, or https://github.com/coin-or/Ipopt. Response times are probably slower places other than here, but you'll have people other than just from JuliaOpt see it.

https://gist.github.com/JackDevine/e9638cc9836f559e0955f0900c5ce861#file-ipopt-config-log-L1490-L1498

configure:36433: checking whether user-supplied Pardiso library "-qsmp=omp /lib/libpardiso500-GNU461-X86-64.so" works
configure:36441: gfortran -o conftest -O3 -pipe    conftest.f -qsmp=omp /lib/libpardiso500-GNU461-X86-64.so  -llapack -lblas >&5
gfortran: error: unrecognized command line option '-qsmp=omp'
configure:36447: $? = 1
configure: failed program was:
|       program main
|       call PARDISO()
|       end
configure:36468: error: Pardiso library -qsmp=omp /lib/libpardiso500-GNU461-X86-64.so does not seem to work

drop the -qsmp=omp, that must be an ifort flag that gfortran doesn't understand

@JackDevine
Copy link
Author

I removed the flag and updated the gist. Thanks for all of your help so far.

@tkelman
Copy link
Contributor

tkelman commented Mar 20, 2018

same lines of the log, now it's /lib/libpardiso500-GNU461-X86-64.so: undefined reference to 'GOMP_loop_dynamic_next' - I think you want -gomp or -fopenmp or -lgomp or something like that, however gfortran spells it

@JackDevine
Copy link
Author

./configure --with-pardiso="-fopenmp /lib/libpardiso500-GNU461-X86-64.so"

Didn't give an error.

I still have to go through the rest of the installation process to check if things are fully working though.

@JackDevine
Copy link
Author

Now when I run make I get a really long error that ends with:

/home/devja964/Ipopt-3.12.8/Ipopt/src/Interfaces/.libs/libipopt.so: undefined reference to `pardiso_'
/home/devja964/Ipopt-3.12.8/Ipopt/src/Interfaces/.libs/libipopt.so: undefined reference to `pardisoinit_'
collect2: error: ld returned 1 exit status
Makefile:496: recipe for target 'ipopt' failed
make[3]: *** [ipopt] Error 1
make[3]: Leaving directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/Apps/AmplSolver'
Makefile:379: recipe for target 'all-recursive' failed
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/Apps'
Makefile:679: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/home/devja964/Ipopt-3.12.8/Ipopt'
Makefile:323: recipe for target 'all-recursive' failed
make: *** [all-recursive] Error 1

@tkelman
Copy link
Contributor

tkelman commented Mar 20, 2018

can you paste or gist the whole thing? I can't really tell what it's trying to link there. Presumably your libpardiso should have pardiso_ and pardisoinit_ symbols in it somewhere? unless maybe the fortran mangling is being inconsistent, but if this was a gfortran-built version of pardiso then it shouldn't be

@JackDevine
Copy link
Author

Making all in ThirdParty/ASL
make[1]: Entering directory '/home/devja964/Ipopt-3.12.8/ThirdParty/ASL'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/home/devja964/Ipopt-3.12.8/ThirdParty/ASL'
Making all in ThirdParty/Mumps
make[1]: Entering directory '/home/devja964/Ipopt-3.12.8/ThirdParty/Mumps'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/home/devja964/Ipopt-3.12.8/ThirdParty/Mumps'
Making all in Ipopt
make[1]: Entering directory '/home/devja964/Ipopt-3.12.8/Ipopt'
Making all in src/Common
make[2]: Entering directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/Common'
make  all-am
make[3]: Entering directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/Common'
make[3]: Leaving directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/Common'
make[2]: Leaving directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/Common'
Making all in src/LinAlg
make[2]: Entering directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/LinAlg'
Making all in TMatrices
make[3]: Entering directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/LinAlg/TMatrices'
make[3]: Nothing to be done for 'all'.
make[3]: Leaving directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/LinAlg/TMatrices'
make[3]: Entering directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/LinAlg'
make[3]: Nothing to be done for 'all-am'.
make[3]: Leaving directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/LinAlg'
make[2]: Leaving directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/LinAlg'
Making all in src/Algorithm
make[2]: Entering directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/Algorithm'
Making all in LinearSolvers
make[3]: Entering directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/Algorithm/LinearSolvers'
make[3]: Nothing to be done for 'all'.
make[3]: Leaving directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/Algorithm/LinearSolvers'
Making all in Inexact
make[3]: Entering directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/Algorithm/Inexact'
make[3]: Nothing to be done for 'all'.
make[3]: Leaving directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/Algorithm/Inexact'
make[3]: Entering directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/Algorithm'
make[3]: Nothing to be done for 'all-am'.
make[3]: Leaving directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/Algorithm'
make[2]: Leaving directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/Algorithm'
Making all in src/contrib/CGPenalty
make[2]: Entering directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/contrib/CGPenalty'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/contrib/CGPenalty'
Making all in src/contrib/LinearSolverLoader
make[2]: Entering directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/contrib/LinearSolverLoader'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/contrib/LinearSolverLoader'
Making all in src/Interfaces
make[2]: Entering directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/Interfaces'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/Interfaces'
Making all in src/Apps
make[2]: Entering directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/Apps'
Making all in CUTErInterface
make[3]: Entering directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/Apps/CUTErInterface'
make[3]: Nothing to be done for 'all'.
make[3]: Leaving directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/Apps/CUTErInterface'
Making all in AmplSolver
make[3]: Entering directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/Apps/AmplSolver'
/bin/bash ../../../../libtool --tag=CXX --mode=link g++  -O3 -pipe -DNDEBUG -Wparentheses -Wreturn-type -Wcast-qual -Wall -Wpointer-arith -Wwrite-strings -Wconversion -Wno-unknown-pragmas -Wno-long-long   -DIPOPT_BUILD   -o ipopt  ampl_ipopt.o libipoptamplinterface.la ../../Interfaces/libipopt.la /home/devja964/Ipopt-3.12.8/ThirdParty/ASL/libcoinasl.la -lm -ldl  -fopenmp /lib/libpardiso500-GNU461-X86-64.so -L/usr/lib/gcc/x86_64-linux-gnu/5 -L/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/5/../../../../lib -L/lib/../lib -L/usr/lib/../lib -L/opt/intel/compilers_and_libraries_2017.4.196/linux/ipp/lib/intel64 -L/opt/intel/compilers_and_libraries_2017.4.196/linux/compiler/lib/intel64_lin -L/opt/intel/compilers_and_libraries_2017.4.196/linux/mkl/lib/intel64_lin -L/opt/intel/compilers_and_libraries_2017.4.196/linux/tbb/lib/intel64/gcc4.7 -L/opt/intel/compilers_and_libraries_2017.4.196/linux/daal/lib/intel64_lin -L/opt/intel/compilers_and_libraries_2017.4.196/linux/daal/../tbb/lib/intel64_lin/gcc4.4 -L/usr/lib/gcc/x86_64-linux-gnu/5/../../.. /home/devja964/Ipopt-3.12.8/ThirdParty/Mumps/libcoinmumps.la -lblas -lgfortran -lm -lquadmath -llapack -lblas -lm  -ldl 
g++ -O3 -pipe -DNDEBUG -Wparentheses -Wreturn-type -Wcast-qual -Wall -Wpointer-arith -Wwrite-strings -Wconversion -Wno-unknown-pragmas -Wno-long-long -DIPOPT_BUILD -o .libs/ipopt ampl_ipopt.o -fopenmp /lib/libpardiso500-GNU461-X86-64.so  ./.libs/libipoptamplinterface.so /home/devja964/Ipopt-3.12.8/Ipopt/src/Interfaces/.libs/libipopt.so ../../Interfaces/.libs/libipopt.so /home/devja964/Ipopt-3.12.8/ThirdParty/ASL/.libs/libcoinasl.so -L/usr/lib/gcc/x86_64-linux-gnu/5 -L/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/5/../../../../lib -L/lib/../lib -L/usr/lib/../lib -L/opt/intel/compilers_and_libraries_2017.4.196/linux/ipp/lib/intel64 -L/opt/intel/compilers_and_libraries_2017.4.196/linux/compiler/lib/intel64_lin -L/opt/intel/compilers_and_libraries_2017.4.196/linux/mkl/lib/intel64_lin -L/opt/intel/compilers_and_libraries_2017.4.196/linux/tbb/lib/intel64/gcc4.7 -L/opt/intel/compilers_and_libraries_2017.4.196/linux/daal/lib/intel64_lin -L/opt/intel/compilers_and_libraries_2017.4.196/linux/daal/../tbb/lib/intel64_lin/gcc4.4 -L/usr/lib/gcc/x86_64-linux-gnu/5/../../.. /home/devja964/Ipopt-3.12.8/ThirdParty/Mumps/.libs/libcoinmumps.so -L/lib/x86_64-linux-gnu -L/usr/lib/x86_64-linux-gnu -lgfortran -lquadmath -llapack -lblas -lm -ldl -Wl,--rpath -Wl,/home/devja964/.julia/v0.6/Ipopt/deps/usr/lib
/home/devja964/Ipopt-3.12.8/Ipopt/src/Interfaces/.libs/libipopt.so: undefined reference to `pardiso_'
/home/devja964/Ipopt-3.12.8/Ipopt/src/Interfaces/.libs/libipopt.so: undefined reference to `pardisoinit_'
collect2: error: ld returned 1 exit status
Makefile:496: recipe for target 'ipopt' failed
make[3]: *** [ipopt] Error 1
make[3]: Leaving directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/Apps/AmplSolver'
Makefile:379: recipe for target 'all-recursive' failed
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory '/home/devja964/Ipopt-3.12.8/Ipopt/src/Apps'
Makefile:679: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/home/devja964/Ipopt-3.12.8/Ipopt'
Makefile:323: recipe for target 'all-recursive' failed
make: *** [all-recursive] Error 1

@tkelman
Copy link
Contributor

tkelman commented Mar 20, 2018

That's odd. I'd check libpardiso500-GNU461-X86-64.so for those functions with nm /lib/libpardiso500-GNU461-X86-64.so | grep pardisoinit_, or maybe try using -L/lib -lpardiso500-GNU461-X86-64 instead of the full path to the file. Not quite sure what's going wrong and this may be at the point where you ask the pardiso authors for support, especially if you have a license through them.

@JackDevine
Copy link
Author

Yes, it seems that somehow the symbols are missing:

devja964@vento15post2:~/Ipopt-3.12.8$ nm /lib/libpardiso500-GNU461-X86-64.so | grep pardisoinit_
nm: /lib/libpardiso500-GNU461-X86-64.so: no symbols
devja964@vento15post2:~/Ipopt-3.12.8$ nm /lib/libpardiso500-GNU461-X86-64.so | grep pardiso_
nm: /lib/libpardiso500-GNU461-X86-64.so: no symbols

If I use

./configure --prefix=$HOME/.julia/v0.6/Ipopt/deps/usr --with-pardiso="-fopenmp -L/lib -lpardiso500-GNU461-X86-64"

Then the configure, make, make test and make install all pass. But,

julia> Pkg.build("Ipopt")
INFO: Building Ipopt
WARNING: imported binding for srcdir overwritten in module __anon__

julia> Pkg.test("Ipopt")
INFO: Testing Ipopt
/home/devja964/julia/bin/julia: symbol lookup error: /home/devja964/.julia/v0.6/Ipopt/deps/usr/lib/libipopt.so: undefined symbol: pardisoinit_
============================================================================[ ERROR: Ipopt ]============================================================================

failed process: Process(`/home/devja964/julia/bin/julia -Cnative -J/home/devja964/julia/lib/julia/sys.so --compile=yes --depwarn=yes --check-bounds=yes --code-coverage=none --color=yes --compilecache=yes /home/devja964/.julia/v0.6/Ipopt/test/runtests.jl`, ProcessExited(127)) [127]

========================================================================================================================================================================
ERROR: Ipopt had test errors

Then if I try solving a model with the mumps linear solver, everything is fine, but if I try pardiso, then

julia: symbol lookup error: /home/devja964/.julia/v0.6/Ipopt/deps/usr/lib/libipopt.so: undefined symbol: pardisoinit_

So I agree that this is the point where I ask the pardiso authors for support. Thank you so much for all of your help so far, I really appreciate it.

@tkelman
Copy link
Contributor

tkelman commented Mar 20, 2018

If it builds as C++ but doesn't load in julia, maybe you need the pardiso library to be pre-emptively dlopen'ed at the julia level, and/or added on the LD_LIBRARY_PATH

@JackDevine
Copy link
Author

I am not completely sure that I understand. I have added /lib to my LD_LIBRARY_PATH, which doesn't seem to fix anything. On the Julia side, the only thing that I know how to do is:

julia> Libdl.dlopen("/lib/libpardiso500-GNU461-X86-64")
Ptr{Void} @0x0000000002773220

@tkelman
Copy link
Contributor

tkelman commented Mar 20, 2018

and if you do that before, and in the same process (so not via Pkg.test), as you try to use Ipopt.jl with the pardiso linear_solver setting, does it help any?

@JackDevine
Copy link
Author

julia> Libdl.dlopen("/lib/libpardiso500-GNU461-X86-64")
Ptr{Void} @0x0000000002ece010

julia> Pkg.build("Ipopt")
INFO: Building Ipopt
WARNING: imported binding for srcdir overwritten in module __anon__

julia> using JuMP, Ipopt

INFO: Recompiling stale cache file /home/devja964/.julia/lib/v0.6/Ipopt.ji for module Ipopt.

julia> m = Model(solver=IpoptSolver(linear_solver="pardiso"))
Feasibility problem with:
 * 0 linear constraints
 * 0 variables
Solver is Ipopt

julia> @variable(m,x[i=1:5])
5-element Array{JuMP.Variable,1}:
 x[1]
 x[2]
 x[3]
 x[4]
 x[5]

julia> @constraint(m,[i=1:5],x[i]-0.1 == 0)
5-element Array{JuMP.ConstraintRef,1}:
 x[1] = 0.1
 x[2] = 0.1
 x[3] = 0.1
 x[4] = 0.1
 x[5] = 0.1

julia> solve(m)
julia: symbol lookup error: /home/devja964/.julia/v0.6/Ipopt/deps/usr/lib/libipopt.so: undefined symbol: pardisoinit_

And then it exits Julia

@JackDevine
Copy link
Author

Hi all,

I am pleased to say that I kind of sort of solved this problem by using the pardiso bundled with MKL. I used

./configure --prefix=$HOME/.julia/v0.6/Ipopt/deps/usr --with-blas="-L${MKLROOT}/lib/intel64 -lmkl_rt -lpthread -lm -ldl"

to configure Ipopt. So basically, the issue that I raised is not resolved, but I can still use the pardiso solver through MKL. I am seeing some outrageous performance gains comparing mumps and pardiso in the models that I am solving.

I have sent an email to the pardiso team and I am waiting to hear what they have to say.

I also want to point out that @tkelman provided far more help than I could have reasonably asked of anyone, so I am very grateful.

Since this is now an upstream issue with a simple workaround, I am happy for this issue to be closed.

@pedroborgesmelo
Copy link

pedroborgesmelo commented Jul 22, 2019

Hello Guys!

I'm also having this kind of problem here..

To be sure that the problem is inside Julia I set the pardiso linear solver in the cpp example provided with Ipopt's code.. it works and prints the correct information saying that it is using Pardiso.

On the Julia side, however, I get:

Exception of type: OPTION_INVALID in file "IpAlgBuilder.cpp" at line 366:
Exception message: Selected linear solver Pardiso not available.
Tried to obtain Pardiso from shared library "libpardiso.so", but the following error occured:
/impa/home/f/pborges/Desktop/installs/pardiso/libpardiso.so: undefined symbol: sgetrf_

If a try to run a simple problem:

using JuMP
using Ipopt
m = Model(solver=IpoptSolver(linear_solver="pardiso"))
@variable(m, x >= 0)
@objective(m, :Min, x)
solve(m)

Moreover, Pardiso.jl works fine with and without MKL.

Any clues?

@JackDevine
Copy link
Author

It is interesting that you managed to prove that this is a Julia issue.

Have you tried using the Pardiso bundled with MKL like described in my last comment? It was quite a while ago, but I remember getting that to work fine.

@pedroborgesmelo
Copy link

pedroborgesmelo commented Jul 22, 2019 via email

@JackDevine
Copy link
Author

That is unfortunate. Since raising this issue I used the work around described above and then I wrote my own custom Julia solver for the problem that I was solving, which is what I use now. So unfortunately, I can't comment on whether my work around still works with the current version of Julia etc. Hopefully you can get this sorted out soon.

@odow odow added the LINALG Codes Relates to linear algebra codes (HSL etc.) label May 26, 2020
@odow odow added Linear Solver: pardiso Relates to Pardiso and removed LINALG Codes Relates to linear algebra codes (HSL etc.) labels Nov 18, 2020
@odow
Copy link
Member

odow commented Nov 23, 2020

Closing as stale. #236 provides instructions on how to use Pardiso on Ubuntu.

@odow odow closed this as completed Nov 23, 2020
@JOHN194872
Copy link

I am trying to compile laplace_mpi.c but get:
laplace_mpi.c:(.text+0x1250): undefined reference to `pardiso_mpi_init_c_' etc
Please help.
Thanks,
John

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Linear Solver: pardiso Relates to Pardiso
Development

No branches or pull requests

5 participants