Error on using SCIP.jl with custom binary

[JunchengLiamLi]

I tried to use SCIP.jl to call a locally installed SCIP which uses CPLEX as its default LP solver and encountered the following error:

julia> using SCIP_jll
ERROR: InitError: could not load library "/home/juncheng_liam_li/optimisation/scip-9.2.4/build/lib/libsoplexshared.so"
/home/juncheng_liam_li/optimisation/scip-9.2.4/build/lib/libsoplexshared.so: cannot open shared object file: No such file or directory
Stacktrace:
[1] #dlopen#3
@ ./libdl.jl:120 [inlined]

I have added the following line in .julia/artifact/Overrides.toml:
b2644ab1add1d76a82a78ec26c2f9ee906c8e162 = "/home/juncheng_liam_li/optimisation/scip-9.2.4/build"

, following the instructions in [https://jump.dev/JuMP.jl/stable/developers/custom_solver_binaries/#jll_structure]

My guess is that SCIP_jll is trying to find the Solplex in my local SCIP, which is non-existent.

[matbesancon]

Thanks for reporting, I hadn't expected this. The issue is that SCIP.jl expects a scipoptsuite build, which always contains the SoPLEX shared library even if SCIP is not compiled with SoPLEX. Clearly that was a wrong assumption.

You can use the Custom installations section of the readme for now, which will point to the correct binary in deps/deps.jl

[odow]

Yeah the Override must have exactly the same structure as the JLL

In this case, it needs /lib/libscip, /lib/libsoplex, and /bin/scipexe:

https://github.com/JuliaBinaryWrappers/SCIP_jll.jl?tab=readme-ov-file#products

[odow]

This is expected behaviour, and the correct way (in SCIP's case) to use a custom binary is following the custom installation instructions: https://github.com/scipopt/SCIP.jl?tab=readme-ov-file#custom-installations

Closing because I don't think there's anything actionable to do here.

[JunchengLiamLi]

This is expected behaviour, and the correct way (in SCIP's case) to use a custom binary is following the custom installation instructions: https://github.com/scipopt/SCIP.jl?tab=readme-ov-file#custom-installations

Closing because I don't think there's anything actionable to do here.

I removed the old SCIP.jl and built a new SCIP.jl followig the custom installation instructions. However, SCIP still uses Soplex as the LP solver, instead of CPLEX

[odow]

What's the output of running this:

ENV["SCIPOPTDIR"] = "... your actual path ..."
Pkg.add("SCIP")
Pkg.build("SCIP")
import SCIP
SCIP.libscip

[JunchengLiamLi]

What's the output of running this:

ENV["SCIPOPTDIR"] = "... your actual path ..."
Pkg.add("SCIP")
Pkg.build("SCIP")
import SCIP
SCIP.libscip

julia> ENV["SCIPOPTDIR"] = "/home/juncheng_liam_li/optimisation/scip-9.2.4/build"
"/home/juncheng_liam_li/optimisation/scip-9.2.4/build"

julia> Pkg.add("SCIP")
Resolving package versions...
Installed SCIP_PaPILO_jll ─ v900.200.200+0
Installed SCIP_jll ──────── v900.200.200+0
Installed SCIP ──────────── v0.12.7
Updating ~/.julia/environments/v1.12/Project.toml
[82193955] + SCIP v0.12.7
Updating ~/.julia/environments/v1.12/Manifest.toml
[82193955] + SCIP v0.12.7
[ae81ac8f] + ASL_jll v0.1.3+0
⌅ [9cc047cb] + Ipopt_jll v300.1400.1400+0
⌅ [d7ed1dd3] + MUMPS_seq_jll v500.600.201+0
[68e3532b] + Ncurses_jll v6.5.1+0
[05236dd9] + Readline_jll v8.2.13+0
[fc9abe76] + SCIP_PaPILO_jll v900.200.200+0
[e5ac4fe4] + SCIP_jll v900.200.200+0
⌅ [319450e9] + SPRAL_jll v2024.1.18+0
[508c9074] + bliss_jll v0.77.0+1
[28df3c45] + boost_jll v1.87.0+0
[1317d2d5] + oneTBB_jll v2022.0.0+1
[781609d7] + GMP_jll v6.3.0+2
Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. To see why use status --outdated -m
Building SCIP → ~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/de60c2cea424a4df4cadcc68700340943aabbbae/build.log

julia> Pkg.build("SCIP")
Building SCIP → ~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/de60c2cea424a4df4cadcc68700340943aabbbae/build.log

julia> import SCIP

julia> SCIP.libscip
[ Warning: libscip is defined in SCIP_PaPILO_jll and is not public in SCIP
"/home/juncheng_liam_li/.julia/artifacts/dcf2760720b54f48ffe83e7ec4af21962e090305/lib/libscip.so"

[odow]

It looks like we're not able to find your custom SCIP binary, so we're falling back to the default.

What are the contents of /home/juncheng_liam_li/optimisation/scip-9.2.4/build/lib?

[JunchengLiamLi]

It looks like we're not able to find your custom SCIP binary, so we're falling back to the default.

What are the contents of /home/juncheng_liam_li/optimisation/scip-9.2.4/build/lib?

Please see the following:

juncheng_liam_li@MSI:~$ cd /home/juncheng_liam_li/optimisation/scip-9.2.4/build

juncheng_liam_li@MSI:~/optimisation/scip-9.2.4/build$ ls
CMakeCache.txt Makefile cmake_install.cmake scip-config.cmake
CMakeFiles Testing doc scip-targets.cmake
CPackConfig.cmake applications examples scip_update_githash.cmake
CPackSourceConfig.cmake bin lib scipbuildflag.c
CTestTestfile.cmake check scip src
DartConfiguration.tcl cmake.log scip-config-version.cmake tests

juncheng_liam_li@MSI:~/optimisation/scip-9.2.4/build$ ls bin
applications examples scip

juncheng_liam_li@MSI:~/optimisation/scip-9.2.4/build$ ls lib
libscip.so libscip.so.9.2 libscip.so.9.2.4.0

[odow]

Hmm. That looks like it should work.

What about

import Libdl
Libdl.dlopen("/home/juncheng_liam_li/optimisation/scip-9.2.4/build/lib/libscip.so")

[JunchengLiamLi]

Hmm. That looks like it should work.

What about

import Libdl
Libdl.dlopen("/home/juncheng_liam_li/optimisation/scip-9.2.4/build/lib/libscip.so")

julia> Libdl.dlopen("/home/juncheng_liam_li/optimisation/scip-9.2.4/build/lib/libscip.so")
Ptr{Nothing}(0x000000001e795d50)

[odow]

That makes no sense 😢 It should be finding the right SCIP.

Try uninstalling and re-installing in a new session:

import Pkg
Pkg.rm("SCIP")

close Julia. Open a new session:

import Pkg
ENV["SCIPOPTDIR"] = "... your actual path ..."
Pkg.add("SCIP")
Pkg.build("SCIP")
import SCIP
SCIP.libscip

[JunchengLiamLi]

Now it works. I got the following result:
julia> SCIP.libscip
"/home/juncheng_liam_li/optimisation/scip-9.2.4/build/lib/libscip.so"

This time I not only removed SCIP from julia REPL, but also delete the folders SCIP, SCIP_jll and SCIP_PaPILO_jll in .julia/packages and restart my machine, before I install SCIP again.