FAILURE (Can't use SPARSE_NORMAL_CHOLESKY with SUITESPARSE because SuiteSparse was not enabled when Ceres was built.)

[edgarriba]

Hi guys,

I'm using TheiaSfM which has ceres as a dependency and got the following exception error when running the global pipeline:

Termination: FAILURE (Can't use SPARSE_NORMAL_CHOLESKY with SUITESPARSE because SuiteSparse was not enabled when Ceres was built.)

I'm pretty sure that I compiled ceres with a sparse solver (tried with Suitesparse, CXSparse and Eigensparse). Did you notice any similar issue in this direction?

PD: you can check the short chat I had with Theia author **** and the complete log I got.

[sweeneychris]

@edgarriba It would be helpful if you could include the CMake output when you are building Ceres

[edgarriba]

sure. This is my CMake output:

-- Detected Ceres version: 1.12.0 from /home/eriba/software/ceres-solver/include/ceres/version.h
-- Found Eigen version 3.2.0: /usr/include/eigen3

===============================================================
Enabling the use of Eigen as a sparse linear algebra library
for solving the nonlinear least squares problems. Enabling
this option results in an LGPL licensed version of
Ceres Solver as the Simplicial Cholesky factorization in Eigen
is licensed under the LGPL.
WARNING:

Your version of Eigen is older than version 3.2.2.
The performance of SPARSE_NORMAL_CHOLESKY and SPARSE_SCHUR
linear solvers will suffer.

-- A library with BLAS API found.
-- Found LAPACK library: /usr/lib/liblapack.so;/usr/lib/libf77blas.so;/usr/lib/libatlas.so
-- Found BLAS library: /usr/lib/libf77blas.so;/usr/lib/libatlas.so
-- A library with BLAS API found.
-- Found AMD library: /usr/local/lib/libamd.so
-- Found AMD header in: /usr/local/include
-- Found CAMD library: /usr/local/lib/libcamd.so
-- Found CAMD header in: /usr/local/include
-- Found COLAMD library: /usr/local/lib/libcolamd.so
-- Found COLAMD header in: /usr/local/include
-- Found CCOLAMD library: /usr/local/lib/libccolamd.so
-- Found CCOLAMD header in: /usr/local/include
-- Found CHOLMOD library: /usr/local/lib/libcholmod.so
-- Found CHOLMOD header in: /usr/local/include
-- Found SuiteSparseQR library: /usr/local/lib/libspqr.so
-- Found SuiteSparseQR header in: /usr/local/include
-- Found Intel Thread Building Blocks (TBB) library: /usr/lib/libtbb.so, assuming SuiteSparseQR was compiled with TBB.
-- Found Intel Thread Building Blocks (TBB) Malloc library: /usr/lib/libtbbmalloc.so
-- Found SuiteSparse_config library: /usr/local/lib/libsuitesparseconfig.so
-- Found SuiteSparse_config header in: /usr/local/include
-- Adding librt: /usr/lib/x86_64-linux-gnu/librt.so to SuiteSparse_config libraries (required on Linux & Unix [not OSX] if SuiteSparse is compiled with timing).
-- Found METIS library: /usr/local/lib/libmetis.so.
-- Found SuiteSparse 4.5.0, building with SuiteSparse.
-- Found CXSparse version: 3.1.7, building with CXSparse.
-- No preference for use of exported gflags CMake configuration set, and no hints for include/library directories provided. Defaulting to preferring an installed/exported gflags CMake configuration if available.
-- Found installed version of gflags: /home/eriba/software/OpenDroneMap-edgarriba/SuperBuild/build/gflags
-- Detected broken gflags install in: /home/eriba/software/OpenDroneMap-edgarriba/SuperBuild/build/gflags, version: 2.1.2 <= 2.1.2 which defines gflags_LIBRARIES = gflags which is not an imported CMake target, see: gflags/gflags#110. Attempting to fix by detecting correct gflags target.
-- Found valid gflags target: gflags-static, updating gflags_LIBRARIES.
-- Detected gflags version: 2.1.2
-- Found Google Flags header in: /home/eriba/software/OpenDroneMap-edgarriba/SuperBuild/build/gflags/include, in namespace: google
-- Found Google Log header in: /usr/include
-- Building with OpenMP.
-- Looking for C++ include unordered_map
-- Looking for C++ include unordered_map - not found
-- Looking for C++ include tr1/unordered_map
-- Looking for C++ include tr1/unordered_map - found
-- Found tr1/unordered_map/set in std::tr1 namespace.
-- Looking for C++ include memory
-- Looking for C++ include memory - found
-- Performing Test HAVE_SHARED_PTR_IN_STD_NAMESPACE
-- Performing Test HAVE_SHARED_PTR_IN_STD_NAMESPACE - Failed
-- Performing Test HAVE_SHARED_PTR_IN_TR1_NAMESPACE
-- Performing Test HAVE_SHARED_PTR_IN_TR1_NAMESPACE - Failed
-- Looking for C++ include tr1/memory
-- Looking for C++ include tr1/memory - found
-- Performing Test HAVE_SHARED_PTR_IN_TR1_NAMESPACE_FROM_TR1_MEMORY_HEADER
-- Performing Test HAVE_SHARED_PTR_IN_TR1_NAMESPACE_FROM_TR1_MEMORY_HEADER - Success
-- Found shared_ptr in std::tr1 namespace using <tr1/memory> header.
-- Building Ceres as a static library.
-- Enabling CERES_USE_EIGEN_SPARSE in Ceres config.h
-- Enabling CERES_USE_OPENMP in Ceres config.h
-- Enabling CERES_HAVE_PTHREAD in Ceres config.h
-- Enabling CERES_HAVE_RWLOCK in Ceres config.h
-- Enabling CERES_TR1_UNORDERED_MAP in Ceres config.h
-- Enabling CERES_TR1_MEMORY_HEADER in Ceres config.h
-- Enabling CERES_TR1_SHARED_PTR in Ceres config.h
-- Do not build any example.
-- Configuring done
-- Generating done
-- Build files have been written to: /home/eriba/software/ceres-solver/build

[sweeneychris]

"-- Enabling CERES_USE_EIGEN_SPARSE in Ceres config.h"

disable CERES_USE_EIGEN_SPARSE

[sandwichmaker]

why does that make a difference?

On Mon, Feb 29, 2016 at 12:13 PM Chris Sweeney notifications@github.com
wrote:

"-- Enabling CERES_USE_EIGEN_SPARSE in Ceres config.h"

disable CERES_USE_EIGEN_SPARSE

—
Reply to this email directly or view it on GitHub
#197 (comment)
.

[sweeneychris]

Not sure why CERES_NO_SUITESPARSE is being set unless it is being overwritten somewhere. I never touch this (or any other Ceres external libs params/macros) in Theia.

[sandwichmaker]

can you replicate this bug chris?

On Mon, Feb 29, 2016 at 12:26 PM Chris Sweeney notifications@github.com
wrote:

Not sure why CERES_NO_SUITESPARSE is being set unless it is being
overwritten somewhere. I never touch this (or any other Ceres libs params)
in Theia.

—
Reply to this email directly or view it on GitHub
#197 (comment)
.

[sweeneychris]

No I have not been able to

On Mon, Feb 29, 2016 at 12:27 PM Sameer Agarwal notifications@github.com
wrote:

can you replicate this bug chris?

On Mon, Feb 29, 2016 at 12:26 PM Chris Sweeney notifications@github.com
wrote:

Not sure why CERES_NO_SUITESPARSE is being set unless it is being
overwritten somewhere. I never touch this (or any other Ceres libs
params)
in Theia.

—
Reply to this email directly or view it on GitHub
<
#197 (comment)

.

—
Reply to this email directly or view it on GitHub
#197 (comment)
.

[sandwichmaker]

@alexsmac any idea how to debug this?

[edgarriba]

Does it has something to do the fact that ceres is built as static?

[edgarriba]

another thing that I've noticed is that during installation nothing is copied in /usr/local/share/Ceres

[alexsmac]

What does the first line from Solver::Summary::FullReport() say for your current Ceres installation? The one you posted earlier for Theia had this:

Solver Summary (v 1.10.0-lapack-no_openmp)

when it should be something more like this:

Solver Summary (v 1.12.0-eigen-(3.2.5)-lapack-suitesparse-(4.4.4)-cxsparse-(3.1.4)-no_openmp)

given the configuration output you posted above.

I suspect that Theia is linking/linked (if static) against an older version of Ceres installed on your machine. Checking the contents of Ceres_DIR in the advanced view in the CMake GUI when building Theia should tell you which one it is using. Note that this should be the path to the directory containing CeresConfig.cmake (e.g. /usr/local/lib/cmake/Ceres). To answer your question about /usr/local/share/Ceres, as of 1.12+, that is no longer used, we use the (potentially) architecture dependent install location: /usr/local/lib<OPTIONAL_PREFIX>/cmake/Ceres.

One way that Theia could be picking up an old Ceres build would be if you exported the Ceres build directory (instead of installing it) by enabling EXPORT_BUILD_DIR. You can easily check for this by looking at the contents of ~/.cmake/packages/Ceres. If that contains any files (which should have SHA codes for names, and contain the full path to a Ceres build directory) which point to existing Ceres build directories, then they will be used in preference to an installed version of Ceres.

[edgarriba]

Definitely we have a Poltergeist here.

I've checked the Ceres_DIR from Theia and is set with its default value (/usr/local/lib/cmake/Ceres). I've also checked in ~/.cmake/packages/Ceres and you were right, there was a file, but it was pointing to the new ceres version. To be sure I removed this file and compiled again setting the EXPORT_BUILD_DIR flag to OFF.

Same result.

[alexsmac]

But what does the first line from Solver::Summary::FullReport() say for your current Ceres installation? Are you still seeing this:

Solver Summary (v 1.10.0-lapack-no_openmp)

? Or something more like this:

Solver Summary (v 1.12.0-eigen-(3.2.5)-lapack-suitesparse-(4.4.4)-cxsparse-(3.1.4)-no_openmp)

[edgarriba]

yes, same summary as before. Each time I uninstall ceres, theia complains about that, so theia is linking the current ceres version.

[sweeneychris]

Is it possible for Theia to detect at Cmake time if Ceres was built with SuiteSparse? If so, I could at least have Cmake halt since SuiteSparse is needed for sparse BA in Theia

[alexsmac]

@sweeneychris - try this: https://ceres-solver-review.googlesource.com/#/c/7370

It still needs a docs update, but should give you an idea of how it might work - feedback welcome :)

[alexsmac]

@sweeneychris the CL has now been merged into master, so you can do things like this:

find_package(Ceres REQUIRED SuiteSparse)

or:

find_package(Ceres REQUIRED SparseLinearAlgebraLibrary)

to get check for at least one sparse matrix backend) (SuiteSparse OR EigenSparse OR CXSparse).

This will be backwards compatible in a way, in that find_package(Ceres REQUIRED SparseLinearAlgebraLibrary) should report Ceres as found even on older Ceres installs that predate this change. However - it will obviously report Ceres as found without checking whether the specified modules are present.

[sweeneychris]

Awesome, this is great @alexsmac thanks for the patch!

@edgarriba If you sync to the latest master branch of Theia, it now explicitly requries SuiteSparse (independently of this Ceres patch) since CHOLMOD is being used for solving some large sparse systems.

[soulslicer]

I am still experiencing this issue in my project on gcc4.8. How exactly do you fix this?