Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Problems installing Trilinos on candi #135

Closed
UfrancoU opened this issue Feb 26, 2020 · 15 comments
Closed

Problems installing Trilinos on candi #135

UfrancoU opened this issue Feb 26, 2020 · 15 comments

Comments

@UfrancoU
Copy link

Hi all,

I am on the process of installing ASPECT on my local cluster.

I am using candi software. Upon building-compiling, the process goes well and far for more than 20 minutes (using 8 cores), it installs p4est. Then during Trilinos installation starts having a bunch of warnings, but goes on. The problem comes with the BLAS library:

-- Searching for libs in BLAS_LIBRARY_DIRS=''
-- Searching for a lib in the set "blas blas_win32":
-- Searching for lib 'blas' ...
-- Searching for lib 'blas_win32' ...
-- ERROR: Did not find a lib in the lib set "blas blas_win32" for the TPL 'BLAS'!
-- ERROR: Could not find the libraries for the TPL 'BLAS'!

But, I do have BLAS and LAPACK on my account.

So, this problem is simple and classic, it is a linking problem.

I have tried two ways to fix this linking issue:

  1. I created a file (/etc/ld.so.conf.d/mylibs.conf ) with dir to a shared-library path for my BLAS: /home/Felipe/BIGSOFTWARE/lapack-release-lapack-3.8.0/BLAS/SRC

  2. I edited my .bashrc file adding export BLAS_LIBRARY_PATH=$BLAS_LIBRARY_PATH:/home/Felipe/BIGSOFTWARE/lapack-release-lapack-3.8.0/BLAS/SRC

I seized the opportunity and also added the path for Lapack..which is best friend of Blas.. :)

After a day learning how and doing both of these things., I started a new session and started over with candi (no traces of the previous trial. I downloaded-cloned a new candi), and executed candi..

The error was the same.. the candi compiling of Trilinos can not find my BLAS library.

Where and how should I specify the path for BLAS and LAPACK for candi to be able to properly do its job?

cheers,
Felipe
@tjhei
Copy link
Member

tjhei commented Feb 26, 2020

Did you set BLAS_LIB and LAPACK_LIB inside candi.cfg to a folder containing the compiled versions? Are there libblas.so etc. found inside /home/Felipe/BIGSOFTWARE/lapack-release-lapack-3.8.0/BLAS/SRC ?

@UfrancoU
Copy link
Author

Hi Timo,

  Thanks a lot for your reply,

  I compiled BLAS 3.8.0 just in November (so from all the .f files y got their corresponding .o subroutine files). But I find no  proper‘libblas.so’ anywhere in my account. I also have LAPACK, which contains a Blas subdirectory.. but I don’t quite understand why these two libraries are united (part of one included inside the other) yet separated. I just expected linking between them.

  Compilation of BLAS yielded .o files. I have nothing else to compile in my BLAS, so how could I get libblas.so files?

   Inside Lapack-release-lapack-3.8.0/BLAS/SRC  I see almost the same collection of .f and .o files.  Is that what I need? Is that what is called libblas.so?

  Beyond this, in the usr/lib64 folder of my cluster, there are 3 files named  libblas.so.3.x.y  (x, y 4 and 2) which I am not sure what they are..

 There are lots of naming-and-linking conventions that I don’t get here. But I think I have all the necessary ingredients.

What should I do?

cheers many many,
Felipe

@tjhei
Copy link
Member

tjhei commented Feb 27, 2020

Your SRC/ folder does not seem to contain the compiled libraries. No wonder Trilinos is unhappy.

Beyond this, in the usr/lib64 folder of my cluster, there are 3 files named libblas.so.3.x.y

Those are the versioned shared libraries, which means you managed to compile them at least. I don't think Trilinos will pick them up (there is typically a symlink from libblas.so to the libblas.so.x.y).

Bottom line: you need to figure out how to install blas/lapack first. Two alternatives:

  1. If you have root access (or a sysadmin), let them install blas/lapack as instructed by candi when you start it (depends on your distribution).
  2. We have the package "openblas" inside candi, that you can install.

@UfrancoU
Copy link
Author

Hi Timo,

Thanks a loooooot for your support.

(My message text is long, but it mostly pasted text from the logfile for you to see)

I started from scratch. Inside candi directory I edited candi.cfg , solely uncomenting two lines, activating the zlib and openblas packages.

I run candi using 8 cores, and it went well until failing. Did this 3 times to understand and check.

I am worried about something at the beginning of the process and logfile: It warns that compiler variables mpicc, mpicxx, mpif90, mpif77 are not set. I checked, I do have mpich (I installed it), which seems to do the job, ..for example:
[Felipe@mu01 candi]$ mpif77
Error: Command line argument is needed!
[Felipe@mu01 candi]$ mpif77 -v
mpifort for MPICH version 3.1.4
Using built-in specs.
COLLECT_GCC=gfortran
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/4.8.5/lto-wrapper
Target: x86_64-redhat-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-linker-hash-style=gnu --enable-languages=c,c++,objc,obj-c++,java,fortran,ada,go,lto --enable-plugin --enable-initfini-array --disable-libgcj --with-isl=/builddir/build/BUILD/gcc-4.8.5-20150702/obj-x86_64-redhat-linux/isl-install --with-cloog=/builddir/build/BUILD/gcc-4.8.5-20150702/obj-x86_64-redhat-linux/cloog-install --enable-gnu-indirect-function --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux
Thread model: posix
gcc version 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC)

  1. Where should I set those compiler variables? In etc/profile ? In /.bashrc ? In candi.cfg?..candi does not seem to have a section for that.

Unsurprisingly, the compilation-installation process goes on without having those variables set. So I guess mpich does the work for candi.

More so, it seems candi effectively did install Openblas. In fact I got a directory OpenBLAS-0.2.20, with subdirs bin, include, lib. Upon reviewing my logfile, I read PASSED THE COMPUTATIONAL TESTS, and I also read:
Generating OpenBLASConfig.cmake in /home/Felipe/BIGSOFTWARE/candi/OpenBLAS-0.2.20/lib/cmake/openblas
Generating OpenBLASConfigVersion.cmake in /home/Felipe/BIGSOFTWARE/candi/OpenBLAS-0.2.20/lib/cmake/openblas
Install OK!
make[1]: Leaving directory `/home/Felipe/BIGSOFTWARE/candi/tmp/build/OpenBLAS-0.2.20'

I guess after this library has been installed, its directory is set by default by candi.. so I don’t have to worry..

But I got an error later, right after the previous line. This is what it outputs:

Building opencascade 0.18.2
./candi_configure: line 105: cmake: command not found
Failure with exit status: 127
Exit message: There was a problem configuring opencascade 0.18.2.

but contradictory, I do have cmake (I installed it):

[Felipe@mu01 candi]$ cmake
Usage
cmake [options]
cmake [options]
cmake [options] -S -B
Specify a source directory to (re-)generate a build system for it in the
current working directory. Specify an existing build directory to
re-generate its build system.
Run 'cmake --help' for more information.

[Felipe@mu01 candi]$ cmake -version
cmake version 3.16.0-rc3

CMake suite maintained and supported by Kitware (kitware.com/cmake).
%%%%%%%%%%%%%%%%%%%

cheers,
Felipe

@tjhei
Copy link
Member

tjhei commented Feb 28, 2020

  1. Where should I set those compiler variables?

This should be on your screen "CXX variable not set, but default mpicxx found". This is not a problem, but you can override it by setting an env variable CXX, etc..

Building opencascade 0.18.2
./candi_configure: line 105: cmake: command not found
Failure with exit status: 127

Not sure, but you can check the files candi_configure and candi_configure.log in <destination_folder>/tmp/build/oce*/

@UfrancoU
Copy link
Author

UfrancoU commented Mar 5, 2020 via email

@MarineLasbleis
Copy link

So, I am trying to install deal II on the new (shared, so I don't have admin rights) computer here, and apparently we are running to the same exact error. Did you manage to install it finally?

-- Searching for libs in BLAS_LIBRARY_DIRS=''
-- Searching for a lib in the set "blas blas_win32":
-- Searching for lib 'blas' ...
-- Searching for lib 'blas_win32' ...
-- ERROR: Did not find a lib in the lib set "blas blas_win32" for the TPL 'BLAS'!
-- ERROR: Could not find the libraries for the TPL 'BLAS'!

According to my sysadmin, blas and lapack are installed, and in /usr/lib64. But I don't understand how to tell trilinos that they are there.
I will try the openblas approach, but I am not sure it will help, considering what you got after.

First of all maybe: the computer is running on Oracle Linux, which is not listed, so I used the "Linux cluster" configuration. Apparently, Oracle is a Red Hat, as CentOS was, so I also tried using the CentOS configuration without more success.

Thanks
Marine

@masterleinad
Copy link
Member

Do you set BLAS_DIR and LAPACK_DIR accordingly? Can you build Trilinos without candi?

@MarineLasbleis
Copy link

I haven't try to build trilinos by itself. I'll try tomorrow.
Could you be more specific on where to set the BLAS_DIR and LAPACK_DIR? I set them as system variables (so in the terminal: export BLAS_DIR=...) But I am worried I need to set them up in candi somewhere, and I don't know where?

Many thanks to take the time to answer :-)

@masterleinad
Copy link
Member

Have a look at https://github.com/dealii/candi#configuration-file-options. You can set BLAS_DIR and LAPACK_DIR in candi.cfg. Also, if you don't need Trilinos in the first place, you can disable it there, too.

@koecher
Copy link
Contributor

koecher commented May 5, 2021

Dear @MarineLasbleis do you have the devel packages for blas and lapack installed on the machine?

On CentOS those are blas-devel and lapack-devel

@MarineLasbleis
Copy link

@koecher That's a very good question. I will verify with the sys admin!
I managed to install deal.II (at least apparently) asking candi to install openblas and zlib (even if my sys admin told me both should already be installed).
I'm installing deal.II for ASPECT, and apparently then the aspect installation is not correct, as it does not find zlib... (which should have been installed with deal ii as well as the path should be updated with the candi deal ii config file...). I have to dig through this.

@koecher
Copy link
Contributor

koecher commented May 7, 2021

@MarineLasbleis
Thanks for reporting. openblas through candi should be fine as well. I do have also the standard blas and lapack package files in a private repo, if you need them, let me know.

For zlib via candi everything should be there, but you may need to explicitly give aspect the new variables as listed in the config file / enable.sh for the configuration of cmake
We export 3 variables; cf. the very last lines of
https://github.com/dealii/candi/blob/dealii-9.2/deal.II-toolchain/packages/zlib.package

@koecher
Copy link
Contributor

koecher commented May 7, 2021 •
edited
Loading

@tjhei do you have additional help for installing aspect with the reported problem above?

@koecher
Copy link
Contributor

koecher commented Jun 21, 2021

Thank you to everyone.
Please reopen this issue or a new one, if the problem is still there.
I got a notice that also the petsc package may have issues with other blas/lapack libraries due to automatic downloads.

@koecher koecher closed this as completed Jun 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@tjhei @masterleinad @koecher @MarineLasbleis @UfrancoU