Homebrew openblas location

[mor1]

Installing owl.0.3.0 just now, I'm told

Warning: Use openblas instead of deprecated homebrew/science/openblas

brew remove openblas ; brew install openblas then gives me notice

==> Pouring openblas-0.2.20.high_sierra.bottle.tar.gz
==> Caveats
This formula is keg-only, which means it was not symlinked into /usr/local,
because macOS provides BLAS and LAPACK in the Accelerate framework.

For compilers to find this software you may need to set:
    LDFLAGS:  -L/usr/local/opt/openblas/lib
    CPPFLAGS: -I/usr/local/opt/openblas/include
For pkg-config to find this software you may need to set:
    PKG_CONFIG_PATH: /usr/local/opt/openblas/lib/pkgconfig

...which would explain why then opam install owl failed with # ld: library not found for -lopenblas.

brew link --force openblas seemed to fix things. (But may break things for other packages that depend on the Apple provided BLAS and LAPACK libs I guess.)

[ryanrhymes]

I think the best way is the path is correctly set during compiling and linking so that info can be saved in owl's lib (if my understanding about how ocaml build system works is correct), I previously made a conf-openblas lib but that does not work very well itself so I am not really using it atm. Yes, owl's compiling and linking is still a bit broken, need to find some time to fix it.

[vietlq]

I created PR: #139

[mars0i]

I'm not having any trouble on MacOS with the current jbuid files in the current master, fwiw. For a while I was modifying src/owl/ jbuild by hand, but the file has changed since then. @mor1 are you using the opam version or compiling from the repo? Maybe the problem has been fixed but the fix hasn't yet made it into an opam release? Then again @vietlq needed to fix something, in #139, so there may be variation in problems with finding libs on MacOS.

[vietlq]

@mars0i I used OPAM to install Owl and got the error. FYI, on Travis build for Owl on Ubuntu Trusty has been failing for quite some time as it couldn't find the library -lgfortran. I plan to add the fix in this PR too.

[mars0i]

My earlier comment was incorrect. I was in fact using a src/owl/jbuilder file that I had modified, but I didn't realize that. The modification was the addition of -L/usr/local/opt/openblas/lib.

[mor1]

(FWIW I was relying only on released things in OPAM and Homebrew.)

[mars0i]

@mor1, yes, that seems reasonable, but at the moment Owl has some issues with building on MacOS. Owl is great, but it's a work in progress in some respects. @vietlq's pull request that's been recently submitted that should fix it. At least that fix works on my systems.

As a temporary fix, one needs to add

    -L/usr/local/opt/openblas/lib 

after

    -L/usr/local/lib/gcc/7

in src/owl/jbuild, as indicated here (no reason that you should have known that). It looks like the only way to do this is to download the source, because even if you modify the source under .opam, opam will overwrite it when you run opam install owl. It's possible you'll also need to comment out or delete -flto.

However, once you get Owl to build using make, make install is all you need to do to add Owl to your OCaml libraries. I've never had a problem with that last step.

[mor1]

Thanks all. As I don't use openblas for anything else, I'm happy enough to have smashed it in place with homebrew :) Happy to go with that until @ryanrhymes has a chance to review #139 -- no rush!

[ryanrhymes]

@vietlq Thank you very much for fixing this, and my apologies for the delay. As @mars0i said, there is still a lot of work to do to improve Owl's building and deployment. I recently fixed the docker image to make sure it reflects the most up-to-date master branch, but the Travis CI is still broken, and I really appreciate if anyone can fix it. @mor1 yes, with homebrew, this jbuild should be fine.

[jhgorse]

I think the best way is the path is correctly set during compiling and linking so that info can be saved in owl's lib (if my understanding about how ocaml build system works is correct), I previously made a conf-openblas lib but that does not work very well itself so I am not really using it atm. Yes, owl's compiling and linking is still a bit broken, need to find some time to fix it.

Perhaps there could be an environment variable such as PKG_CONFIG_PATH or LDFLAGS and CPPFLAGS which would communicate to conf-openblas.0.2.1. The latest brew info openblas indicates this path for pkgconfig:

export PKG_CONFIG_PATH="/opt/homebrew/opt/openblas/lib/pkgconfig"

This broke for me with the following output:

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><>  🐫
⬇ retrieved owl.0.10.0  (cached)
[ERROR] The compilation of conf-openblas.0.2.1 failed at "sh -exc cc $CFLAGS
        -I/usr/local/opt/openblas/include test.c -L/usr/local/opt/openblas/lib -lopenblas".
⬇ retrieved eigen.0.3.0  (cached)
[ERROR] The compilation of eigen.0.3.0 failed at "dune build -p eigen -j 9".
... 
### output ###
# + cc -I/usr/local/opt/openblas/include test.c -L/usr/local/opt/openblas/lib -lopenblas
# test.c:2:10: fatal error: 'cblas.h' file not found
# #include <cblas.h>
#          ^~~~~~~~~
# 1 error generated.

[jhg:~] % brew link --force openblas does not resolve the issue.

How do I tell conf-openblas.0.2.1 where to find openblas?

Cheers,
Joe

[anentropic]

for me I initially get:

# [...]
# to the PKG_CONFIG_PATH environment variable
# No package 'openblas' found
# ++ pkg-config --libs openblas
# Package openblas was not found in the pkg-config search path.
# Perhaps you should add the directory containing `openblas.pc'
# to the PKG_CONFIG_PATH environment variable
# No package 'openblas' found
# + cc test.c
# test.c:2:10: fatal error: 'cblas.h' file not found
# #include <cblas.h>
#          ^~~~~~~~~
# 1 error generated.

from brew info openblas I got this:

For pkg-config to find openblas you may need to set:
  export PKG_CONFIG_PATH="/opt/homebrew/opt/openblas/lib/pkgconfig"

after exporting that:

$ opam install owl
The following actions will be performed:
  ∗ install conf-openblas 0.2.1   [required by owl]
  ∗ install owl           1.0.2-1
===== ∗ 2 =====
Do you want to continue? [Y/n] y

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><>  🐫
⬇ retrieved owl.1.0.2-1  (cached)
∗ installed conf-openblas.0.2.1
[ERROR] The compilation of owl.1.0.2-1 failed at "dune build -p owl -j 7".

#=== ERROR while compiling owl.1.0.2-1 ========================================#
# context     2.1.3 | macos/arm64 | ocaml-base-compiler.4.14.0 | https://opam.ocaml.org#c1188f86
# path        ~/.opam/oplottemp/.opam-switch/build/owl.1.0.2-1
# command     ~/.opam/opam-init/hooks/sandbox.sh build dune build -p owl -j 7
# exit-code   1
# env-file    ~/.opam/log/owl-35170-c91e6f.env
# output-file ~/.opam/log/owl-35170-c91e6f.out
### output ###
# File "src/aeos/dune", line 9, characters 9-32:
# 9 |   (names owl_aeos_tuner_map_stub owl_aeos_tuner_fold_stub)
#              ^^^^^^^^^^^^^^^^^^^^^^^
# (cd _build/default/src/aeos && /usr/bin/cc -O2 -fno-strict-aliasing -fwrapv -pthread -D_FILE_OFFSET_BITS=64 -g -O3 -Ofast -march=native -funroll-loops -ffast-math -DSFMT_MEXP=19937 -fno-strict-aliasing -g -I /Users/paul/.opam/oplottemp/lib/ocaml -o owl_aeos_tuner_map_stub.o -c owl_aeos_tuner_map_stub.c)
# clang: error: the clang compiler does not support '-march=native'
# File "src/aeos/dune", line 9, characters 33-57:
# 9 |   (names owl_aeos_tuner_map_stub owl_aeos_tuner_fold_stub)
#                                      ^^^^^^^^^^^^^^^^^^^^^^^^
# (cd _build/default/src/aeos && /usr/bin/cc -O2 -fno-strict-aliasing -fwrapv -pthread -D_FILE_OFFSET_BITS=64 -g -O3 -Ofast -march=native -funroll-loops -ffast-math -DSFMT_MEXP=19937 -fno-strict-aliasing -g -I /Users/paul/.opam/oplottemp/lib/ocaml -o owl_aeos_tuner_fold_stub.o -c owl_aeos_tuner_fold_stub.c)
# clang: error: the clang compiler does not support '-march=native'

<><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><>  🐫
┌─ The following actions failed
│ λ build owl 1.0.2-1
└─
┌─ The following changes have been performed
│ ∗ install conf-openblas 0.2.1
└─

I'm guessing this is because I'm on an M1 mac

I was previously able to install successfully in another project on this laptop but I forget if/what I did for it to work, not sure what might have changed

(not sure if this info is relevant: https://stackoverflow.com/a/66333485/202168 and https://stackoverflow.com/a/69926065/202168 ...FWIW I have clang 14.0.0 and apple-m1 cpu target is available)

[mseri]

For apple arm you need #625 (comment)

[anentropic]

For apple arm you need #625 (comment)

LOL thanks, I can see in the comments that 'old me' from 6 weeks ago had previously found that solution and then forgot about it 🤦‍♂️

[nilsbecker]

maybe it's worth looking at how the lacaml and gsl ocaml bindings handle this. i got those to install on my intel mac by setting environment variables that these bindings use for obtaining the lib locations, called LACAML_CFLAGS, LACAML_LIBS and GSL_CBLAS_LIB. so far no luck with owl + homebrew.

[2BitSalute]

Installed successfully by first finding the location of openblas.pc:

find `brew --prefix` -name openblas.pc

Then, I did what the error suggested and exported the path I got from the above in the environment PKG_CONFIG_PATH variable in .zshrc:

export PKG_CONFIG_PATH="/usr/local/Cellar/openblas/0.3.23/lib/pkgconfig/"

I then opened a new terminal window and reran opam install owl - success.