add additional environment variables and add libraries/flags to linki…

[jcreinhold]

RFC

This PR addresses the problems raised in #635

For non-standard build setups, this adds OWL_LDFLAGS, OWL_CPPFLAGS, and OWL_LDLIBS which correspond to the usual gcc flags.

I personally don't think these are excessive, but we could change this to only add OWL_LDFLAGS (or, e.g., OWL_LIBRARY_FLAGS) which would be sufficient if this PR adds too many new environment variables. The current implementation in this PR, however, allows the user to specify flags like they would if they were directly using gcc. Even though the flags are eventually combined into c_flags and c_library_flags, this allows users to not cram everything into something labeled OWL_CFLAGS and (the minimally required for this to build) OWL_LDFLAGS.

This also adds the generated flags to the OpenBLAS linking tests with c_test which were previously failing for the reasons mentioned in #635. This change is required for non-standard builds; the linking test will fail simply because it's lacking the correct flags—even though owl would build with OWL_CFLAGS and something akin to OWL_LDFLAGS.

[jcreinhold]

FWIW, this solves many of the install problems (really linking problems) that occur with this library; albeit, this places the responsibility on the user to correctly configure the library. However, we're implicitly relying on them doing this by asking them to ln -s certain libraries to certain locations so they can be found by the compiler.

For example, I encountered problems linking libquadmath.dylib on my Macbook when I was installing the library. I am using a Apple M1 processor, so I incorporated the modifications to owl_core_utils.c and owl_macros.h listed in this PR—keeping everything else as in this current PR.

Instead of doing ad hoc ln -s or other work arounds that are particularly fragile, I was able to add OWL_LDFLAGS="-L/Users/jcreinhold/homebrew/Cellar/gcc/12.2.0/lib/gcc/current" and OWL_LDLIBS="-lquadmath" and successfully installed owl (building from source).

I was also able to use this PR (and @mseri aarch64 fix) to build this package at work on a very painful build system.

While it's a bit of a burden for the user to learn about how to configure this, I could add in some tutorial material, and perhaps an install FAQ based on the install issues, and the future install problems can be reduced to figuring out the right combination of flags to pass via these proposed environment variables.

[nilsbecker]

I would very much welcome such a PR. I am used to setting linking and C flag variables for installing lacaml and gsl, and it works well there to use homebrew installed libs after reading some basic instructions in the respective READMEs.
(Currently, not sure how to get owl running on my m1 mac. Can I opam pin some fork of owl to get this PR + the relevant bits of the arm64 PR?)

[mseri]

@nilsbecker I included the change both in my arm64 and arm64-bak (which includes also changes from the open PRs) branches if you want to test it out

[nilsbecker]

i just tied #625 (comment) and that worked for me -- i guess it's using your updated arm64 branch yes?

[mseri]

Yes

[nilsbecker]

On another M1 mac I tried the same but keep getting a linker error that says ld: library 'lapacke' not found. any hints? brew install lapack did not help so far. It's confusing as I'm not aware of relevant differences between the two computers.

one more try:
PKG_CONFIG_PATH="/opt/homebrew/opt/lapack/lib/pkgconfig:/opt/homebrew/opt/openblas/lib/pkgconfig" opam install owl
fails with the same error, although pkg-config can find version 3.11 of lapacke with this path...

[oemmerson]

@mseri How come the changes in your fork to support the M1 haven't been merged?

Thanks for the work to fix it BTW.

[mseri]

On another M1 mac I tried the same but keep getting a linker error that says ld: library 'lapacke' not found. any hints? brew install lapack did not help so far. It's confusing as I'm not aware of relevant differences between the two computers.

I don't know. You can check if OWL_DISABLE_LAPACKE_LINKING_FLAG=1 fixes it

[nilsbecker]

OWL_DISABLE_LAPACKE_LINKING_FLAG=1

indeed that did allow it to install! the working command line was:
PKG_CONFIG_PATH="/opt/homebrew/opt/openblas/lib/pkgconfig" OWL_DISABLE_LAPACKE_LINKING_FLAG=1 opam install owl.1.1.0

i.e. without adding a path to the separately installed lapack library. no idea why it was necessary on this machine and not the other one.

[nilsbecker]

ok, while disabling the lapacke linking did allow owl to get installed, it then does not work properly in an application. i get loads of linking errors saying that some lapacke symbols are not found.

adding the brew-installed lapack to the PKG_CONFIG_PATH when installing owl does not work either. so still no luck overall

[mseri]

I don't know, indeed OWL_DISABLE_LAPACKE_LINKING_FLAG=1 should not exist imho. If pkg-config has reasons to link lapacke it means it should be linked, but it was worth a try. Not sure what happens, do you have a mix of x86 and arm pagackes installed via homebrew? That can mess it up for example

[nilsbecker]

no that should not be the problem. i did a rebuild of the homebrew installation from an exported package list.

i just noticed that i have an env variable set:
OWL_MANUAL_LDFLAGS=-L/opt/homebrew/opt/gfortran/lib/gcc/current -L/opt/homebrew/opt/openblas/lib
which i define in my bash_profile. i don't remember where this came from (shame on me). is this something recognized by owl? should i unset it?

also my default PKG_CONFIG_PATH=/opt/homebrew/opt/openblas/lib/pkgconfig:/opt/X11/lib/pkgconfig:/usr/X11/lib/pkgconfig...

[nilsbecker]

i think the manual linking flags were from a previous workaround for installing owl. i just retried with LDFLAGS=$OWL_MANUAL_LDFLAGS opam install owl.1.1.0 but that did not remove the lapacke linking error, alas.

[nilsbecker]

Ok, I've got it now. Mea culpa. For a reason I cannot fathom, my full opam switch was an x86_64 binary without me knowing it. It silently ran under Rosetta2 but when trying to link arm64-only libraries, that did not work anymore somehow. So @mseri 's initial hunch was correct: some x86 cruft was still around: all of the switch!

I was really convinced that this was a new install of ocaml via opam and not an old file tree transferred from a backup, but now i'm not so sure anymore.

Bottom line: the intallation instructions referenced above, pinning @mseri 's branch do work for me now.

[jzstark]

Close for now since the configuration has been updated.