CMake can't find libomp on Apple M1

[IliaOzhmegov]

Description

CMake can't find libomp while using AppleClang Compiler.

Reproducible example

When running pip install lightgbm it tries to compile or link the lightgbm but it can't find libomp library.

Environment info

Apple M1
AppleClang 13.1.6.13160021
AppleClang 13.1.6.13160021
gcc 12.2.0
cmake 3.24.2
lightgbm 3.3.3
libomp 15.0.2
Python3.10

Command I used to install LightGBM

brew install gcc
brew install CMake
brew install libomp
brew install lightgbm
pip install lightgbm

-- The C compiler identification is AppleClang 13.1.6.13160021
-- The CXX compiler identification is AppleClang 13.1.6.13160021
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at /opt/homebrew/Cellar/cmake/3.24.1/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find OpenMP_C (missing: OpenMP_C_FLAGS OpenMP_C_LIB_NAMES)
Call Stack (most recent call first):
  /opt/homebrew/Cellar/cmake/3.24.1/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
  /opt/homebrew/Cellar/cmake/3.24.1/share/cmake/Modules/FindOpenMP.cmake:545 (find_package_handle_standard_args)
  CMakeLists.txt:135 (find_package)


-- Configuring incomplete, errors occurred!
See also "/private/var/folders/3j/d5d8004j5ts5z2xw7b7pdg2h0000gn/T/pip-install-6mkr3bp5/lightgbm_285ff78c433b47eabcf0fa3fab28cc02/build_cpp/CMakeFiles/CMakeOutput.log".
See also "/private/var/folders/3j/d5d8004j5ts5z2xw7b7pdg2h0000gn/T/pip-install-6mkr3bp5/lightgbm_285ff78c433b47eabcf0fa3fab28cc02/build_cpp/CMakeFiles/CMakeError.log".
-- The C compiler identification is AppleClang 13.1.6.13160021
-- The CXX compiler identification is AppleClang 13.1.6.13160021
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at /opt/homebrew/Cellar/cmake/3.24.1/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find OpenMP_C (missing: OpenMP_C_FLAGS OpenMP_C_LIB_NAMES)
Call Stack (most recent call first):
  /opt/homebrew/Cellar/cmake/3.24.1/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
  /opt/homebrew/Cellar/cmake/3.24.1/share/cmake/Modules/FindOpenMP.cmake:545 (find_package_handle_standard_args)
  CMakeLists.txt:135 (find_package)

[hcho3]

The new version of libomp (15.0) is keg-only, which means that we need to specify extra compilation flags to enable OpenMP. LightGBM used to specify these flags before. See dmlc/xgboost#8384 for an example fix.

[jameslamb]

Thanks @IliaOzhmegov for the report, and to @hcho3 for the extra information!

I've marked this blocking, since I see the exact same issue affecting most of LightGBM's macOS CI jobs.

Example builds from #5559:

• Azure DevOps (all Mac builds failing): https://dev.azure.com/lightgbm-ci/lightgbm-ci/_build/results?buildId=13788&view=logs&j=31d31d6b-9897-5af8-16dd-840b6b6b4fac
• GitHub Actions, R package (Mac + clang failing, Mac + gcc working): https://github.com/microsoft/LightGBM/actions/runs/3350110047/jobs/5550687360

---

Workarounds

If you're finding this from search and it's still open, that means we haven't fixed it yet.

Option 1: downgrade to OpenMP <12 (#4229)

Option 2: compile LightGBM without OpenMP support

• if compiling directly with CMake:

  •  mkdir build
     cd build
     cmake -DUSE_OPENMMP=OFF ..
     make -j2

• if building the Python package

  •  cd python-package
     pip install --nomp .

• if building the R package

  •  sh build-cran-package.sh --no-build-vignettes
     R CMD INSTALL ./lightgbm_*.tar.gz

[michaelgfeldman]

Hi!
I see you have closed the issue. Does it mean you fixed the bug?
I'm asking because i'm facing the very same bug right now on my m1 mac...
(libomp 15.0.4)

[jameslamb]

Yes, #5563 fixed this issue.

Please upgrade to the latest commit on master, clear your CMake cache, and try again. If it's still broken, please post the exact commit you're on, command(s) you ran, and error text and we'd be happy to investigate.

[michaelgfeldman]

thanks for such a quick response! 2 follow up questions:

1. How can i clean CMake cache?
2. Will this work for installing directly from a latest master?
   pip3 install git+https://github.com/microsoft/LightGBM.git#egg=lightgbm

[IliaOzhmegov]

@michaelgfeldman the CMake cache in the folder from which you were launching the command cmake ... In the instruction you used build folder.
https://lightgbm.readthedocs.io/en/latest/Installation-Guide.html#macos

So in order to clean cache you should run rm -rf * in the build folder.

[jameslamb]

Will this work for installing directly from a latest master?

I'd be very very surprised to learn that using a pip install git+ ever worked with LightGBM, given that LightGBM's Python package is not at the root of the repo.

When I try running that command.

pip install git+https://github.com/microsoft/LightGBM.git#egg=lightgbm

I get the following error basically saying that.

ERROR: lightgbm from git+https://github.com/microsoft/LightGBM.git#egg=lightgbm does not appear to be a Python project: neither 'setup.py' nor 'pyproject.toml' found

To build a development version of the Python package from source, follow the instructions at https://github.com/microsoft/LightGBM/tree/master/python-package#install-from-github.

[goraj]

Hi,
@jameslamb I have the same issue as @IliaOzhmegov. Building from recent main f1d3181 works.
Any ETA when the 3.3.3.99 will be available via pip?

[jameslamb]

The next release will be v4.0.0. You can subscribe to #5153 to be notified when it's released.

We're trying to get it out, but suffering from a severe lack of maintainer availability and attention.

[github-actions]

This issue has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.