Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update CMakeLists.txt which fixes debug build undefined reference errors #154

Closed
wants to merge 1 commit into from

Conversation

ChenYuHo
Copy link

@ChenYuHo ChenYuHo commented Nov 27, 2023

Building with debug

./build/astra_analytical/build.sh -d

gives errors while linking:

ld: [100%] Linking CXX executable ../bin/AstraSim_Analytical_Congestion_Aware
../lib/libAstraSim.a(et_feeder.cpp.o): in function `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >* absl::lts_20230802::log_internal::MakeCheckOpString<int const&, int const&>(int const&, int const&, char const*)':
env/include/absl/log/internal/check_op.h:292: undefined reference to `absl::lts_20230802::log_internal::CheckOpMessageBuilder::CheckOpMessageBuilder(char const*)'
env/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: env/include/absl/log/internal/check_op.h:294: undefined reference to `absl::lts_20230802::log_internal::CheckOpMessageBuilder::ForVar2()'
env/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: env/include/absl/log/internal/check_op.h:295: undefined reference to `absl::lts_20230802::log_internal::CheckOpMessageBuilder::NewString[abi:cxx11]()'

# skipped for brevity

Following protocolbuffers/protobuf#12292 issue discussions:

I believe the issue is that something like find_package(protobuf REQUIRED) uses the embedded FindProtobuf package from CMake:

https://cmake.org/cmake/help/latest/module/FindProtobuf.html

This package has not been updated to know about the Abseil dependency. One needs to use:

find_package(protobuf REQUIRED CONFIG)

and protocolbuffers/protobuf#4806:

You should not need any of these variables. Just link against protobuf::libprotobuf target, it will also find the headers correctly!

This PR fixes the problem in my environment.

@TaekyungHeo
Copy link
Collaborator

Hello, @ChenYuHo.

I am experiencing the following error when I build ASTRA-sim with your PR.

$ git clone --recurse-submodules git@github.com:ChenYuHo/astra-sim.git
$ cd astra-sim
$ git checkout patch-1
$ docker build -t astra-sim . && docker run -it astra-sim
root@5d065abfb9ae:/app/astra-sim# ./build/astra_analytical/build.sh
-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at ../../CMakeLists.txt:17 (find_package):
  Could not find a package configuration file provided by "Protobuf" with any
  of the following names:

    ProtobufConfig.cmake
    protobuf-config.cmake

  Add the installation prefix of "Protobuf" to CMAKE_PREFIX_PATH or set
  "Protobuf_DIR" to a directory containing one of the above files.  If
  "Protobuf" provides a separate development package or SDK, be sure it has
  been installed.


-- Configuring incomplete, errors occurred!
See also "/app/astra-sim/build/astra_analytical/build/CMakeFiles/CMakeOutput.log".
root@5d065abfb9ae:/app/astra-sim# ./build/astra_analytical/build.sh
CMake Error at ../../CMakeLists.txt:17 (find_package):
  Could not find a package configuration file provided by "Protobuf" with any
  of the following names:

    ProtobufConfig.cmake
    protobuf-config.cmake

  Add the installation prefix of "Protobuf" to CMAKE_PREFIX_PATH or set
  "Protobuf_DIR" to a directory containing one of the above files.  If
  "Protobuf" provides a separate development package or SDK, be sure it has
  been installed.


-- Configuring incomplete, errors occurred!
See also "/app/astra-sim/build/astra_analytical/build/CMakeFiles/CMakeOutput.log".

@TaekyungHeo TaekyungHeo self-assigned this Nov 28, 2023
@ChenYuHo
Copy link
Author

ChenYuHo commented Nov 28, 2023

I'm using an conda environment which is built with

mamba create --strict-channel-priority --yes -p ./env \
gdb fsspec gitpython scipy future numpy nvidia-apex setuptools python-graphviz graphviz networkx pydot pip cuda-cudart-dev cuda-toolkit cuda-nvcc cuda-version=11.8 cuda-libraries-dev make openmpi mpi cmake mkl mkl-include pkg-config git boost-cpp libboost-devel libprotobuf=4  protobuf=4 \
transformers datasets tqdm sentencepiece sentencepiece-python `# for pytorch examples FSDP` \
pytorch=2.1 torchvision torchaudio cuda pytorch-cuda=11.8 compilers sysroot_linux-64==2.17 gcc==11.4 ninja py-cpuinfo libaio ca-certificates certifi openssl python=3 pydantic `#deepspeed` \
-c pytorch -c nvidia/label/cuda-11.8.0 -c conda-forge

I believe the issue stems from how CMake handles protobuf libraries, and since I'm using a newer CMake from conda, maybe this error only happens with the newer CMake, and my patch thus applies only to that case.

If the original CMakeLists.txt works for the older CMake, maybe the correct way is to check whether ProtobufConfig.cmake or protobuf-config.cmake exists?

@TaekyungHeo
Copy link
Collaborator

It seems that ProtobufConfig.cmake or protobuf-config.cmake is not installed by default. When building ASTRA-sim in debug mode, I encountered a different error than what you provided. It appears that the issue isn't confined to debug mode alone but rather pertains to linking errors encountered during the ASTRA-sim build process.

@jinsun-yoo
Copy link
Collaborator

We're closing this PR for now. If the issue still persists, please resubmit a new issue/PR and we'll have another look at a new environment.

@jinsun-yoo jinsun-yoo closed this Apr 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants