Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support risc-v architecture #598

Merged
merged 3 commits into from Apr 1, 2022
Merged

Conversation

SpriteOvO
Copy link
Contributor

Currently this project cannot be compiled for risc-v architecture, gcc compiler raises an error:

/usr/bin/ld: libopendht.so.2.3.5: undefined reference to `__atomic_exchange_1'

This is because the risc-v architecture does not support 1-byte and 2-byte lock-free atomic operations.

By passing the -latomic option, gcc will statically link the libatomic library. Inside that library, the std::atomic<bool> used in the project will fallback to the mutex implementation, which is allowed by C++ standard.

Ref: riscv-collab/riscv-gnu-toolchain#183 (comment)

@r-value
Copy link

r-value commented Mar 28, 2022

It would be better to check if libatomic is necessary before linking against it.

The CheckAtomic module from llvm could be a reference: https://github.com/llvm/llvm-project/blob/main/llvm/cmake/modules/CheckAtomic.cmake

@SpriteOvO
Copy link
Contributor Author

I just noticed that CheckAtomic.cmake was included in the project over a year ago, but it doesn't seem to be working and I'm doing some investigation.

@SpriteOvO
Copy link
Contributor Author

Updated. Link libatomic only if the atomic tests fail, to avoid other architectures linking it unnecessarily.

CMakeLists.txt Outdated Show resolved Hide resolved
CMakeLists.txt Outdated Show resolved Hide resolved
CMakeLists.txt Outdated Show resolved Hide resolved
# This isn't necessary on MSVC, so avoid command-line switch annoyance
# by only running on GCC-like hosts.
if (LLVM_COMPILER_IS_GCC_COMPATIBLE)
# First check if atomics work without the library.
check_working_cxx_atomics(HAVE_CXX_ATOMICS_WITHOUT_LIB)
# If not, check if the library exists, and atomics work with it.
if(NOT HAVE_CXX_ATOMICS_WITHOUT_LIB)
check_library_exists(atomic __atomic_fetch_add_4 "" HAVE_LIBATOMIC)
Copy link

@r-value r-value Mar 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to make some further explanation here.

CHECK_LIBRARY_EXISTS won't work if the function to look for has a built-in version. The test program will cause a signature conflict thus fail to compile, leading to a false negative. This line and related branches are removed to avoid that.

@aberaud aberaud merged commit e753d20 into savoirfairelinux:master Apr 1, 2022
@aberaud
Copy link
Member

aberaud commented Apr 2, 2022

I reverted the changes in CMakeLists because adding the library is supposed to be automatic through CMAKE_REQUIRED_LIBRARIES if I understand correctly.
After the merge of this commit, the build was failing on macOS because libatomic couldn't be found.

@SpriteOvO
Copy link
Contributor Author

SpriteOvO commented Apr 2, 2022

@aberaud In fact, in my previous tests, CMake does not automatically link to libraries in CMAKE_REQUIRED_LIBRARIES, and I didn't find documentation describing this behavior. If we search in the CMake repository, we can find some set(CMAKE_REQUIRED_LIBRARIES xxx) usage, so we presume that the variable is volatile.

Could we get some error messages on macOS? I didn't find it from the CI. This change is not expected to affect macOS, maybe there are some bugs and I will try to fix it :)

@SpriteOvO
Copy link
Contributor Author

Could we get some error messages on macOS?

A CMakeError.log file would be better.

@aberaud
Copy link
Member

aberaud commented Apr 2, 2022

Looking at various examples as well as the original CheckAtomic from llvm, it seems like using CMAKE_REQUIRED_LIBRARIES is how people do it:
https://stackoverflow.com/questions/69281559/cmake-library-order-in-cmake-required-libraries-to-test-a-minimal-program-while

The error on macOS:
Note that the same error happens when linking the first executable no matter what build options are used.

cmake:

% cmake -DOPENDHT_C=On -DOPENDHT_TESTS=On -DCMAKE_BUILD_TYPE=Release -DCMAKE_INTERPROCEDURAL_OPTIMIZATION=On ..
-- The C compiler identification is AppleClang 13.1.6.13160021
-- The CXX compiler identification is AppleClang 13.1.6.13160021
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PkgConfig: /opt/homebrew/bin/pkg-config (found version "0.29.2") 
-- Performing Test HAVE_CXX_ATOMICS64_WITHOUT_LIB
-- Performing Test HAVE_CXX_ATOMICS64_WITHOUT_LIB - Success
-- Performing Test LLVM_HAS_ATOMICS
-- Performing Test LLVM_HAS_ATOMICS - Success
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) 
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Found GnuTLS: /opt/homebrew/Cellar/gnutls/3.7.4/lib/libgnutls.dylib (found suitable version "3.7.4", minimum required is "3.3") 
-- Checking for one of the modules 'nettle'
-- Found Readline: /Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/lib/libreadline.tbd (Required is at least version "6") 
-- Checking for one of the modules 'libargon2'
-- Checking for one of the modules 'jsoncpp'
-- Checking for one of the modules 'cppunit'
-- Configuring done
-- Generating done
-- Build files have been written to: opendht/build

build:

% make VERBOSE=1
...
[ 82%] Linking CXX executable opendht_unit_tests
/opt/homebrew/Cellar/cmake/3.23.0/bin/cmake -E cmake_link_script CMakeFiles/opendht_unit_tests.dir/link.txt --verbose=1
/Library/Developer/CommandLineTools/usr/bin/c++  -Wno-return-type -Wall -Wextra -Wnon-virtual-dtor -pedantic-errors -fvisibility=hidden -DMSGPACK_NO_BOOST -DMSGPACK_DISABLE_LEGACY_NIL -DMSGPACK_DISABLE_LEGACY_CONVERT -O3 -DNDEBUG -flto=thin -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk -mmacosx-version-min=12.2 -Wl,-search_paths_first -Wl,-headerpad_max_install_names CMakeFiles/opendht_unit_tests.dir/tests/tests_runner.cpp.o CMakeFiles/opendht_unit_tests.dir/tests/infohashtester.cpp.o CMakeFiles/opendht_unit_tests.dir/tests/valuetester.cpp.o CMakeFiles/opendht_unit_tests.dir/tests/cryptotester.cpp.o CMakeFiles/opendht_unit_tests.dir/tests/dhtrunnertester.cpp.o CMakeFiles/opendht_unit_tests.dir/tests/threadpooltester.cpp.o CMakeFiles/opendht_unit_tests.dir/tests/peerdiscoverytester.cpp.o -o opendht_unit_tests   -L/opt/homebrew/Cellar/nettle/3.7.3/lib  -L/opt/homebrew/Cellar/argon2/20190702_1/lib  -L/opt/homebrew/Cellar/cppunit/1.15.1/lib  -Wl,-rpath,/opt/homebrew/Cellar/nettle/3.7.3/lib -Wl,-rpath,/opt/homebrew/Cellar/argon2/20190702_1/lib -Wl,-rpath,/opt/homebrew/Cellar/cppunit/1.15.1/lib -Wl,-rpath,opendht/build libopendht.2.4.0.dylib -lcppunit /opt/homebrew/Cellar/gnutls/3.7.4/lib/libgnutls.dylib -latomic 
ld: library not found for -latomic
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [opendht_unit_tests] Error 1
make[1]: *** [CMakeFiles/opendht_unit_tests.dir/all] Error 2
make: *** [all] Error 2

@SpriteOvO
Copy link
Contributor Author

SpriteOvO commented Apr 3, 2022

@aberaud I did some investigation and I now understand how everything is going on.


Looking at various examples as well as the original CheckAtomic from llvm, it seems like using CMAKE_REQUIRED_LIBRARIES is how people do it: stackoverflow.com/questions/69281559/cmake-library-order-in-cmake-required-libraries-to-test-a-minimal-program-while

I wrote a minimal example for testing and the result is that CMAKE_REQUIRED_LIBRARIES does not work for a target. It works for CHECK_CXX_SOURCE_COMPILES.

Quoted from the documentation of check_cxx_source_compiles:

CMAKE_REQUIRED_LIBRARIES

A ;-list of libraries to add to the link command. These can be the name of system libraries or they can be Imported Targets (see try_compile() for further details).

The code in the StackOverflow question you mentioned also implies my point.

Quoted from the StackOverflow question code:

# Before setting CMAKE_REQUIRED_FLAGS, we preserve the current state
cmake_push_check_state()

set(CMAKE_REQUIRED_LIBRARIES "-L${atomic_lib_dir} -latomic")
check_cxx_source_compiles("${ATOMIC32_TEST_CODE}" atomic32_test_with_atomic_linking)
check_cxx_source_compiles("${ATOMIC64_TEST_CODE}" atomic64_test_with_atomic_linking)

cmake_pop_check_state()

I treat the CMAKE_REQUIRED_FLAGS in the comment as a typo, which should be CMAKE_REQUIRED_LIBRARIES, because CMAKE_REQUIRED_FLAGS is not used anywhere in his code.

Quoted from the documentation of cmake_push_check_state:

This module defines three macros: CMAKE_PUSH_CHECK_STATE() CMAKE_POP_CHECK_STATE() and CMAKE_RESET_CHECK_STATE() These macros can be used to save, restore and reset (i.e., clear contents) the state of the variables CMAKE_REQUIRED_FLAGS, CMAKE_REQUIRED_DEFINITIONS, CMAKE_REQUIRED_LINK_OPTIONS, CMAKE_REQUIRED_LIBRARIES, CMAKE_REQUIRED_INCLUDES and CMAKE_EXTRA_INCLUDE_FILES used by the various Check-files coming with CMake, like e.g. check_function_exists() etc. The variable contents are pushed on a stack, pushing multiple times is supported. This is useful e.g. when executing such tests in a Find-module, where they have to be set, but after the Find-module has been executed they should have the same value as they had before.

CMAKE_PUSH_CHECK_STATE() macro receives optional argument RESET. Whether it's specified, CMAKE_PUSH_CHECK_STATE() will set all CMAKE_REQUIRED_* variables to empty values, same as CMAKE_RESET_CHECK_STATE() call will do.


The error on macOS:

This is because the LLVM_COMPILER_IS_GCC_COMPATIBLE variable in the CheckAtomic.cmake file is not working properly.

Let's take a look at the code snippet:

# This isn't necessary on MSVC, so avoid command-line switch annoyance
# by only running on GCC-like hosts.
if (LLVM_COMPILER_IS_GCC_COMPATIBLE)
  # First check if atomics work without the library.
  check_working_cxx_atomics(HAVE_CXX_ATOMICS_WITHOUT_LIB)
  # If not, check if the library exists, and atomics work with it.
  if(NOT HAVE_CXX_ATOMICS_WITHOUT_LIB)
    list(APPEND CMAKE_REQUIRED_LIBRARIES "atomic")
    check_working_cxx_atomics(HAVE_CXX_ATOMICS_WITH_LIB)
    if (NOT HAVE_CXX_ATOMICS_WITH_LIB)
      message(FATAL_ERROR "Host compiler must support std::atomic!")
    endif()
  endif()
endif()

The LLVM_COMPILER_IS_GCC_COMPATIBLE variable is actually set from DetermineGCCCompatible.cmake file, obviously it does not exist in this repo.

When LLVM_COMPILER_IS_GCC_COMPATIBLE is not set (is OFF), so the atomic check is not performed, so HAVE_CXX_ATOMICS_WITHOUT_LIB is always OFF, so

if (NOT HAVE_CXX_ATOMICS_WITHOUT_LIB)
    target_link_libraries(opendht PUBLIC atomic)
endif ()

always links to libatomic for the target.

Here's an same bug report from another project: CheckAtomic doesn't set LLVM_COMPILER_IS_GCC_COMPATIBLE and fails check on Apple Clang


I would wait for the macOS CI of this repo to work properly and then open a new PR. If you have any questions let me know :)

@aberaud
Copy link
Member

aberaud commented Apr 3, 2022

Thanks for the detailed analysis and explanations !

  • Shall we then use the generic link_libraries instead of per-target with target_link_libraries ?
  • The macOS CI job should now be working well enough :)

@SpriteOvO
Copy link
Contributor Author

  • Shall we then use the generic link_libraries instead of per-target with target_link_libraries ?

Of course, it should work.

  • The macOS CI job should now be working well enough :)

Nice! I will open a PR later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants