Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
New build system (#730)
* something working

* progress

* progress

* progress

* cmake: fix common and add ctypes

* src/serdes: new build system

* new build system for pdbparser

* new build system for yaracpp

* new build system for crypto component

* new build system for config

* refactor cmake for retdec-configtool

* new build system for idr2pat

* new build system for ar-extractor and its LLVM dependency

* refactor the new build system

* new build for ar-extractortool

* new build system for macho-extractor and macho-extractortool

* new build for ctypesparser, demangler, and demanglertool

* new build for llvm-support and llvmir-emul

* new build for capstone2llvmir and capstone2llvmirtool

* new build for fileformat, loader, cpdetect, fileinfo

* new build for most remaining modules

* cmake/options.cmake: fix debugformat enable settings

* some build system fixes

* cmake/options.cmake: make RETDEC_ENABLE_ALL into option

this allows us to set it to ODD and build only documentation

* tests: new build system for unit tests

* pelib: fix doxygen comment

* retdec/retdec: remove unused include of llvm-support/diagnostics.h

* fileformat: do not include openssl in headers

* cmake: make install paths relative.

When they are absolute, then all the paths in instaled cmake scripts are absolute - install dir could not be moved.

* deps/yara: refactor cmake to properly install libs

* deps/yara: small cosmetic changes

* deps/llvm: refactor cmake to properly install libs

* deps/capstone: refactor cmake to properly install libs

* deps: refactor cmake files

* deps: refactor cmake files

* deps/yaramod: refactor cmake to properly install libs

* CMakeLists.txt: fix files installation

* config/retdec-config-config.cmake: fix required component order

Looks like this may matter (on some machines or cmake version).

* deps/llvm: fix include installation

* fileformat/cmake: do not use openssl-crypto

For some reasons, this can cause linking problems on some machines.

* deps/yaramod: fix link library order

* deps/googletests: fix target link library order

* rtti-finder: make deps PUBLIC.

I have no idea why it doesn't work with PRIVATE.

* deps/yaramod: fix formatting

* cmake: more refactorization...

1. Protect against including the same target file multiple times. It looks like this was not a problem, but who knows.
2. Use find_package() instead of find_dependency(). It looks like the later does not work correctly for components on some CMake versions.

* cmake: do not create version files for all components

Only the main RetDec version file is needed.

* cmake: propagate current project version to package config files

It is used to find the same version retdec components.

* CMakeLists.txt: replace AnyNewerVersion -> ExactVersion in compatibility settings

Since we probably will make breaking changes, make retdec installation compatible only with the same version.

* cmake: remove all COMPONENT options, these are not really needed

* cmake: move dependency finders inside double target protection conditions

* cmake: prefix all deps targets with retdec::deps::

* fileinfo: add openssl-crypto dep

* Revert "fileinfo: add openssl-crypto dep"

This reverts commit e1645d7.

* remove RETDEC_FORCE_OPENSSL_BUILD option, always build OpenSSL

* cmake: refactorization

* deps/whereami: fix include paths

* deps/tinyxml+whereami: fix included dirs

* deps/yaramod: fix pog_fmt lib installation

* deps/whereami: properly install target, not as part of utils.

Previous version should work, but there is a CMake bug in older versions which screws it up on Windows and macOS.

* crypto: link retdec::deps::openssl-crypto as PUBLIC

* bin2llvmir/providers/debugformat: fix segfault

Co-authored-by: Peter Matula <imatula@fit.vutbr.cz>
  • Loading branch information
PeterMatula and Peter Matula committed Apr 5, 2020
1 parent 5c9c785 commit 8fea90b
Show file tree
Hide file tree
Showing 412 changed files with 3,860 additions and 904 deletions.
69 changes: 63 additions & 6 deletions CMakeLists.txt
@@ -1,28 +1,62 @@

cmake_minimum_required(VERSION 3.6)
cmake_minimum_required(VERSION 3.8)

project(retdec C CXX)

include(GNUInstallDirs)
project(retdec
LANGUAGES C CXX
VERSION 4.0
)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)

include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/utils.cmake)

# Set the default build type to 'Release'.
if(NOT CMAKE_BUILD_TYPE)
set(default_build_type "Release")
message(STATUS "Setting build type to '${default_build_type}' as none was specified.")
set(CMAKE_BUILD_TYPE "${default_build_type}" CACHE STRING "Choose the type of build." FORCE)
endif()

## Includes.
include(GNUInstallDirs)
include(CMakePackageConfigHelpers)
include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/utils.cmake)
include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/options.cmake)

# RetDec, and some dependencies (e.g. LLVM, Keystone), require Python 3.
find_package(PythonInterp 3.4 REQUIRED)

### Variables.

## Repository directories.
set(RETDEC_CMAKE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/cmake")
set(RETDEC_DEPS_DIR "${CMAKE_CURRENT_SOURCE_DIR}/deps")
set(RETDEC_DOC_DIR "${CMAKE_CURRENT_SOURCE_DIR}/doc")
set(RETDEC_INCLUDE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/include")
set(RETDEC_SCRIPTS_DIR "${CMAKE_CURRENT_SOURCE_DIR}/scripts")
set(RETDEC_SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/src")
set(RETDEC_SUPPORT_DIR "${CMAKE_CURRENT_SOURCE_DIR}/support")
set(RETDEC_TESTS_DIR "${CMAKE_CURRENT_SOURCE_DIR}/tests")
## Installation directories.
# Bins.
set(RETDEC_INSTALL_BIN_DIR "${CMAKE_INSTALL_BINDIR}")
set(RETDEC_INSTALL_BIN_DIR_ABS "${CMAKE_INSTALL_PREFIX}/${RETDEC_INSTALL_BIN_DIR}")
set(RETDEC_INSTALL_TESTS_DIR "${RETDEC_INSTALL_BIN_DIR}")
# Includes.
set(RETDEC_INSTALL_INCLUDE_DIR "${CMAKE_INSTALL_INCLUDEDIR}")
set(RETDEC_INSTALL_DEPS_INCLUDE_DIR "${RETDEC_INSTALL_INCLUDE_DIR}/retdec/deps")
# Libs.
set(RETDEC_INSTALL_LIB_DIR "${CMAKE_INSTALL_LIBDIR}")
set(RETDEC_INSTALL_LIB_DIR_ABS "${CMAKE_INSTALL_PREFIX}/${RETDEC_INSTALL_LIB_DIR}")
set(RETDEC_INSTALL_DEPS_LIB_DIR "${RETDEC_INSTALL_LIB_DIR}/deps")
set(RETDEC_INSTALL_DEPS_LIB_DIR_ABS "${CMAKE_INSTALL_PREFIX}/${RETDEC_INSTALL_DEPS_LIB_DIR}")
# Data.
set(RETDEC_INSTALL_DATA_DIR "${CMAKE_INSTALL_DATADIR}/retdec")
set(RETDEC_INSTALL_CMAKE_DIR "${RETDEC_INSTALL_DATA_DIR}/cmake")
set(RETDEC_INSTALL_DOC_DIR "${RETDEC_INSTALL_DATA_DIR}/doc")
set(RETDEC_INSTALL_SUPPORT_DIR "${RETDEC_INSTALL_DATA_DIR}/support")
set(RETDEC_INSTALL_SUPPORT_DIR_ABS "${CMAKE_INSTALL_PREFIX}/${RETDEC_INSTALL_SUPPORT_DIR}")

# On Linux and macOS, set RPATH relative to the origin of the installed
# executables (i.e. relative to the bin directory). This allows us to move the
# installation directory into a different location after installation, which is
Expand Down Expand Up @@ -111,3 +145,26 @@ cond_add_subdirectory(scripts RETDEC_ENABLE_SCRIPTS)
add_subdirectory(src)
cond_add_subdirectory(support RETDEC_ENABLE_SUPPORT)
add_subdirectory(tests)

# Create config version file.
write_basic_package_version_file(
"${CMAKE_CURRENT_BINARY_DIR}/retdec-config-version.cmake"
VERSION ${PROJECT_VERSION}
COMPATIBILITY ExactVersion
)

# Create main RetDec CMake config file.
configure_file(
"${CMAKE_CURRENT_LIST_DIR}/retdec-config.cmake"
"${CMAKE_CURRENT_BINARY_DIR}/retdec-config.cmake"
@ONLY
)

# Install CMake files.
install(
FILES
"${CMAKE_CURRENT_BINARY_DIR}/retdec-config.cmake"
"${CMAKE_CURRENT_BINARY_DIR}/retdec-config-version.cmake"
DESTINATION
"${RETDEC_INSTALL_CMAKE_DIR}"
)
2 changes: 0 additions & 2 deletions README.md
Expand Up @@ -155,7 +155,6 @@ sudo pacman --needed -S base-devel cmake git perl python3 autoconf automake libt
* [CMake](https://cmake.org/) (version >= 3.6)
* [Git](https://git-scm.com/)
* [Active Perl](https://www.activestate.com/activeperl). It needs to be the first Perl in `PATH`, or it has to be provided to CMake using `CMAKE_PROGRAM_PATH` variable, e.g. `-DCMAKE_PROGRAM_PATH=/c/perl/bin`. Does NOT work with Strawberry Perl or MSYS2 Perl (you would have to install a pre-built version of OpenSSL, see below).
* Alternatively, you can install OpenSSL directly from [here](https://slproweb.com/products/Win32OpenSSL.html). This means OpenSSL won't be built and you don't need to install any Perl. Do not install Light version of OpenSSL as they don't contain development files.
* [Python](https://www.python.org/) (version >= 3.4)
* Optional: [Doxygen](http://ftp.stack.nl/pub/users/dimitri/doxygen-1.8.13-setup.exe) and [Graphviz](https://graphviz.gitlab.io/_pages/Download/windows/graphviz-2.38.msi) for generating API documentation

Expand Down Expand Up @@ -238,7 +237,6 @@ You can pass the following additional parameters to `cmake`:
* `-DRETDEC_DOC=ON` to build with API documentation (requires Doxygen and Graphviz, disabled by default).
* `-DRETDEC_TESTS=ON` to build with tests (disabled by default).
* `-DRETDEC_DEV_TOOLS=ON` to build with development tools (disabled by default).
* `-DRETDEC_FORCE_OPENSSL_BUILD=ON` to force OpenSSL build even if it is installed in the system (disabled by default).
* `-DRETDEC_COMPILE_YARA=OFF` to disable YARA rules compilation at installation step (enabled by default).
* `-DCMAKE_BUILD_TYPE=Debug` to build with debugging information, which is useful during development. By default, the project is built in the `Release` mode. This has no effect on Windows, but the same thing can be achieved by running `cmake --build .` with the `--config Debug` parameter.
* `-DCMAKE_PROGRAM_PATH=<path>` to use Perl at `<path>` (probably useful only on Windows).
Expand Down
21 changes: 7 additions & 14 deletions cmake/options.cmake
Expand Up @@ -4,7 +4,6 @@
option(RETDEC_DOC "Build public API documentation (requires Doxygen)." OFF)
option(RETDEC_TESTS "Build tests." OFF)
option(RETDEC_DEV_TOOLS "Build dev tools." OFF)
option(RETDEC_FORCE_OPENSSL_BUILD "Force OpenSSL build." OFF)
option(RETDEC_COMPILE_YARA "Compile YARA rules at installation." ON)
option(RETDEC_MSVC_STATIC_RUNTIME "Use a multi-threaded statically-linked runtime library." OFF)

Expand Down Expand Up @@ -59,7 +58,7 @@ include(${PROJECT_SOURCE_DIR}/cmake/deps.cmake)

# Default: ALL build is enabled.
#
set(RETDEC_ENABLE_ALL ON)
option(RETDEC_ENABLE_ALL "" ON)

# Convert target list in RETDEC_ENABLE to RETDEC_ENABLE_<component> flags.
#
Expand Down Expand Up @@ -298,20 +297,12 @@ set_if_at_least_one_set(RETDEC_ENABLE_DEBUGFORMAT
RETDEC_ENABLE_ALL
RETDEC_ENABLE_BIN2LLVMIR)

set_if_at_least_one_set(RETDEC_ENABLE_CTYPESPARSER
RETDEC_ENABLE_ALL
RETDEC_ENABLE_BIN2LLVMIR
RETDEC_ENABLE_DEMANGLER)

set_if_at_least_one_set(RETDEC_ENABLE_DEMANGLER
RETDEC_ENABLE_ALL
RETDEC_ENABLE_BIN2LLVMIR
RETDEC_ENABLE_CTYPESPARSER
RETDEC_ENABLE_DEBUGFORMAT
RETDEC_ENABLE_DEMANGLERTOOL)

# TODO: second run because demangler and ctypesparser are cyclically dependent.
# This dependency should be avoided.
set_if_at_least_one_set(RETDEC_ENABLE_CTYPESPARSER
RETDEC_ENABLE_ALL
RETDEC_ENABLE_BIN2LLVMIR
Expand All @@ -322,12 +313,12 @@ set_if_at_least_one_set(RETDEC_ENABLE_CTYPES
RETDEC_ENABLE_CTYPESPARSER)

set_if_at_least_one_set(RETDEC_ENABLE_PDBPARSER
RETDEC_ENABLE_CPDETECT
RETDEC_ENABLE_DEBUGFORMAT)

set_if_at_least_one_set(RETDEC_ENABLE_LOADER
RETDEC_ENABLE_ALL
RETDEC_ENABLE_BIN2LLVMIR
RETDEC_ENABLE_DEBUGFORMAT
RETDEC_ENABLE_FILEINFO
RETDEC_ENABLE_RTTI_FINDER
RETDEC_ENABLE_STACOFIN
Expand All @@ -339,6 +330,7 @@ set_if_at_least_one_set(RETDEC_ENABLE_FILEFORMAT
RETDEC_ENABLE_ALL
RETDEC_ENABLE_BIN2LLVMIR
RETDEC_ENABLE_CPDETECT
RETDEC_ENABLE_DEBUGFORMAT
RETDEC_ENABLE_FILEINFO
RETDEC_ENABLE_GETSIG
RETDEC_ENABLE_LOADER
Expand All @@ -362,7 +354,6 @@ set_if_at_least_one_set(RETDEC_ENABLE_CONFIG
RETDEC_ENABLE_ALL
RETDEC_ENABLE_BIN2LLVMIR
RETDEC_ENABLE_CONFIGTOOL
RETDEC_ENABLE_DEBUGFORMAT
RETDEC_ENABLE_FILEINFO
RETDEC_ENABLE_LLVMIR2HLL
RETDEC_ENABLE_STACOFIN)
Expand Down Expand Up @@ -391,6 +382,7 @@ set_if_at_least_one_set(RETDEC_ENABLE_UTILS
RETDEC_ENABLE_BIN2PAT
RETDEC_ENABLE_CAPSTONE2LLVMIRTOOL
RETDEC_ENABLE_CONFIG
RETDEC_ENABLE_COMMON
RETDEC_ENABLE_CRYPTO
RETDEC_ENABLE_CTYPES
RETDEC_ENABLE_CTYPESPARSER
Expand All @@ -402,7 +394,7 @@ set_if_at_least_one_set(RETDEC_ENABLE_UTILS
RETDEC_ENABLE_LOADER
RETDEC_ENABLE_MACHO_EXTRACTOR
RETDEC_ENABLE_MACHO_EXTRACTORTOOL
RETDEC_ENABLE_PRETDEC_ENABLE_CPDETECTAT2YARA
RETDEC_ENABLE_CPDETECT
RETDEC_ENABLE_PATTERNGEN
RETDEC_ENABLE_RTTI_FINDER
RETDEC_ENABLE_STACOFIN
Expand Down Expand Up @@ -518,7 +510,8 @@ set_if_at_least_one_set(RETDEC_ENABLE_RAPIDJSON

set_if_at_least_one_set(RETDEC_ENABLE_TINYXML2
RETDEC_ENABLE_TINYXML2
RETDEC_ENABLE_FILEINFO)
RETDEC_ENABLE_FILEINFO
RETDEC_ENABLE_CPDETECT)

set_if_at_least_one_set(RETDEC_ENABLE_VARIANT
RETDEC_ENABLE_UTILS)
Expand Down
16 changes: 15 additions & 1 deletion deps/CMakeLists.txt
@@ -1,6 +1,20 @@

include(ExternalProject)

include(ProcessorCount)
ProcessorCount(CPUS)

if(CMAKE_C_COMPILER)
set(CMAKE_C_COMPILER_OPTION "-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}")
endif()
if(CMAKE_CXX_COMPILER)
set(CMAKE_CXX_COMPILER_OPTION "-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}")
endif()

if(MSVC)
set(DEBUG_DIR "Debug/")
set(RELEASE_DIR "Release/")
set(DEBUG_DIR "Debug/")
set(DEBUG_LIB_SUFFIX "d")
endif()

cond_add_subdirectory(capstone RETDEC_ENABLE_CAPSTONE)
Expand Down
84 changes: 71 additions & 13 deletions deps/capstone/CMakeLists.txt
@@ -1,11 +1,3 @@
include(ExternalProject)

if(CMAKE_C_COMPILER)
set(CMAKE_C_COMPILER_OPTION "-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}")
endif()
if(CMAKE_CXX_COMPILER)
set(CMAKE_CXX_COMPILER_OPTION "-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}")
endif()

if(CAPSTONE_LOCAL_DIR)
message(STATUS "Capstone: using local Capstone directory.")
Expand Down Expand Up @@ -104,10 +96,76 @@ endif()
ExternalProject_Get_Property(capstone-project source_dir)
ExternalProject_Get_Property(capstone-project binary_dir)

# Add libraries.
set(CAPSTONE_LIB_NAME ${CMAKE_STATIC_LIBRARY_PREFIX}capstone${CMAKE_STATIC_LIBRARY_SUFFIX})
set(CAPSTONE_LIB_DEBUG ${binary_dir}/${DEBUG_DIR}${CAPSTONE_LIB_NAME})
set(CAPSTONE_LIB_OPT ${binary_dir}/${RELEASE_DIR}${CAPSTONE_LIB_NAME})

# Create target.
add_library(capstone INTERFACE)
add_library(retdec::deps::capstone ALIAS capstone)
add_dependencies(capstone capstone-project)
target_include_directories(capstone SYSTEM INTERFACE ${source_dir}/include)
target_include_directories(capstone SYSTEM INTERFACE ${source_dir}/arch)
target_link_libraries(capstone INTERFACE debug ${binary_dir}/${DEBUG_DIR}${CMAKE_STATIC_LIBRARY_PREFIX}capstone${CMAKE_STATIC_LIBRARY_SUFFIX})
target_link_libraries(capstone INTERFACE optimized ${binary_dir}/${RELEASE_DIR}${CMAKE_STATIC_LIBRARY_PREFIX}capstone${CMAKE_STATIC_LIBRARY_SUFFIX})

target_include_directories(capstone
SYSTEM INTERFACE
$<BUILD_INTERFACE:${source_dir}/include>
$<BUILD_INTERFACE:${source_dir}/arch>
$<INSTALL_INTERFACE:${RETDEC_INSTALL_DEPS_INCLUDE_DIR}>
)

target_link_libraries(capstone INTERFACE
$<INSTALL_INTERFACE:retdec::deps::capstone-libs>
debug
$<BUILD_INTERFACE:${CAPSTONE_LIB_DEBUG}>
optimized
$<BUILD_INTERFACE:${CAPSTONE_LIB_OPT}>
)

# Install includes.
install(
DIRECTORY
${source_dir}/include/
DESTINATION
${RETDEC_INSTALL_DEPS_INCLUDE_DIR}
)

# Install libs.
# Install both Release and Debug variant to the same location.
# We assume that only one variant will be present at the time.
install(
FILES
${CAPSTONE_LIB_DEBUG}
${CAPSTONE_LIB_OPT}
DESTINATION
${RETDEC_INSTALL_DEPS_LIB_DIR}
OPTIONAL
)

# Install targets.
install(TARGETS capstone
EXPORT capstone-targets
)

# Export targets.
install(EXPORT capstone-targets
FILE "retdec-capstone-targets.cmake"
NAMESPACE retdec::deps::
DESTINATION ${RETDEC_INSTALL_CMAKE_DIR}
)

# Configure config file.
set(CAPSTONE_LIB_INSTALLED "${RETDEC_INSTALL_DEPS_LIB_DIR_ABS}/${CAPSTONE_LIB_NAME}")
configure_package_config_file(
"retdec-capstone-config.cmake"
"${CMAKE_CURRENT_BINARY_DIR}/retdec-capstone-config.cmake"
INSTALL_DESTINATION ${RETDEC_INSTALL_CMAKE_DIR}
PATH_VARS
CAPSTONE_LIB_INSTALLED
)

# Install CMake files.
install(
FILES
"${CMAKE_CURRENT_BINARY_DIR}/retdec-capstone-config.cmake"
DESTINATION
"${RETDEC_INSTALL_CMAKE_DIR}"
)
12 changes: 12 additions & 0 deletions deps/capstone/retdec-capstone-config.cmake
@@ -0,0 +1,12 @@
@PACKAGE_INIT@

if(NOT TARGET retdec::deps::capstone-libs)
add_library(retdec::deps::capstone-libs STATIC IMPORTED)
set_target_properties(retdec::deps::capstone-libs PROPERTIES
IMPORTED_LOCATION @PACKAGE_CAPSTONE_LIB_INSTALLED@
)
endif()

if(NOT TARGET retdec::deps::capstone)
include(${CMAKE_CURRENT_LIST_DIR}/retdec-capstone-targets.cmake)
endif()
35 changes: 34 additions & 1 deletion deps/elfio/CMakeLists.txt
@@ -1,3 +1,36 @@

add_library(elfio INTERFACE)
target_include_directories(elfio SYSTEM INTERFACE ${CMAKE_CURRENT_LIST_DIR}/include)
add_library(retdec::deps::elfio ALIAS elfio)

target_include_directories(elfio
SYSTEM INTERFACE
$<BUILD_INTERFACE:${RETDEC_DEPS_DIR}/elfio/include>
$<INSTALL_INTERFACE:${RETDEC_INSTALL_DEPS_INCLUDE_DIR}>
)

# Install includes.
install(
DIRECTORY ${RETDEC_DEPS_DIR}/elfio/include/
DESTINATION ${RETDEC_INSTALL_DEPS_INCLUDE_DIR}
)

# Install targets.
install(TARGETS elfio
EXPORT elfio-targets
# Header only -> there are no actual libs to install to destination.
)

# Export targets.
install(EXPORT elfio-targets
FILE "retdec-elfio-targets.cmake"
NAMESPACE retdec::deps::
DESTINATION ${RETDEC_INSTALL_CMAKE_DIR}
)

# Install CMake files.
install(
FILES
"${CMAKE_CURRENT_LIST_DIR}/retdec-elfio-config.cmake"
DESTINATION
"${RETDEC_INSTALL_CMAKE_DIR}"
)
4 changes: 4 additions & 0 deletions deps/elfio/retdec-elfio-config.cmake
@@ -0,0 +1,4 @@

if(NOT TARGET retdec::deps::elfio)
include(${CMAKE_CURRENT_LIST_DIR}/retdec-elfio-targets.cmake)
endif()

0 comments on commit 8fea90b

Please sign in to comment.