Skip to content

Commit

Permalink
Build enhancement (homenc#43)
Browse files Browse the repository at this point in the history 
* Enhanced cmake building script

* Added missing override qualifiers

* Added GMP and NTL version checks

* Fixed path mistake

* Added pedantic build flags -Wall -Wextra -Werror

* Moved minimum cmake version to 3.5
Moved to explicit listing of sources

* Added googletest to build process and converted extractDigits to gtest

* Fixed issue with NTL include directory

* Converted Test_General to google test

* Added test guards in CMakeLists.txt
Added extra checks for NTL thread configuration

* Plumbed in noPrint and dry, plus misc code tidyups

* google test for binaryArith, and slight tidyup of General

* Tidyup of binaryArith and ported binaryCompare

* Moved google tests to a separate folder.
Moved the cmake gtest logic to the test folder

* Ported Test_Bin_IO

* Tidyup, mostly standardising Parameters-printers

* Converted bootstrapping test

* Fixed make test and Bin_IO

* Ported Test_EvalMap to google test

* Ported Test_intraSlot to google test

* Added missing << operator to EvalMap

* Ported matmul test to gtest framework

* Fixed some incorrect parameter types in bootstrapping test

* Ported PAlgebra test

* Ported permutations test to gtest

* Corrected bracketing to remove clang warning

* Ported PolyEval to google test

* Ported Replicate test to google test

* Ported tableLookup test to google test

* Ported ThinBootstrapping test to google test

* Ported GTest_ThinEvalMap

* Ported final two tests

* Slight indentation fix

* Fixed mixing headers in install

* init partially executing Dockerfile

* add docker ignore file

* Toward the DEFINITIVE superbuild cmake script

* Building packaged

* Succesfully generating export targets for the relocatable package

* Fixed case typo

* Relocatable package build working

* Fixed some Linux issues and added FIXMEs

* Fixed rpath absolute/relative rpath patch process

* Made Linux rpath work and added dependency notes

* Removed chrpath support (never worked)

* Working options 1 2 3. No tests

* Test added and working

* Added compiler selection in deps

* Improved comilation when not a package build

* Fixed assert side-effects in binary IO

* Added fixme for broken legacy test path

* Polishing

* Polished

* Still polishing

* Fixed rpath in-out

* Fixed minor issues/comments in build

* Changed version of gmp to 6.1.0

* Decreased version of GMP in CMakeLists for consistency

* Added global variables for GMP and NTL versions

* Added installation instructions and tidied up some messages

* Added workaround for NTL's bad version detection

* Added fixme for treatment of threads

* Minor refactoring ad better pthreads control

* Minor install guide tweaks

* Removed redundant FIXME

* Fixed missing FFT_NATIVE when Armadillo not found

* Brought some merge changes through to new tests

* Ported test changes

* Removed GTest_EaCx from the CMakeLists due to bug with no armadillo

* Disabled approxNums test and removed deprecated INSTANTIATE_TEST_CASE_P

* Guarded agaist test exits in EaCx and approxNums

* Removed some merge fixmes

* Handle Armadillo in a more consistent way

* Changed requirement autoconf to m4. Reformatted instructions. Added
example

* Added instructions for cmake-based build and tests

* Added example program

* Fixed legacy test executable naming

* Fixed comments in tests

* nasty binio bug fix for IndexSet

* Updated gtest parameters as old make check ones

* Changed example name

* Fixed GTest_PolyEval parameters generation
  • Loading branch information
@olimasters
olimasters authored and E. Steffinlongo committed Mar 1, 2019
1 parent 7091789 commit 29ee12d
Show file tree
Hide file tree
Showing 53 changed files with 7,984 additions and 78 deletions.
2 changes: 2 additions & 0 deletions .dockerignore
View file
@@ -0,0 +1,2 @@
# A local cmake cache will affect the docker image build
CMakeCache.txt
286 changes: 242 additions & 44 deletions CMakeLists.txt
View file
@@ -1,45 +1,243 @@
cmake_minimum_required(VERSION 3.2)
project(fhe)
set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin)

option(ENABLE_TEST "Build the test files" OFF)
option(ENABLE_THREADS "To use multi-thread. Make Sure the NTL is built with NTL_THREADS=on" OFF)
option(BUILD_SHARED "To build shared objects" OFF)
option(BUILD_AES "To build homomorphic AES" OFF)

## Might modify the following path to NTL according to your environment
set(NTL_HEADER /usr/local/include)
set(NTL_LIB /usr/local/lib)
include_directories(${NTL_HEADER})
link_directories(${NTL_LIB})
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fPIC")
## Use -std=c++11 as dedeault.
cmake_minimum_required(VERSION 3.5 FATAL_ERROR)

project(helib_superbuild
LANGUAGES C CXX)

# GMP version to be used (and eventually downloaded)
set(GMP_REQUIRED_VERSION "6.1.2")
# NTL version to be used (and eventually downloaded)
set(NTL_REQUIRED_VERSION "11.3.0")

# Define standard installation directories (GNU)
include(GNUInstallDirs)

## Use -std=c++11 as default.
set(CMAKE_CXX_STANDARD 11)
## On old platform, might change to -std=c++0x
#set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++0x")

## To use multi-threads.
## Make sure the NTL is built with NTL_THREADS=on
## For NTL's verison older than 7.0, should turn FHE_THREADS off.
if (ENABLE_THREADS)
set(FHE_THREADS ON)
add_definitions(-DFHE_THREADS)
set(RUN_LIB fhe ntl gmp pthread)
else (ENABLE_THREADS)
set(FHE_THREADS OFF)
set(RUN_LIB fhe ntl gmp)
endif (ENABLE_THREADS)

# Use Armadillo if you can find it
find_package(Armadillo)
if (ARMADILLO_FOUND)
include_directories(${ARMADILLO_INCLUDE_DIRS})
link_libraries(${ARMADILLO_LIBRARIES})
add_definitions(-DFFT_ARMA)
else (ARMADILLO_FOUND)
add_definitions(-DFFT_NATIVE)
endif (ARMADILLO_FOUND)

add_subdirectory(src)
## Disable C++ extensions
set(CMAKE_CXX_EXTENSIONS OFF)
## Require full C++ standard
set(CMAKE_CXX_STANDARD_REQUIRED ON)

# Adding possible gui values to CMAKE_BUILD_TYPE variable
set_property(CACHE CMAKE_BUILD_TYPE PROPERTY STRINGS
"Debug" "RelWithDebInfo" "Release" "MinSizeRel")

# Setting up Debug as default CMAKE_BUILD_TYPE
# TODO: set to Release when assertions are removed
if (NOT CMAKE_BUILD_TYPE)
# Setting Debug as default since many checks are performed as assert (possibly with side effects)
set(CMAKE_BUILD_TYPE Debug CACHE STRING "Choose the type of build, options are: Debug RelWithDebInfo Release MinSizeRel" FORCE)
endif (NOT CMAKE_BUILD_TYPE)

# Path containing FindGMP.cmake and FindNTL.cmake
list(APPEND CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/cmake")

option(BUILD_SHARED "Build as shared library" ON)

option(PACKAGE_BUILD "Download dependencies and build as a self-contained package" OFF)

option(ENABLE_THREADS "Enable threads support. Requires NTL built with NTL_THREADS=on" OFF)

# TODO: is this required for CKKS?
option(ENABLE_ARMADILLO "Use Armadillo for faster FFT. Requires the Armadillo library on the system." OFF)

option(ENABLE_TEST "Enable tests" OFF)
option(PEDANTIC_BUILD "Use -Wall -Wpedantic -Wextra -Werror during build" OFF)

# Add proerties dependent to PACKAGE_BUILD
if (PACKAGE_BUILD)
# Properties to be enabled when using PACKAGE_BUILD
set(PACKAGE_DIR "" CACHE STRING "Folder with the compilation output directory (DEFAULT: helib_pack)")
else (PACKAGE_BUILD)
# Properties to be enabled when not using PACKAGE_BUILD
option(ENABLE_LEGACY_TEST "Build the legacy test files (does not work with PACKAGE_BUILD)" OFF)
option(BUILD_AES "Build homomorphic AES (does not work with PACKAGE_BUILD)" OFF)

set(GMP_DIR "" CACHE STRING "Prefix of the GMP library (ignored when PACKAGE_BUILD is ON)")
set(NTL_DIR "" CACHE STRING "Prefix of the NTL library (ignored when PACKAGE_BUILD is ON)")
endif(PACKAGE_BUILD)

# Setting flag lists to avoid polluting CMAKE_CXX_FLAGS
list(APPEND PRIVATE_HELIB_CXX_FLAGS "-fPIC")
# Add extra checks during build
if (PEDANTIC_BUILD)
list(APPEND PRIVATE_HELIB_CXX_FLAGS "-Wall" "-Wpedantic" "-Wextra" "-Werror")
endif(PEDANTIC_BUILD)

if (ENABLE_TEST)
enable_testing()
endif(ENABLE_TEST)

# Look for pthread using default FindThreads.cmake script
find_package(Threads)
if (ENABLE_THREADS AND Threads_NOTFOUND)
message(FATAL_ERROR "Cannot find pthreads (ENABLE_THREADS is ON).")
endif()

if (ENABLE_ARMADILLO)
find_package(Armadillo REQUIRED)
endif (ENABLE_ARMADILLO)

# NOTE: Consider reconfiguring everything when PACKAGE_BUILD changes value. Options from the previous value will remain otherwise.
# Set up extra properties depending on the value of PACKAGE_BUILD
if (PACKAGE_BUILD)
# Setting up default compilation output directory
if (NOT PACKAGE_DIR)
set(PACKAGE_DIR "helib_pack")
endif(NOT PACKAGE_DIR)
if (NOT IS_ABSOLUTE ${PACKAGE_DIR})
# Make CMAKE_PACKAGE_DIR absolute
set(PACKAGE_DIR "${CMAKE_BINARY_DIR}/${PACKAGE_DIR}" CACHE STRING "Folder with the compilation output directory (DEFAULT: helib_pack)" FORCE)
endif()

# Setting up download/build path of external dependencies
set(DEPENDENCIES_FOLDER "${CMAKE_BINARY_DIR}/dependencies")
set_property(DIRECTORY PROPERTY EP_BASE "${DEPENDENCIES_FOLDER}")

#raising warning when PACKAGE_BUILD is ON
# warn if installing globally
# NOTE: this is a somewhat fragile check that can be enhanced
string(FIND "${CMAKE_INSTALL_PREFIX}" "/usr" install_in_usr)
if ("${install_in_usr}" EQUAL 0)
message(WARNING "CAUTION: Package build should not be installed globally as it will potentially override dependencies.")
endif()
unset(install_in_usr)

# Warn existing dependencies are ignored and rebuilt
if (GMP_DIR)
message(WARNING "GMP_DIR is ignored when PACKAGE_BUILD is ON.")
endif(GMP_DIR)

if (NTL_DIR)
message(WARNING "NTL_DIR is ignored when PACKAGE_BUILD is ON.")
endif(NTL_DIR)

# Add an imported target to propagate the library locations
add_library(gmp_external SHARED IMPORTED)
add_library(ntl_external SHARED IMPORTED)

# RPATH will be empty since ntl, gmp, and helib are all in PACKAGE_DIR/lib
set(PACKAGE_RPATH "")

# Add the external dependencies (done in dependencies/CMakeLists.txt)
add_subdirectory(dependencies)

# get gmp and ntl include/link directories
get_target_property(GMP_INCLUDE_PATHS gmp_external INTERFACE_INCLUDE_DIRECTORIES)
get_target_property(GMP_LIBRARIES gmp_external IMPORTED_LOCATION)
get_target_property(NTL_INCLUDE_PATHS ntl_external INTERFACE_INCLUDE_DIRECTORIES)
get_target_property(NTL_LIBRARIES ntl_external IMPORTED_LOCATION)

# Track if helib requires pthreads as dependency
set(HELIB_REQUIRES_PTHREADS ${ENABLE_THREADS})

else (PACKAGE_BUILD)
# find GMP library
# Try to find the GMP package (using cmake/FindGMP.cmake script)
# REQUIRED arg make cmake to fail if GMP is not found
find_package(GMP "${GMP_REQUIRED_VERSION}" EXACT REQUIRED)

# find NTL library
# Try to find the NTL package (using cmake/FindNTL.cmake script)
# REQUIRED arg make cmake to fail if NTL is not found
find_package(NTL "${NTL_REQUIRED_VERSION}" EXACT REQUIRED)

# Setting compiler output directories
set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/${CMAKE_INSTALL_LIBDIR})
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/${CMAKE_INSTALL_LIBDIR})
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/${CMAKE_INSTALL_BINDIR})
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/${CMAKE_INSTALL_BINDIR})

## Thread enabling checks
## Checking if NTL is built with NTL_THREADS=on
set(ntl_config_file "${NTL_INCLUDE_PATHS}/NTL/config.h")
if (EXISTS "${ntl_config_file}")
include(CheckCXXSymbolExists)
check_cxx_symbol_exists("NTL_THREADS" "${ntl_config_file}" ntl_has_threads)
else()
message(FATAL_ERROR "Cannot locate NTL configuration file (${ntl_config_file}).")
endif()

# Raising errors when threads are misconfigured
if (ENABLE_THREADS AND NOT ntl_has_threads)
message(FATAL_ERROR "Cannot enable threads since NTL was built without. Consider re-building NTL with NTL_THREADS=on.")
endif(ENABLE_THREADS AND NOT ntl_has_threads)

# This should not really happen
if (ntl_has_threads AND Threads_NOTFOUND)
message(FATAL_ERROR "NTL requires pthreads that has not been found.")
endif(ntl_has_threads AND Threads_NOTFOUND)

# Track if helib requires pthreads as dependency
set(HELIB_REQUIRES_PTHREADS ${ntl_has_threads})

unset(ntl_config_file)
unset(ntl_has_threads)
endif(PACKAGE_BUILD)


# Building HELIB here
if (PACKAGE_BUILD)
# Adding HELIB as an external project
include(ExternalProject)

# Before building helib_external wait compilation of gmp and ntl.
list(APPEND helib_external_deps "gmp_external"
"ntl_external")

ExternalProject_Add(helib_external
DEPENDS
# await compilation of gmp and ntl
gmp_external
ntl_external
SOURCE_DIR
${CMAKE_SOURCE_DIR}/src
CMAKE_ARGS
-DCMAKE_INSTALL_PREFIX=${PACKAGE_DIR}
-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
-DCMAKE_CXX_FLAGS=${CMAKE_CXX_FLAGS}
-DCMAKE_CXX_STANDARD=${CMAKE_CXX_STANDARD}
-DCMAKE_CXX_STANDARD_REQUIRED=${CMAKE_CXX_STANDARD_REQUIRED}
-DCMAKE_CXX_EXTENSIONS=${CMAKE_CXX_EXTENSIONS}
-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}
-DPRIVATE_HELIB_CXX_FLAGS=${PRIVATE_HELIB_CXX_FLAGS}
-DPUBLIC_HELIB_CXX_FLAGS=${PUBLIC_HELIB_CXX_FLAGS}
-DGMP_LIBRARIES=${GMP_LIBRARIES}
-DNTL_INCLUDE_PATHS=${NTL_INCLUDE_PATHS}
-DNTL_LIBRARIES=${NTL_LIBRARIES}
-DENABLE_ARMADILLO=${ENABLE_ARMADILLO}
-DENABLE_THREADS=${ENABLE_THREADS}
-DHELIB_REQUIRES_PTHREADS=${HELIB_REQUIRES_PTHREADS}
-DBUILD_SHARED=${BUILD_SHARED}
-DPACKAGE_BUILD=${PACKAGE_BUILD}
-DENABLE_TEST=${ENABLE_TEST}
-DENABLE_LEGACY_TEST=OFF
-DBUILD_AES=OFF
BUILD_ALWAYS ON
)

if (ENABLE_TEST)
add_test(
NAME
helib_check
COMMAND
${CMAKE_MAKE_PROGRAM} test
WORKING_DIRECTORY
"${DEPENDENCIES_FOLDER}/Build/helib_external"
)
endif(ENABLE_TEST)

# To install copy the whole PACKAGE_DIR directory to the defined prefix
install(
DIRECTORY
${PACKAGE_DIR}
DESTINATION
# this is interpreted relative to the value of the CMAKE_INSTALL_PREFIX variable
.
USE_SOURCE_PERMISSIONS
)
else (PACKAGE_BUILD)
# If not compiling as PACKAGE_BUILD then add helib as subfolder and not as an external project
add_subdirectory(src)
endif(PACKAGE_BUILD)

23 changes: 23 additions & 0 deletions Dockerfile
View file
@@ -0,0 +1,23 @@
FROM alpine:3.8

# Location of the src in the contrainer
ARG helibDir=/helib
# Location of the build in the contrainer
ARG buildDir=/build

COPY . ${helibDir}

# Currently fails when cmake is invoked
RUN apk add --update \
g++ \
make \
cmake \
libtool \
&& rm -rf /var/cache/apk/* \
&& mkdir ${buildDir} \
&& cd ${buildDir} \
&& cmake ${helibDir} \
&& make

CMD [ "" ]

0 comments on commit 29ee12d

Please sign in to comment.