`soci` not found when adding as subproject #762

msis · 2019-10-20T21:38:08Z

I can't seem to be able to add soci as a subproject.

I have tried with ExternalProject_Add, and FetchContent but I can't figure out why I keep getting:

CMake Error at src/test_app/CMakeLists.txt:21 (add_dependencies):
  The dependency target "soci" of target "test_app" does not exist.

Project description

My project structure is the following:

└── src
    ├── CMakeLists.txt
    ├── cmake
    │   ├── soci.cmake
    │   └── spdlog.cmake
    ├── test_app
    │   └── CMakeLists.txt
    └── (...)

`src/CMakeLists.txt`

cmake_minimum_required(VERSION 3.11)

set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)

include(cmake/spdlog.cmake)
include(cmake/soci.cmake)

add_subdirectory(test_app)

`src/test_app/CMakeLists.txt`

cmake_minimum_required(VERSION 3.11)

project(test_app LANGUAGES CXX)

add_executable(test_app main.cpp)

target_link_libraries(test_app PRIVATE
    spdlog
    SOCI::core
)

add_dependencies(test_app spdlog soci)

`spdlog.cmake`

cmake_minimum_required(VERSION 3.11)

message(STATUS "Extern: spdlog v1.4.2")

set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)

include(FetchContent)

FetchContent_Declare( spdlog
    GIT_REPOSITORY https://github.com/gabime/spdlog
    GIT_TAG v1.4.2
    GIT_SHALLOW ON
)
FetchContent_GetProperties(spdlog)
if(NOT spdlog_POPULATED)
    set(SPDLOG_BUILD_EXAMPLE OFF)
    set(SPDLOG_BUILD_EXAMPLE_HO OFF)
    set(SPDLOG_BUILD_TESTS OFF)
    set(SPDLOG_BUILD_TESTS_HO OFF)
    set(SPDLOG_BUILD_BENCH OFF)
    set(SPDLOG_SANITIZE_ADDRESS OFF)
    set(SPDLOG_INSTALL ON)
    set(SPDLOG_FMT_EXTERNAL OFF)
    FetchContent_Populate(spdlog)
    add_subdirectory(
        ${spdlog_SOURCE_DIR}
        ${spdlog_BINARY_DIR}
        EXCLUDE_FROM_ALL)
endif()

`soci.cmake`

cmake_minimum_required(VERSION 3.11)

message(STATUS "Extern: SOCI pre4.0+git_blksail")

set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)

include(FetchContent)

FetchContent_Declare( soci
    GIT_REPOSITORY https://github.com/SOCI/soci
    GIT_TAG master
    GIT_SHALLOW ON
)
FetchContent_GetProperties(soci)
if(NOT soci_POPULATED)
    set(SOCI_STATIC ON)
    set(SOCI_SHARED ON)
    set(SOCI_TESTS OFF)
    # set(SOCI_ASAN OFF)
    set(SOCI_CXX11 ON)
    set(SOCI_LIBDIR lib)
    set(WITH_SQLITE3 ON)
    set(WITH_POSTGRESQL ON)
    set(WITH_BOOST OFF)
    set(WITH_DB2 OFF)
    set(WITH_ODBC OFF)
    set(WITH_ORACLE OFF)
    set(WITH_MYSQL OFF)
    set(SOCI_EMPTY OFF)
    FetchContent_Populate(soci)
    add_subdirectory(
        ${soci_SOURCE_DIR}
        ${soci_BINARY_DIR}
        EXCLUDE_FROM_ALL)
endif()

The text was updated successfully, but these errors were encountered:

mloskot · 2019-10-20T21:50:35Z

I'm sorry but I can't help you. I almost never use dependencies/projects via ExternalProject_Add

msis · 2019-10-20T22:42:36Z

Thanks @mloskot .
I believe it has to do with a missing export in SOCI's CMakeLists.txt.
I will keep digging.

tt4g · 2019-10-21T00:57:52Z

@msis For your information.

Although it is a little complicated, it is important to understand that CMake operations are roughly divided into two steps: configuration and build.

FetchContent_Declare() and add_dependencies() are executed in configure step.

At this point, soci is not built by CMake (soci is a library that needs to be built), because the configure step only downloads the source.
If it is not built and installed, find_package(soci) cannot find the soci CMake target.

NOTE: That's because FetchContent_Declare() and ExternalProject_Add() are supposed to get the source code necessary for your project from outside. It is possible to obtain the source of the third party library, but the build work is performed in the CMake build step: FetchContent — CMake 3.16.0-rc2 Documentation

To build SOCI with CMake alone and use it in your project, you need to build the SOCI with CMake configure step and define its CMake target.
This requires complex CMake operations, but there are googletest samples.

In the googletest sample below, execute_process() is executed by CMake configure step, and the cmake command is called to execute googletest build.
googletest built with add_subdirectory () can be referenced in the project.

https://github.com/google/googletest/blob/master/googletest/README.md

Since spdlog is a header-only-library, it seems to be working.

msis · 2019-10-21T01:06:22Z

Thanks @tt4g. I will try what you suggested right away.

However, I added a static library that worked too.
I see it build, and I suppose it's because of the add_dependencies().
And it doesn't complain that the dependency target does not exist.

PS: So far it seems that all other libraries build a package with include(CMakePackageConfigHelpers) and soci doesn't.

msis · 2019-10-21T01:29:22Z

@tt4g , I followed the steps for googletest.
I see it compile, but when it is time to build test_app, I do still have the same error:

CMake Error at src/test_app/CMakeLists.txt:19 (add_dependencies):
  The dependency target "soci" of target "test_app" does not exist.

tt4g · 2019-10-21T01:58:02Z

soci's CMake target name is soci_core.
The backend library must also specify each CMake target (e.g. soci_${backend}).

Try replacing the dependency name soci with soci_core.

msis · 2019-10-21T02:29:49Z

We're making progress!
It now compiles soci with the new dependency name.

But it seems like there's another issue with the include directories:
The following will not compile:

#include "soci.h"

Error:

src/test_app/main.cpp:7:10: fatal error: soci.h: No such file or directory
 #include "soci.h"
          ^~~~~~~~
compilation terminated.

And it seems like the header files are further down, this will error differently:

#include "soci/soci.h"

Error:

build/_deps/soci-src/src/core/../../include/soci/soci-platform.h:24:10: fatal error: soci/soci-config.h: No such file or directory
 #include "soci/soci-config.h" // for SOCI_HAVE_CXX11
          ^~~~~~~~~~~~~~~~~~~~
compilation terminated.

tt4g · 2019-10-21T02:43:00Z

Try adding the soci include directory to the project include directory: include_directories — CMake 3.16.0-rc2 Documentation

example src/CMakeLists.txt:

cmake_minimum_required(VERSION 3.11)

set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)

include(cmake/spdlog.cmake)
include(cmake/soci.cmake)

+ include_directories("${PATH_TO_SOCI_INCLUDE_DIR}")

add_subdirectory(test_app)

msis · 2019-10-21T02:52:32Z

+ include_directories("${PATH_TO_SOCI_INCLUDE_DIR}")

But isn't the purpose of adding the dependency to avoid that?

PS: I'm trying to avoid such an include as it is not recommended anymore...

tt4g · 2019-10-21T03:23:37Z

I suggested include_directories() as the simplest solution, with priority on problem solving.
Because I thought it would be difficult to provide a better solution for complex CMake projects.

Of course you can use target_include_directories() to minimize the impact (e.g. target_include_directories(test_app PUBLIC "${PATH_TO_SOCI_INCLUDE_DIR}")).

target_include_directories — CMake 3.16.0-rc2 Documentation

msis · 2019-10-21T03:52:51Z

I understand, but it didn't solve the issue neither.
Here's what I out in the test_app/CMakeLists.txt:

set(PATH_TO_SOCI_INCLUDE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/../../build/_deps/soci-src/include)
message(STATUS ${PATH_TO_SOCI_INCLUDE_DIR})
target_include_directories(test_app
    PRIVATE
        ${PATH_TO_SOCI_INCLUDE_DIR}
)

And I still get the same error...
So I suppose it was exposed right the first time..

tt4g · 2019-10-21T04:10:57Z

Is the following error message? #762 (comment):

build/_deps/soci-src/src/core/../../include/soci/soci-platform.h:24:10: fatal error: soci/soci-config.h: No such file or directory
 #include "soci/soci-config.h" // for SOCI_HAVE_CXX11
          ^~~~~~~~~~~~~~~~~~~~
compilation terminated.

If so, the file generated by the soci project's CMAKELists.txt may not exist.

soci/CMakeLists.txt

Lines 152 to 155 in dbf9397

    
           set(CONFIG_INCLUDE_DIR ${CMAKE_CURRENT_BINARY_DIR}/include) 
        
           install(DIRECTORY ${CONFIG_INCLUDE_DIR}/soci DESTINATION ${INCLUDEDIR}) 
        
           set(CONFIG_FILE_IN "include/soci/soci-config.h.in") 
        
           set(CONFIG_FILE_OUT "${CONFIG_INCLUDE_DIR}/soci/soci-config.h")

Does soci-config.h exist in ${PROJECT}/build/_deps/soci-src/include/soci? .

tt4g · 2019-10-21T07:04:51Z

@msis I created a small project to build soci.
Please refer to this project: https://github.com/tt4g/soci_issue_762/tree/master

The problem of missing soci/soci-config.h is solved by CMake in-source build: tt4g/soci_issue_762@68faa17

msis · 2019-10-21T12:01:13Z

Does soci-config.h exist in ${PROJECT}/build/_deps/soci-src/include/soci? .

So that file isn't there. It's in ${PROJECT}/build/_deps/soci-build/include/soci

Is it possible the include files are not copied correctly?
Because that is the only header file in that directory.

msis · 2019-10-21T12:58:12Z

@tt4g, I resolved the issue by changing the following in the soci/CMakeLists.txt:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 004dd07..2125bd1 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -149,7 +149,7 @@ set(INCLUDEDIR "include" CACHE PATH "The directory to install includes into.")
 ###############################################################################
 # Configuration files
 ###############################################################################
-set(CONFIG_INCLUDE_DIR ${CMAKE_CURRENT_BINARY_DIR}/include)
+set(CONFIG_INCLUDE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/include)
 install(DIRECTORY ${CONFIG_INCLUDE_DIR}/soci DESTINATION ${INCLUDEDIR})
 set(CONFIG_FILE_IN "include/soci/soci-config.h.in")
 set(CONFIG_FILE_OUT "${CONFIG_INCLUDE_DIR}/soci/soci-config.h")

I have this as a patch during the FetchContent_Declare() in soci.cmake:

cmake_minimum_required(VERSION 3.11)

message(STATUS "Extern: SOCI pre4.0+git_blksail")

set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)

include(FetchContent)

FetchContent_Declare( soci
    GIT_REPOSITORY https://github.com/SOCI/soci
    GIT_TAG master
    GIT_SHALLOW ON
    PATCH_COMMAND git apply "${CMAKE_SOURCE_DIR}/src/patches/soci.patch"
)
FetchContent_GetProperties(soci)
if(NOT soci_POPULATED)
    set(SOCI_STATIC ON)
    set(SOCI_SHARED ON)
    set(SOCI_TESTS OFF)
    # set(SOCI_ASAN OFF)
    set(SOCI_CXX11 ON)
    set(SOCI_LIBDIR lib)
    set(WITH_SQLITE3 ON)
    set(WITH_POSTGRESQL ON)
    set(WITH_BOOST OFF)
    set(WITH_DB2 OFF)
    set(WITH_ODBC OFF)
    set(WITH_ORACLE OFF)
    set(WITH_MYSQL OFF)
    set(SOCI_EMPTY OFF)
    FetchContent_Populate(soci)
    add_subdirectory(
        ${soci_SOURCE_DIR}
        ${soci_BINARY_DIR}
        EXCLUDE_FROM_ALL)
endif()

msis · 2019-10-21T13:33:12Z

While this solution is a hack, it let's users import soci as dependency.

I will leave it to the maintainers to close it if they see fit.

PS: before v4 , it would be great to give CMake some love.
Some great resources:

tt4g · 2019-10-21T13:33:49Z

So that file isn't there. It's in ${PROJECT}/build/_deps/soci-build/include/soci

Is it possible the include files are not copied correctly?
Because that is the only header file in that directory.

@msis Probably the difference between CMake in-souce builds and out-of-source builds.
If the output destination (binary directory) when building is in-souce build, it will be the source directory. On the other hand, if it is out-of-source build, it becomes a dedicated directory.

Since SOCI outputs a header file at build time, I think that soci/soci-config.h could not be found if the source directory and another directory were specified as the binary directory.

Your project can now be built because you changed CONFIG_INCLUDE_DIR to the source directory.

PS: It's very difficult to build external third party projects together in your own project like this one.
If possible, install the package on your system, or consider introducing a C/C ++ package manager.

msis · 2019-10-21T13:40:30Z

PS: It's very difficult to build external third party projects together in your own project like this one.
If possible, install the package on your system, or consider introducing a C/C ++ package manager.

@tt4g Which one do you recommend?
I looked at conan and vcpkg but CMake looked easier still...

tt4g · 2019-10-21T13:49:16Z

PS: It's very difficult to build external third party projects together in your own project like this one.
If possible, install the package on your system, or consider introducing a C/C ++ package manager.

@tt4g Which one do you recommend?
I looked at conan and vcpkg but CMake looked easier still...

@msis Sorry for letting you expect.
I don't know any C/C++ package manager that fully supports soci (I found some published conan recipes but it wasn't satisfactory).
As far as I know, the most stable way to use soci in a CMake project is to install it with CMake.

mloskot · 2019-10-21T13:58:37Z

@msis

PS: before v4 , it would be great to give CMake some love.

It would, but it is not going to happen, unless someone does the job (not much time left!).
I am not able to pursue this task.

If anyone is interested in the task, I have started CMake modernisation here https://github.com/SOCI/soci/tree/wip/ml/modern-cmake

Some great resources:

I have been well aware of all those for quite some time, including this https://crascit.com/professional-cmake/ which is single best one.

msis · 2019-10-21T14:39:44Z

Thanks @mloskot .

What's the minimum CMake version you're ready to have?

mloskot · 2019-10-21T14:43:25Z

Personally, I'm always on the latest release. For SOCI, not sure what would be suitable, certainly not older than 3.5 or 3.8. @vadz may want to comment.

I'm not sure if it is feasible to get the CMake rewrite for SOCI 4.0.0 which I'm ready to start releasing any time soon (in fact, release candidate 1 could have happened even now).

craigscott-crascit · 2019-10-22T08:29:26Z

In case it's useful, here's a CMake function we use at work to bring in soci into our build (note the restrictions and assumptions as mentioned in the comments, etc.):

# Download soci using dependency details from "soci" and adds
# it to the build via add_subdirectory(). It will be assumed
# that Boost should be enabled unless the NO_BOOST option
# is given.
#
# Arguments to the function should be the required backends
# to enable in the soci build (all lowercase). Eg:
#
#  myCustomDependencyAddSoci(oracle postgresql)
#
function(myCustomDependencyAddSoci)
    if(NOT UNIX)
        message(FATAL_ERROR "Soci support only implemented for Unix")
    endif()

    include(FetchContent)
    FetchContent_GetProperties(soci)
    if(soci_POPULATED)
        return()
    endif()
    FetchContent_Populate(soci)

    # Set up for our customizations before bringing soci into the build
    set(options   "NO_BOOST")
    set(singleVal "")
    set(multiVal  "")
    cmake_parse_arguments(ARGS "${options}" "${singleVal}" "${multiVal}" ${ARGN})

    if(ARGS_NO_BOOST)
        set(WITH_BOOST OFF CACHE INTERNAL "")
    else()
        find_package(Boost QUIET)
        if(Boost_FOUND)
            set(WITH_BOOST ON  CACHE INTERNAL "")
        else()
            set(WITH_BOOST OFF CACHE INTERNAL "")
        endif()
    endif()

    set(requestedBackends ${ARGS_UNPARSED_ARGUMENTS})
    set(supportedBackends
        odbc
        oracle
        postgresql
        sqlite3
        db2
        firebird
        mysql
    )

    # Assumes you want C++11, no soci tests and build shared libs
    set(SOCI_CXX_C11 ON  CACHE INTERNAL "")
    set(SOCI_TESTS   OFF CACHE INTERNAL "")
    set(SOCI_STATIC  OFF CACHE INTERNAL "")

    foreach(backend IN LISTS supportedBackends)
        string(TOUPPER "${backend}" beUpper)
        string(TOLOWER "${backend}" beLower)
        if("${beLower}" IN_LIST requestedBackends)
            set(WITH_${beUpper} ON  CACHE INTERNAL "")
        else()
            set(WITH_${beUpper} OFF CACHE INTERNAL "")
        endif()
    endforeach()

    # Take over where soci will install things and under what component name
    include(GNUInstallDirs)
    set(BINDIR ${CMAKE_INSTALL_BINDIR} CACHE INTERNAL "")
    set(LIBDIR ${CMAKE_INSTALL_LIBDIR} CACHE INTERNAL "")
    set(CMAKE_INSTALL_DEFAULT_COMPONENT_NAME soci)

    # Because of manually defined SOCI_VERSION
    set(CMAKE_POLICY_DEFAULT_CMP0048 NEW)

    add_subdirectory(${soci_SOURCE_DIR} ${soci_BINARY_DIR})

    # Ensure we have namespaced targets. These are what projects
    # should reference directly, not the unnamespaced soci_* targets.
    foreach(lib IN LISTS requestedBackends ITEMS core empty)
        if(NOT TARGET SOCI::soci_${lib})
            add_library(SOCI::soci_${lib} ALIAS soci_${lib})
        endif()
    endforeach()

    # Fix missing information in some targets (haven't tested all backends)
    set_property(TARGET soci_core APPEND PROPERTY
        INTERFACE_INCLUDE_DIRECTORIES
            $<BUILD_INTERFACE:${soci_BINARY_DIR}/include>
    )
    if(WITH_BOOST)
        set_property(TARGET soci_core APPEND PROPERTY
            INTERFACE_INCLUDE_DIRECTORIES
                $<TARGET_PROPERTY:Boost::boost,INTERFACE_INCLUDE_DIRECTORIES>
        )
        set_property(TARGET soci_core APPEND PROPERTY
            INTERFACE_COMPILE_DEFINITIONS SOCI_USE_BOOST
        )
    endif()
    if("oracle" IN_LIST requestedBackends)
        set_property(TARGET soci_oracle APPEND PROPERTY
            INTERFACE_INCLUDE_DIRECTORIES $<BUILD_INTERFACE:${ORACLE_HOME}/sdk/include>
        )
    endif()

    # Work around build warning that gets turned into an error when building
    # for release with more recent compilers. The warning seems questionable
    # for the specific case that gets triggered in vector-into-type.cpp.
    if(WITH_ORACLE AND CMAKE_CXX_COMPILER_ID MATCHES "Clang|GNU")
        target_compile_options(soci_oracle PRIVATE -Wno-maybe-uninitialized)
    endif()

endfunction()

It assumes you've declared the details you want before calling the function. Use it something like the following:

FetchContent_Declare(soci
    GIT_REPOSITORY https://github.com/SOCI/soci
    GIT_TAG 125753ff2fa8457b980fc4186606b9b08b2612b5
)

myCustomDependencySoci(oracle postgresql)

- Issue `soci` not found when adding as subproject SOCI#762

msis closed this as completed Oct 21, 2019

mloskot added the CMake label Oct 21, 2019

abhinavpappu mentioned this issue Sep 8, 2020

SOCI CMake Sub Project help. #792

Closed

spjuanjoc pushed a commit to spjuanjoc/soci that referenced this issue Nov 25, 2020

change dst dir for soci-config.h

1bbe7ff

- Issue `soci` not found when adding as subproject SOCI#762

spjuanjoc mentioned this issue Nov 28, 2020

change source dir from where cmake will copy the "soci-config.h" file #836

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`soci` not found when adding as subproject #762

`soci` not found when adding as subproject #762

msis commented Oct 20, 2019

mloskot commented Oct 20, 2019

msis commented Oct 20, 2019

tt4g commented Oct 21, 2019 •

edited

Loading

msis commented Oct 21, 2019

msis commented Oct 21, 2019

tt4g commented Oct 21, 2019 •

edited

Loading

msis commented Oct 21, 2019

tt4g commented Oct 21, 2019

msis commented Oct 21, 2019 •

edited

Loading

tt4g commented Oct 21, 2019

msis commented Oct 21, 2019

tt4g commented Oct 21, 2019

tt4g commented Oct 21, 2019 •

edited

Loading

msis commented Oct 21, 2019 •

edited

Loading

msis commented Oct 21, 2019 •

edited

Loading

msis commented Oct 21, 2019

tt4g commented Oct 21, 2019 •

edited

Loading

msis commented Oct 21, 2019

tt4g commented Oct 21, 2019 •

edited

Loading

mloskot commented Oct 21, 2019 •

edited

Loading

msis commented Oct 21, 2019

mloskot commented Oct 21, 2019

craigscott-crascit commented Oct 22, 2019

soci not found when adding as subproject #762

soci not found when adding as subproject #762

Comments

msis commented Oct 20, 2019

Project description

src/CMakeLists.txt

src/test_app/CMakeLists.txt

spdlog.cmake

soci.cmake

mloskot commented Oct 20, 2019

msis commented Oct 20, 2019

tt4g commented Oct 21, 2019 • edited Loading

msis commented Oct 21, 2019

msis commented Oct 21, 2019

tt4g commented Oct 21, 2019 • edited Loading

msis commented Oct 21, 2019

tt4g commented Oct 21, 2019

msis commented Oct 21, 2019 • edited Loading

tt4g commented Oct 21, 2019

msis commented Oct 21, 2019

tt4g commented Oct 21, 2019

tt4g commented Oct 21, 2019 • edited Loading

msis commented Oct 21, 2019 • edited Loading

msis commented Oct 21, 2019 • edited Loading

msis commented Oct 21, 2019

tt4g commented Oct 21, 2019 • edited Loading

msis commented Oct 21, 2019

tt4g commented Oct 21, 2019 • edited Loading

mloskot commented Oct 21, 2019 • edited Loading

msis commented Oct 21, 2019

mloskot commented Oct 21, 2019

craigscott-crascit commented Oct 22, 2019

`soci` not found when adding as subproject #762

`soci` not found when adding as subproject #762

`src/CMakeLists.txt`

`src/test_app/CMakeLists.txt`

`spdlog.cmake`

`soci.cmake`

tt4g commented Oct 21, 2019 •

edited

Loading

tt4g commented Oct 21, 2019 •

edited

Loading

msis commented Oct 21, 2019 •

edited

Loading

tt4g commented Oct 21, 2019 •

edited

Loading

msis commented Oct 21, 2019 •

edited

Loading

msis commented Oct 21, 2019 •

edited

Loading

tt4g commented Oct 21, 2019 •

edited

Loading

tt4g commented Oct 21, 2019 •

edited

Loading

mloskot commented Oct 21, 2019 •

edited

Loading