Latest Boost refactor breaks CMake Boost targets

[jasjuang]

Prior to the Boost 1.66 refactor, I can use cmake to find boost with the below without a problem

cmake_minimum_required(VERSION 3.10)

project(Example)

set(PROJECT_SRCS ${PROJECT_SOURCE_DIR}/mycpp.cpp) 

find_package(Boost COMPONENTS filesystem REQUIRED)

add_executable(${PROJECT_NAME} ${PROJECT_SRCS})

target_link_libraries(${PROJECT_NAME} Boost::filesystem)

but now it shows me the error

CMake Error at C:/dev/vcpkg/scripts/buildsystems/vcpkg.cmake:138 (_add_executable):
  Target "Example" links to target "Boost::filesystem" but the target was not
  found.  Perhaps a find_package() call is missing for an IMPORTED target, or
  an ALIAS target is missing?
Call Stack (most recent call first):
  CMakeLists.txt:28 (add_executable)

Even if I change back to old way of using boost in cmake

cmake_minimum_required(VERSION 3.10)

project(Example)

set(PROJECT_SRCS ${PROJECT_SOURCE_DIR}/mycpp.cpp) 

find_package(Boost COMPONENTS filesystem REQUIRED)

add_executable(${PROJECT_NAME} ${PROJECT_SRCS})

target_include_directories(${PROJECT_NAME} SYSTEM PUBLIC
                                          ${Boost_INCLUDE_DIRS})

target_link_libraries(${PROJECT_NAME} ${Boost_LIBRARIES})

I still get the below warnings in this minimal example

-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Warning at C:/Program Files/CMake/share/cmake-3.10/Modules/FindBoost.cmake:801 (message):
  New Boost version may have incorrect or missing dependencies and imported
  targets
Call Stack (most recent call first):
  C:/Program Files/CMake/share/cmake-3.10/Modules/FindBoost.cmake:907 (_Boost_COMPONENT_DEPENDENCIES)
  C:/Program Files/CMake/share/cmake-3.10/Modules/FindBoost.cmake:1542 (_Boost_MISSING_DEPENDENCIES)
  C:/dev/vcpkg/scripts/buildsystems/vcpkg.cmake:176 (_find_package)
  CMakeLists.txt:24 (find_package)


CMake Warning at C:/Program Files/CMake/share/cmake-3.10/Modules/FindBoost.cmake:801 (message):
  New Boost version may have incorrect or missing dependencies and imported
  targets
Call Stack (most recent call first):
  C:/Program Files/CMake/share/cmake-3.10/Modules/FindBoost.cmake:907 (_Boost_COMPONENT_DEPENDENCIES)
  C:/Program Files/CMake/share/cmake-3.10/Modules/FindBoost.cmake:1542 (_Boost_MISSING_DEPENDENCIES)
  C:/dev/vcpkg/scripts/buildsystems/vcpkg.cmake:176 (_find_package)
  CMakeLists.txt:24 (find_package)


-- Boost version: 1.66.0
-- Found the following Boost libraries:
--   filesystem

However in my main project it still breaks and I haven't pinpoint down on what exactly the reason is.

Can we fix it so the new boost refactor will work nicely when used by the downstream project?

[ras0219-msft]

This is unfortunately due to the boost upgrade, not the refactoring.

Every time boost updates, the CMake targets are broken until CMake updates to include the new version's dependency info. I personally don't like that arrangement, but that's where we are. This is also why you get warnings.

As for why your main project still breaks, I'm not sure. What is broken specifically? It could be that you no longer have dependency information, so you need to manually request the system component as well.

[jasjuang]

Got it, this is indeed a bad arrangement. I will file an issue on cmake and hopefully they can release a new cmake 3.10 with a minor version bump.

[jasjuang]

@ras0219-msft Someone already filed an issue and some are attempting to fix it https://gitlab.kitware.com/cmake/cmake/issues/17575.

In the mean time I would like to temporarily downgrade boost to 1.65 in order to get my project to compile again. However, the problem is with the new boost modularization, there is no trivial way for me to downgrade it to 1.65 unless I go to every boost portfile to change the ref to 1.65 and update the SHA512 which is quite tedious. Do you have any recommendations?

[JLospinoso]

Same issue.

[ras0219-msft]

To downgrade boost with modularization, I think the best approach would be to edit the powershell scripts inside ports\boost-vcpkg-helpers\ (specifically generate-ports.ps1). You could then re-run them with the older boost version and they should regenerate all the ports.

To downgrade boost without modularization, you can use git checkout 5335d17 -- ports/boost. Note that this may break the dependency information for libraries which were updated to use the modularized form.

My personal recommendation is (unfortunately) to just use the macros (Boost_LIBRARIES and Boost_INCLUDE_DIRS), which means you'll be able to update to Boost 1.67, 1.68, etc as soon as it's released instead of always being months behind.

[jasjuang]

@ras0219-msft I am trying to downgrade with modularization. I went into generate-ports.ps1 and change it to 1.65. When I execute it in powershell I get the below error

Unpacking boost/lambda...
      [known] config detail utility mpl type_traits bind
    [unknown]
Handling boost/lexical_cast...
Downloading boost/lexical_cast...
vcpkg : The term 'vcpkg' is not recognized as the name of a cmdlet, function, script file, or operable program. Check
the spelling of the name, or if a path was included, verify that the path is correct and try again.
At C:\dev\vcpkg\ports\boost-vcpkg-helpers\generate-ports.ps1:216 char:13
+     $hash = vcpkg hash $archive
+             ~~~~~
    + CategoryInfo          : ObjectNotFound: (vcpkg:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

When I look into the actual portfile of boost modules, it got downgraded to 1.65 but the SHA512 section is empty. Can you help me?

[ras0219-msft]

Yeah, it seems i depend on vcpkg being on the path in the script. Adding it should fix the error.

[jasjuang]

@ras0219-msft After I did $env:Path += ";C:\dev\vcpkg" to add vcpkg to path. There is another weird problem, some of the SHA512 is wrong because of the survey

# Automatically generated by boost-vcpkg-helpers/generate-ports.ps1

include(vcpkg_common_functions)
include(${CURRENT_INSTALLED_DIR}/share/boost-vcpkg-helpers/boost-modular.cmake)

vcpkg_from_github(
    OUT_SOURCE_PATH SOURCE_PATH
    REPO boostorg/algorithm
    REF boost-1.65.0
    SHA512 Your feedback is important to improve Vcpkg! Please take 3 minutes to complete our survey by running: vcpkg contact --survey 8177292ef7103bf147c0c449e39d9529609dea3cf1da276bed062550559e8c0bc48391137fe71484ae136ca5644bf083fd8f42be0035e15d8b03550549d245e2
    HEAD_REF master
)

boost_modular_headers(SOURCE_PATH ${SOURCE_PATH})

How to fix this?

[ras0219-msft]

The immediate fix is to run vcpkg contact --survey :) However, this definitely indicates a need for a command line argument to disable that prompt.

[jasjuang]

@ras0219-msft Run vcpkg contact --survey solves it, but another new problem is some of the hashes are not extracted correctly by the script. For example, boost locale

-- CURRENT_PORT_DIR=C:/dev/vcpkg/ports/boost-locale/.
-- Using cached C:/dev/vcpkg/downloads/boostorg-locale-boost-1.65.0.tar.gz
-- Testing integrity of cached file...
CMake Error at C:/dev/vcpkg/scripts/cmake/vcpkg_download_distfile.cmake:66 (message):


  File does not have expected hash:

          File path: [ C:/dev/vcpkg/downloads/boostorg-locale-boost-1.65.0.tar.gz ]
      Expected hash: [ 8c4eb6dcfffa15e2e9f3405aa1deb883283d2a07612fc310208c71c82b3405a30795eec8394cd5377cfc09269e102354b03960f1d56972138a3ff6c1a8c726e2 ]
        Actual hash: [ b3de95b19578f7c731aac4cbfd60d889a5f28c78d77c101ee71e610918298927958f22c8358d50b2d8216948623a63bd0afb8ef51923ccdca6b799f4988051a1 ]

  Please delete the file and retry if this file should be downloaded again.

even if I update the hash manually there are new compilation errors

Unknown compiler version - please run the configure tests and report the results
src\win32\lcid.cpp(22): fatal error C1083: Cannot open include file: 'boost/thread/mutex.hpp': No such file or directory

    call "C:/dev/vcpkg/installed/x64-windows/share/boost-vcpkg-helpers/nothing.bat" amd64 >nul
cl @"C:\dev\vcpkg\buildtrees\boost-locale\x64-windows-rel\boost\build\c0e6c0ef6c68bf3b53954fb6c0dfabbc\win32\lcid.obj.rsp" 

...failed compile-c-c++ C:\dev\vcpkg\buildtrees\boost-locale\x64-windows-rel\boost\build\c0e6c0ef6c68bf3b53954fb6c0dfabbc\win32\lcid.obj...
compile-c-c++ C:\dev\vcpkg\buildtrees\boost-locale\x64-windows-rel\boost\build\c0e6c0ef6c68bf3b53954fb6c0dfabbc\win32\collate.obj
collate.cpp
Unknown compiler version - please run the configure tests and report the results
c:\dev\vcpkg\buildtrees\boost-locale\src\locale-boost-1.65.0\src\win32\api.hpp(149): warning C4244: 'argument': conversion from '__int64' to 'int', possible loss of data
c:\dev\vcpkg\buildtrees\boost-locale\src\locale-boost-1.65.0\src\win32\api.hpp(153): warning C4267: 'argument': conversion from 'size_t' to 'int', possible loss of data
c:\dev\vcpkg\buildtrees\boost-locale\src\locale-boost-1.65.0\src\win32\api.hpp(153): warning C4244: 'argument': conversion from '__int64' to 'int', possible loss of data
c:\dev\vcpkg\buildtrees\boost-locale\src\locale-boost-1.65.0\src\win32\api.hpp(170): warning C4244: 'argument': conversion from '__int64' to 'int', possible loss of data
c:\dev\vcpkg\buildtrees\boost-locale\src\locale-boost-1.65.0\src\win32\api.hpp(245): warning C4244: 'argument': conversion from '__int64' to 'int', possible loss of data
c:\dev\vcpkg\buildtrees\boost-locale\src\locale-boost-1.65.0\src\win32\api.hpp(249): warning C4244: 'argument': conversion from '__int64' to 'int', possible loss of data
compile-c-c++ C:\dev\vcpkg\buildtrees\boost-locale\x64-windows-rel\boost\build\c0e6c0ef6c68bf3b53954fb6c0dfabbc\encoding\codepage.obj
codepage.cpp
Unknown compiler version - please run the configure tests and report the results
c:\dev\vcpkg\buildtrees\boost-locale\src\locale-boost-1.65.0\src\encoding\wconv_codepage.ipp(129): warning C4244: 'argument': conversion from '__int64' to 'int', possible loss of data
c:\dev\vcpkg\buildtrees\boost-locale\src\locale-boost-1.65.0\src\encoding\wconv_codepage.ipp(139): warning C4267: 'argument': conversion from 'size_t' to 'int', possible loss of data
c:\dev\vcpkg\buildtrees\boost-locale\src\locale-boost-1.65.0\src\encoding\wconv_codepage.ipp(139): warning C4244: 'argument': conversion from '__int64' to 'int', possible loss of data
c:\dev\vcpkg\buildtrees\boost-locale\src\locale-boost-1.65.0\src\encoding\wconv_codepage.ipp(152): warning C4244: 'argument': conversion from '__int64' to 'int', possible loss of data
c:\dev\vcpkg\buildtrees\boost-locale\src\locale-boost-1.65.0\src\encoding\wconv_codepage.ipp(155): warning C4244: 'argument': conversion from '__int64' to 'int', possible loss of data
c:\dev\vcpkg\buildtrees\boost-locale\src\locale-boost-1.65.0\src\encoding\wconv_codepage.ipp(415): warning C4244: 'argument': conversion from '__int64' to 'unsigned int', possible loss of data
c:\dev\vcpkg\buildtrees\boost-locale\src\locale-boost-1.65.0\src\encoding\wconv_codepage.ipp(406): note: while compiling class template member function 'std::string boost::locale::conv::impl::wconv_from_utf<CharType,2>::convert(const CharType *,const CharType *)'
        with
        [
            CharType=wchar_t
        ]
src\encoding\codepage.cpp(106): note: see reference to class template instantiation 'boost::locale::conv::impl::wconv_from_utf<CharType,2>' being compiled
        with
        [
            CharType=wchar_t
        ]
src\encoding\codepage.cpp(161): note: see reference to function template instantiation 'std::string boost::locale::conv::impl::convert_from<wchar_t>(const CharType *,const CharType *,const char *,boost::locale::conv::method_type)' being compiled
        with
        [
            CharType=wchar_t
        ]
c:\dev\vcpkg\buildtrees\boost-locale\src\locale-boost-1.65.0\src\encoding\wconv_codepage.ipp(424): warning C4244: 'argument': conversion from '__int64' to 'unsigned int', possible loss of data
compile-c-c++ C:\dev\vcpkg\buildtrees\boost-locale\x64-windows-rel\boost\build\c0e6c0ef6c68bf3b53954fb6c0dfabbc\util\info.obj
info.cpp
Unknown compiler version - please run the configure tests and report the results
compile-c-c++ C:\dev\vcpkg\buildtrees\boost-locale\x64-windows-rel\boost\build\c0e6c0ef6c68bf3b53954fb6c0dfabbc\std\collate.obj
collate.cpp
Unknown compiler version - please run the configure tests and report the results
compile-c-c++ C:\dev\vcpkg\buildtrees\boost-locale\x64-windows-rel\boost\build\c0e6c0ef6c68bf3b53954fb6c0dfabbc\shared\ids.obj
ids.cpp
Unknown compiler version - please run the configure tests and report the results
...failed updating 3 targets...
...updated 17 targets...

EDIT: Another question is how can I install all boost modules? After the vcpkg upgrade fail I lose track of the boost I installed previously and vcpkg install boost-* doesn't work.

EDIT2: I found the answer for my previous edit, it's simply vcpkg install boost, but I am still having installing some of the boost modules for 1.65 version.

[jasjuang]

The PR to support boost 1.66 is merged https://gitlab.kitware.com/cmake/cmake/merge_requests/1625. Now we just need to wait for the next minor release for cmake to come out.

[jasjuang]

@ras0219-msft I figured out that CMake actually provides nightly builds so I tried it out and unfortunately it still doesn't work. I filed an issue here https://gitlab.kitware.com/cmake/cmake/issues/17637 but the comments below said it might be a vcpkg problem. Could you take a look?

The comments from the cmake issue:

The libraries cannot be found because of the missing architecture in the library name.

Filename created by vcpkg: boost_system-vc140-mt-1_66.lib
Filename cmake expects: boost_system-vc140-mt-x64-1_66.lib (with -x64)

Issue might be caused by vcpkg, see https://github.com/Microsoft/vcpkg/blob/master/ports/boost-vcpkg-helpers/boost-modular.cmake#L244

[jasjuang]

Another guy in the cmake issue confirms that the fix works for him and his boost is not installed via vcpkg, so this error is caused by vcpkg, can someone look into it and fix it?

[ras0219-msft]

Ok, given those changes the only way I can see to make Boost 1.66 compatible with CMake < 3.10 as well as this new version is to remove the version tag from the lib files.

I've done this in a branch (dev/roschuma/boost-cmake-fix-2459) and made sure it still works with the current versions of CMake. Could you confirm that it fixes your issue with the new nightlies?

[jasjuang]

@ras0219-msft I just tried your branch and I can confirm it is working for me. Would love to see it merged to master.

On a side note, why is there a need to make it compatible with CMake < 3.10? Isn't the minimal CMake requirements for using vcpkg already CMake 3.10?

[ras0219-msft]

No, not at all actually! We do demand the "latest" cmake internally (to make life easy for library authors), but we don't require users to use that in their own projects.

Additionally, the version of CMake inside VS2017 can get up to one minor version behind because it gets updated during VS releases.