Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[vcpkg] Improve make builds #10402

Merged
merged 59 commits into from
May 21, 2020
Merged

Conversation

Neumann-A
Copy link
Contributor

@Neumann-A Neumann-A commented Mar 12, 2020

Trying to correctly fix make builds for msvc

related to #9966 (main logic to make msvc builds work are taken from there)

@JackBoosY
Copy link
Contributor

Thanks @Neumann-A !

@Neumann-A
Copy link
Contributor Author

Neumann-A commented Mar 13, 2020

hmm some of those build failures I am unable to reproduce locally

This one is also kind of strange and only appears in some ports:

CMake Error at scripts/cmake/vcpkg_execute_required_process.cmake:58 (file):
  file failed to open for reading (No such file or directory):

    /mnt/g/qt/buildtrees/healpix/config-x64-linux-dbg-out.log
Call Stack (most recent call first):
  scripts/cmake/vcpkg_configure_make.cmake:388 (vcpkg_execute_required_process)
  ports/healpix/portfile.cmake:17 (vcpkg_configure_make)

wrong subpaths

@Neumann-A
Copy link
Contributor Author

@ras0219-msft: Please tell us how to deal with the compiler flags in the different configure_<scripts> theoretically this should be all done from within the toolchain file!

@MVoz
Copy link
Contributor

MVoz commented Mar 15, 2020

no package under MSVC is compiled, each one has an error in the package code

the edits themselves for the build from MAKE are working

plus the problem of finding libraries, in the absence of pkgconfig (*.pc)

working test MSVC gperf-tools:x64-windows

vcpkg_download_distfile(ARCHIVE
    URLS "http://git.savannah.gnu.org/cgit/gperf.git/snapshot/gperf-3.1.tar.gz"
    FILENAME "gperf-3.1.tar.gz"
    SHA512 0ff3a9e8962a5ed29907d035d7ff54138620ce77caab04ac85f364f2a4d695a3d5d5a522117b71d35455cb1523ef5297a39e24eae3a89a84e7f6734959d08176
)

vcpkg_extract_source_archive_ex(
    OUT_SOURCE_PATH SOURCE_PATH
    ARCHIVE ${ARCHIVE}
)

vcpkg_configure_make(
        SOURCE_PATH ${SOURCE_PATH}
)

vcpkg_install_make()

@MVoz
Copy link
Contributor

MVoz commented Mar 15, 2020

not work code, script configure work MSVC test

vcpkg install antioch:x64-windows

Computing installation plan...
The following packages will be built and installed:
    antioch[core]:x64-windows
Starting package 1/1: antioch:x64-windows
Building package antioch[core]:x64-windows...
-- Acquiring MSYS Packages...
-- Acquiring MSYS Packages... OK
-- Prerun shell with x64-windows-dbg
-- Configuring x64-windows-dbg
-- Configuring x64-windows-rel
-- Building x64-windows-dbg
CMake Error at
...
find_program(GIT NAMES git git.cmd)
set(GIT_URL "https://github.com/libantioch/antioch")
set(GIT_REV master)
set(SOURCE_PATH ${CURRENT_BUILDTREES_DIR}/src/${PORT})
if(NOT EXISTS "${SOURCE_PATH}/.git")
    message(STATUS "Cloning")
    vcpkg_execute_required_process(
      COMMAND ${GIT} clone --recurse-submodules -q --depth=20 --branch=${GIT_REV} ${GIT_URL} ${SOURCE_PATH}
#      COMMAND ${GIT} clone submodule sync --recursive -q --depth=20 --branch=${GIT_REV} ${GIT_URL} ${SOURCE_PATH}
      WORKING_DIRECTORY ${SOURCE_PATH}
      LOGNAME clone
    )
    message(STATUS "Fetching submodules")
    vcpkg_execute_required_process(
      COMMAND ${GIT} submodule update --init --recursive
      WORKING_DIRECTORY ${SOURCE_PATH}
      LOGNAME submodules
    )
endif()

vcpkg_configure_make(
    SOURCE_PATH ${SOURCE_PATH}
    COPY_SOURCE
    PRERUN_SHELL ${SOURCE_PATH}/bootstrap
    OPTIONS
      "--with-eigen-include='\${prefix}'/include"
    OPTIONS_DEBUG
      "--with-boost='\${prefix}'/debug/lib" # work (/e/...) , not work ${CURRENT_INSTALLED_DIR}/debug/lib" (e:/...)
    OPTIONS_RELEASE
      "--with-boost='\${prefix}'/lib"
)

vcpkg_install_make()

@MVoz
Copy link
Contributor

MVoz commented Mar 17, 2020

@dan-shaw
merge, I see no reason for the delay, everything works, it is not possible to test ports without merging

@JackBoosY
Copy link
Contributor

REGRESSION:

  • freexl:x64-osx
  • farmhash:x64-linux
  • libudns:x64-linux

@MVoz
Copy link
Contributor

MVoz commented May 16, 2020

@Neumann-A

had a good start, it would have been good to finish and combine/merge

ports/libwandio/portfile.cmake Outdated Show resolved Hide resolved
ports/x264/portfile.cmake Outdated Show resolved Hide resolved
Neumann-A and others added 2 commits May 18, 2020 14:54
Co-authored-by: Jack·Boos·Yu <47264268+JackBoosY@users.noreply.github.com>
@JackBoosY JackBoosY added the info:reviewed Pull Request changes follow basic guidelines label May 19, 2020
@JackBoosY
Copy link
Contributor

Ping @strega-nil for review this PR.

@strega-nil
Copy link
Contributor

Alright, thanks so much @Neumann-A, this is incredible!

@strega-nil strega-nil merged commit 27c6c94 into microsoft:master May 21, 2020
@Neumann-A Neumann-A deleted the improve_make_builds branch May 21, 2020 23:20
@dg0yt
Copy link
Contributor

dg0yt commented Apr 10, 2021

This PR changed the pkg-config configuration from using PKG_CONFIG_PATH to --define-variable=prefix=..., and I wonder why this was done.

I'm asking because I see a compilation error for GDAL on Linux related to OpenEXR. OpenEXR seems to be installed in the Linux image (Azure pipelines), as it is found via pkg-config by GDAL, but not a dependency in vcpkg. There may be an insufficient configuration test on the gdal side (it tests linking, but not including headers). On the vcpkg side, IIUC the consequence of --define-variable=prefix=... is that .pc files are found in the system directories, but headers and libraries are redirected to the vcpkg installation. I.e. what is meant as /usr/lib/x86_64-linux-gnu by the system .pc file becomes /home/vsts/work/1/s/installed/x64-linux/lib/x86_64-linux-gnu when used in vcpkg. Is this intented?

I suppose what is desired is that .pc files, includes and libraries are used from vcpkg only, similar to cross-compilation contexts. In my experience, this requires setting PKG_CONFIG_PATH, PKG_CONFIG_LIBDIR and PKG_CONFIG_SYSROOT_DIR, and this is often done in wrapper scripts. Cf. https://autotools.io/pkgconfig/cross-compiling.html

@Neumann-A
Copy link
Contributor Author

@dg0yt at the time vcpkg_configure_make was not really useable on any platform. Pc files were not corrected and the prefix was a hardcoded absolute path. The --define-variable was introduced before $(pcfiledir) was known about and integrated into vcpkg_fixup_pkgconfig. Before that the prefix variable was completly removed which made setting it a requirement. If it is still in vcpkg_configure_make and creating problems just remove it. It is no longer necessary

@dg0yt
Copy link
Contributor

dg0yt commented Apr 10, 2021

@Neumann-A Thank you. I will try without. Looking forward to test with Android triplets soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
info:reviewed Pull Request changes follow basic guidelines
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

10 participants