Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[libxml2,libxslt] Revise dependency handling #24935

Merged
merged 39 commits into from
Jul 6, 2022

Conversation

dg0yt
Copy link
Contributor

@dg0yt dg0yt commented May 26, 2022

  • What does your PR fix?

    Feature crypto:
    Crypto support in libxslt requires gcrypt, so port libgcrypt is an unconditional dependency of feature crypto.
    But port libgcrypt doesn't support mingw ATM, so crypto cannot be a default feature for mingw.
    libgcrypt also needs libgpg-error. This is not handled by the embedded Find module, so this PR uses pkg-config instead.
    Crypto support is opt-in the CMake build system. That's why this PR removes the crypto feature from the default features.

    Misc:
    Feature python fails to build for mingw (on Linux), too. So it is marked as unsupported now.
    Removes unused direct dependency liblzma.
    Disables installation of documentation.
    Revises the libxml2 wrapper to resolve lookup and config mode quirks. (Related: find_package should not return system installed version if same package is installed from vcpkg #24995, [Part1|xwindow PR] Split up to dbus #22642 (comment).)
    Fixes openscap build issues due to vendored FindThreads.cmake.

  • Which triplets are supported/not supported? Have you updated the CI baseline?

    unchanged, no.

  • Does your PR follow the maintainer guide?

    yes

  • If you have added/updated a port: Have you run ./vcpkg x-add-version --all and committed the result?

    yes

@Neumann-A
Copy link
Contributor

if you are already touching libxslt could you remove the CONFIG from the find_dependency call to libxml2 in the installed libxslt-config.cmake to solve #22642 (comment)

@dg0yt dg0yt marked this pull request as draft May 26, 2022 11:50
@dg0yt
Copy link
Contributor Author

dg0yt commented May 26, 2022

if you are already touching libxslt could you remove the CONFIG from the find_dependency call to libxml2 in the installed libxslt-config.cmake to solve #22642 (comment)

I will check the suggestion.

I have more changes:

  • The crypto feature:
    • It defaults to off in the CMake build system. It should be removed from the default features.
    • It depends on libgcrypt, but doesn't deal with libgpg-error. I will probably switch to pkg-config for this dependency.
  • Do we want to turn off LIBXSLT_WITH_PROFILER? Edit: No.
  • I don't see that the liblzma dependency is used.

@dg0yt
Copy link
Contributor Author

dg0yt commented May 26, 2022

if you are already touching libxslt could you remove the CONFIG from the find_dependency call to libxml2 in the installed libxslt-config.cmake to solve #22642 (comment)

Removing CONFIG from libxslt-config.cmake is different from #22642 (comment) where you proposed to add the CONFIG keyword to the libxml2 wrapper instead.

Adding CONFIG to the libxml2 wrapper would enforce a consistent mode for all ports, but at the price of possibly diverging from
FindLibXml2.cmake behaviour.

I really don't know what is more robust in general, but IMO libxslt-config.cmake should use the same find_package mode as CMakeLists.txt. Which is CONFIG now.

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a new experimental fast check for PR issues. Please let us know if this bot is helpful!

PRs must add only one version and must not modify any published versions

When making any changes to a library, the version or port-version in vcpkg.json or CONTROL must be modified.

error: checked-in files for libxslt have changed but the version was not updated
version: 1.1.35#1
old SHA: 57c65cc1e575ef799b76c703e96bb48344118ba4
new SHA: 0871a88f0f9e95c81944d2aafc26328d7e6cec80
Did you remember to update the version or port version?
Use --overwrite-version to bypass this check
***No files were updated***

@dg0yt dg0yt changed the title [libxslt] Fix mingw build [libxslt] Revise dependency handling May 26, 2022
@dg0yt dg0yt marked this pull request as ready for review May 26, 2022 13:55
@Neumann-A
Copy link
Contributor

I really don't know what is more robust in general, but IMO libxslt-config.cmake should use the same find_package mode as CMakeLists.txt. Which is CONFIG now.

If it(libxslt) doesn't need CONFIG the CONFIG should go. Otherwise it makes it impossible to use other stuff which does not use CONFIG for libxml2. Forcing CONFIG in the libxml2 wrapper possibly also breaks custom FindLibXml2.

@Adela0814 Adela0814 added the category:port-feature The issue is with a library, which is requesting new capabilities that didn’t exist label May 27, 2022
@JackBoosY JackBoosY self-assigned this May 27, 2022
ports/libxslt/fix-gcrypt-deps.patch Outdated Show resolved Hide resolved
@dg0yt
Copy link
Contributor Author

dg0yt commented May 27, 2022

If it(libxslt) doesn't need CONFIG the CONFIG should go. Otherwise it makes it impossible to use other stuff which does not use CONFIG for libxml2.

This is what I say for our patches. But can I make this decision for upstream?

Forcing CONFIG in the libxml2 wrapper possibly also breaks custom FindLibXml2.

True, but OTOH we need to patch custom find modules quite often anyways (multi-config, debug postfix, platform support, transitive usage requirements, downstream support). With an official CMake Find module and an official exported config, I really don't need another custom find module...

@Neumann-A
Copy link
Contributor

But can I make this decision for upstream?

Yes since they don't need the CONFIG and create interoperability issues with it.

if(LIBXSLT_WITH_CRYPTO)
- find_package(Gcrypt REQUIRED)
+ find_package(PkgConfig REQUIRED)
+ pkg_check_modules(Gcrypt REQUIRED IMPORTED_TARGET libgcrypt)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
+ pkg_check_modules(Gcrypt REQUIRED IMPORTED_TARGET libgcrypt)
+ pkg_check_modules(xslt_gcrypt REQUIRED IMPORTED_TARGET libgcrypt)

better name this to something unique

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also better move the find_package(PkgConfig REQUIRED) into the FindGcrypt.cmake?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dislike patching and installing the custom find module.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dislike patching and installing the custom find module.

I understand that but patching FindGcrypt.cmake makes it upstream-able if the code is guarded by if(PkgConfig_FOUND)

Copy link
Contributor Author

@dg0yt dg0yt May 31, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From trying to build openscap after installing libxslt[crypto]:
No matter where I implement pkg_check_modules: The maintainer functions for CMake don't set PKG_CONFIG_PATH, so it doesn't don't find vcpkg's libgcrypt. (And for openscap and most other consumers, it is only needed at configuration time: These port use libxslt, not libexslt.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

openscap uses libgcrypt and libexslt, and it comes with another load of custom find modules. Oh no.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From trying to build openscap after installing libxslt[crypto]:
No matter where I implement pkg_check_modules: The maintainer functions for CMake don't set PKG_CONFIG_PATH, so it doesn't don't find vcpkg's libgcrypt.

The maintainer functions for CMake don't need to set PKG_CONFIG_PATH when we can rely on PKG_CONFIG_USE_CMAKE_PREFIX_PATH being initialized to true. But this requires CMAKE_MINIMUM_REQUIRED_VERSION to be at least 3.1.
openscap comes with CMAKE_MINIMUM_REQUIRED_VERSION set to 2.6. Since this is a particular issue of this port, I decided to set PKG_CONFIG_USE_CMAKE_PREFIX_PATH in that portfile.

@dan-shaw dan-shaw removed the info:reviewed Pull Request changes follow basic guidelines label Jun 23, 2022
@JackBoosY JackBoosY added the info:reviewed Pull Request changes follow basic guidelines label Jun 24, 2022
@Osyotr
Copy link
Contributor

Osyotr commented Jun 28, 2022

I've noticed that libxslt installs shell script into tools/bin and tools/debug/bin. It prevents finding xsltproc executable. These shell scripts are probably broken and should be removed. Could that be included in this PR?

@dg0yt
Copy link
Contributor Author

dg0yt commented Jun 29, 2022

I've noticed that libxslt installs shell script into tools/bin and tools/debug/bin. It prevents finding xsltproc executable. These shell scripts are probably broken and should be removed. Could that be included in this PR?

Probabyly the scripts are not broken because they are explicitly handled by the portfile, and I often verify such items.

But the toolchain is not adding tools/libxslt to CMAKE_PROGRAM_PATH if there is a bin subdir. It is unknown if this is a bug, due to #17607 being unsolved. I will move the scripts around.

github-actions[bot]
github-actions bot previously approved these changes Jun 29, 2022
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have modified or added at least one portfile where deprecated functions are used.

If you feel able to do so, please consider migrating them to the new functions:
vcpkg_install_cmake -> vcpkg_cmake_install (from port vcpkg-cmake)
vcpkg_build_cmake -> vcpkg_cmake_build (from port vcpkg-cmake)
vcpkg_configure_cmake -> vcpkg_cmake_configure (Please remove the option PREFER_NINJA) (from port vcpkg-cmake)
vcpkg_fixup_cmake_targets -> vcpkg_cmake_config_fixup (from port vcpkg-cmake-config)

In the ports that use the new function, you have to add the corresponding dependencies:

{
  "name": "vcpkg-cmake",
  "host": true
},
{
  "name": "vcpkg-cmake-config",
  "host": true
}

The following files are affected:

  • ports/openscap/portfile.cmake

You have modified or added at least one vcpkg.json where you should check the license field.

If you feel able to do so, please consider adding a "license" field to the following files:

  • ports/openscap/vcpkg.json

Valid values for the license field can be found in the documentation

@dg0yt
Copy link
Contributor Author

dg0yt commented Jun 29, 2022

Probabyly the scripts are not broken because they are explicitly handled by the portfile, and I often verify such items.

Found and fixed prefix issues.

@dg0yt
Copy link
Contributor Author

dg0yt commented Jun 29, 2022

Update: there is tools/libxslt in CMAKE_PROGRAM_PATH now.

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have modified or added at least one portfile where deprecated functions are used.

If you feel able to do so, please consider migrating them to the new functions:
vcpkg_install_cmake -> vcpkg_cmake_install (from port vcpkg-cmake)
vcpkg_build_cmake -> vcpkg_cmake_build (from port vcpkg-cmake)
vcpkg_configure_cmake -> vcpkg_cmake_configure (Please remove the option PREFER_NINJA) (from port vcpkg-cmake)
vcpkg_fixup_cmake_targets -> vcpkg_cmake_config_fixup (from port vcpkg-cmake-config)

In the ports that use the new function, you have to add the corresponding dependencies:

{
  "name": "vcpkg-cmake",
  "host": true
},
{
  "name": "vcpkg-cmake-config",
  "host": true
}

The following files are affected:

  • ports/openscap/portfile.cmake

You have modified or added at least one vcpkg.json where you should check the license field.

If you feel able to do so, please consider adding a "license" field to the following files:

  • ports/openscap/vcpkg.json

Valid values for the license field can be found in the documentation

@JackBoosY JackBoosY removed the requires:vcpkg-team-review This PR or issue requires someone on the vcpkg team to take a further look. label Jun 30, 2022
@dan-shaw dan-shaw added the requires:vcpkg-team-review This PR or issue requires someone on the vcpkg team to take a further look. label Jul 2, 2022
@dg0yt
Copy link
Contributor Author

dg0yt commented Jul 2, 2022

"Requires team review" again? Feedback, please.

@vicroms vicroms removed the requires:vcpkg-team-review This PR or issue requires someone on the vcpkg team to take a further look. label Jul 6, 2022
@vicroms
Copy link
Member

vicroms commented Jul 6, 2022

LGTM

@vicroms vicroms merged commit 5a8ff00 into microsoft:master Jul 6, 2022
@dg0yt dg0yt deleted the libxslt-mingw branch July 7, 2022 02:57
@JackBoosY
Copy link
Contributor

When building tmx[core]:x64-linux after buiding libxml2[core,tools]:x64-linux:

CMake Error at /mnt/vcpkg-ci/installed/x64-linux/share/libxml2/libxml2-export.cmake:37 (message):
  Some (but not all) targets in this export set were already defined.

  Targets Defined: LibXml2::LibXml2;LibXml2::xmllint

  Targets not yet defined: LibXml2::xmlcatalog

Call Stack (most recent call first):
  /mnt/vcpkg-ci/installed/x64-linux/share/libxml2/libxml2-config.cmake:21 (include)
  /mnt/vcpkg-ci/installed/x64-linux/share/libxml2/vcpkg-cmake-wrapper.cmake:2 (_find_package)
  /agent/_work/1/s/scripts/buildsystems/vcpkg.cmake:778 (include)
  CMakeLists.txt:52 (find_package)

In share/libxml2/libxml2-export.cmake:

set(_targetsDefined)
set(_targetsNotDefined)
set(_expectedTargets)
foreach(_expectedTarget LibXml2::LibXml2 LibXml2::xmlcatalog LibXml2::xmllint)
  list(APPEND _expectedTargets ${_expectedTarget})
  if(NOT TARGET ${_expectedTarget})
    list(APPEND _targetsNotDefined ${_expectedTarget})
  endif()
  if(TARGET ${_expectedTarget})
    list(APPEND _targetsDefined ${_expectedTarget})
  endif()
endforeach()
if("${_targetsDefined}" STREQUAL "${_expectedTargets}")
  unset(_targetsDefined)
  unset(_targetsNotDefined)
  unset(_expectedTargets)
  set(CMAKE_IMPORT_FILE_VERSION)
  cmake_policy(POP)
  return()
endif()
if(NOT "${_targetsDefined}" STREQUAL "")
  message(FATAL_ERROR "Some (but not all) targets in this export set were already defined.\nTargets Defined: ${_targetsDefined}\nTargets not yet defined: ${_targetsNotDefined}\n")
endif()
unset(_targetsDefined)
unset(_targetsNotDefined)
unset(_expectedTargets)

@dg0yt
Copy link
Contributor Author

dg0yt commented Jul 7, 2022

Targets Defined: LibXml2::LibXml2;LibXml2::xmllint

Targets not yet defined: LibXml2::xmlcatalog

Did you really test with the last version of port libxml2? I remember that this was an issue in the past, when the find module was loaded before the config. But now the wrapper always loads the config. Just because of this issue.

@dg0yt
Copy link
Contributor Author

dg0yt commented Jul 7, 2022

.. okay, reproduced.

@dg0yt
Copy link
Contributor Author

dg0yt commented Jul 7, 2022

tmx bug:

include(FindLibXml2)
find_package(LibXml2 REQUIRED)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category:port-feature The issue is with a library, which is requesting new capabilities that didn’t exist info:reviewed Pull Request changes follow basic guidelines
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants