-
-
Notifications
You must be signed in to change notification settings - Fork 6.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
curl-config in C supporting CMake builds #15854
Conversation
modified: CMakeLists.txt Update config to have correct LINK.exe formatting for --libs flags and replace use of curl-config script with curl-config C program new file: curl-config.c.in Add .in file for generated source that will serve as the cross-platform curl-config script replacement. This supports the CMake FindCURL module better, as on Windows the config mode search can fail.
modified: CMakeLists.txt Add lit_includedir, lit_libdir, lit_exec_prefix that are fully expanded paths as we cannot use something like ${prefix}/lib in C code modified: curl-config.c.in Refactor hardcoded @Prefix@/lib and @Prefix@/include to use @lit_libdir@ and @lib_includedir@ respectively as appropriate. Make STREQ just a name, not a function-like macro, so we don't worry about macro expansion. Update comments and fix libcurl_imp.lib typo (was libcurl_impl.lib).
…onfiguration was done with CMake
/* exec prefix (fully expanded). we don't use this however */ | ||
/* static const char *exec_prefix = "@lit_exec_prefix@"; */ | ||
/* pkg-config C/C++ flags */ | ||
static const char *cppflag_curl_staticlib = "@LIBCURL_PC_CFLAGS@"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I kept the variable names similar for the most part to make it easier to compare against curl-config.in
.
" --cc compiler\n" | ||
" --cflags preprocessor and compiler flags\n" | ||
" --checkfor [version] check for (lib)curl of the specified version\n" | ||
" --configure the arguments given to configure when building curl\n" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This still prints nothing (see CMakeLists.txt
TODO for CONFIGURE_OPTIONS
).
It could be changed to some CMake configure options, e.g. -DCURL_USE_SCHANNEL=ON -DCURL_USE_LIBPSL=OFF
but how exactly this is done probably will require some CMake scripting to get a consistent format.
Ok, so across the failing CI runs it's the
# curl-config as C program (from CMake)
if (-B $ARGV[0]) {
open(CURLCONFIG, "$ARGV[0] --$what|") ||
die "Can't get curl-config C program $what list\n";
}
# curl-config as shell script (autotools)
else {
open(CURLCONFIG, "sh $ARGV[0] --$what|") ||
die "Can't get curl-config shell script $what list\n";
} This seems to work locally after I finally figured out the second argument to Edit: Was being overly optimistic. Some CI jobs don't seem like they allow |
modified: tests/libtest/test1013.pl modified: tests/libtest/test1022.pl Insert conditional to invoke directly if curl-config is binary, otherwise use the shell to run the text curl-config script
Analysis of PR #15854 at 6caf23f4: Test http/test_14_auth.py::TestAuth::test_14_03_digest_put_auth[0-h3] failed, which has NOT been flaky recently, so there could be a real issue in this PR. Test 1014 failed, which has NOT been flaky recently, so there could be a real issue in this PR. Note that this test has failed in 8 different CI jobs (the link just goes to one of them). Test 1013 failed, which has NOT been flaky recently, so there could be a real issue in this PR. Note that this test has failed in 8 different CI jobs (the link just goes to one of them). Test 1022 failed, which has NOT been flaky recently, so there could be a real issue in this PR. Note that this test has failed in 8 different CI jobs (the link just goes to one of them). Test 1023 failed, which has NOT been flaky recently, so there could be a real issue in this PR. Note that this test has failed in 8 different CI jobs (the link just goes to one of them). Generated by Testclutch |
Curl provides a Is there a reaon why it did not work with |
I'm not entirely sure why the CMake config mode search failed, but it definitely is failing. E.g. consider: find_package(CURL 7.68 CONFIG REQUIRED COMPONENTS HTTPS) Here find module mode is disabled, so only
As an addendum, the provided find_package(CURL QUIET NO_MODULE) |
I propose #15858 which adds feature/protocol info to The A workaround may be to use Other methods is to not rely on this FindCURL feature but implement it # method 1
find_package(PkgConfig QUIET)
pkg_get_variable(_proto_str libcurl supported_protocols)
message(STATUS "protocols method 1|${_proto_str}|")
# method 2
find_program(CURL_EXECUTABLE NAMES curl)
execute_process(COMMAND ${CURL_EXECUTABLE} --disable --version OUTPUT_VARIABLE _curl_V
ERROR_QUIET OUTPUT_STRIP_TRAILING_WHITESPACE)
set(_proto_regex "Protocols: (.+)")
string(REGEX MATCH "${_proto_regex}" _proto_str ${_curl_V})
string(REGEX REPLACE "${_proto_regex}" "\\1" _proto_str "${_proto_str}")
message(STATUS "protocols method 2|${_proto_str}|")
# method 3 requires #15858
find_package(CURL)
message(STATUS "protocols method 3|${CURL_SUPPORTED_PROTOCOLS}|")
message(STATUS "HTTPS supported|${CURL_SUPPORTED_HTTPS_FOUND}|") As for converting |
I think this is good. However, maybe some extra work needs to be done for On a related note, I think
If config search mode is failing, however, then something must not be right with the
Yes, I 100% agree. Rewriting Anyways, I think for now I'll use a CMake workaround, and if my opinion matters at all, I am definitely in favor of having some changes done to CURL's CMake config file as a more targeted solution. Thanks for the discusssion! |
Ah, interesting find after I decided to directly edit the installed After manually inserting Furthermore, confirming what I read in the I suppose more work is required for full integration with CMake's own |
I'm still trying to figure out where In my non-Windows env We can emulate some of the stuff done by FindCURL, but it looks kind of messy. Maybe curl should set its own variables names where things are in order? Here's a patch implementing some of these: diff --git a/CMake/curl-config.cmake.in b/CMake/curl-config.cmake.in
index cce77d122..7cf798bfb 100644
--- a/CMake/curl-config.cmake.in
+++ b/CMake/curl-config.cmake.in
@@ -52,3 +52,21 @@ set_and_check(CURL_INCLUDE_DIRS "@PACKAGE_CMAKE_INSTALL_INCLUDEDIR@")
string(REPLACE " " ";" CURL_SUPPORTED_PROTOCOLS "@SUPPORT_PROTOCOLS@")
string(REPLACE " " ";" CURL_SUPPORTED_FEATURES "@SUPPORT_FEATURES@")
+
+foreach(_item IN LISTS CURL_SUPPORTED_PROTOCOLS)
+ set(CURL_PROTOCOL_${_item}_FOUND TRUE)
+endforeach()
+foreach(_item IN LISTS CURL_SUPPORTED_FEATURES)
+ set(CURL_FEATURE_${_item}_FOUND TRUE)
+endforeach()
+
+foreach(_item IN LISTS CURL_FIND_COMPONENTS)
+ if(NOT "${CURL_PROTOCOL_${_item}_FOUND}" AND
+ NOT "${CURL_FEATURE_${_item}_FOUND}")
+ if(CURL_FIND_REQUIRED)
+ message(FATAL_ERROR "CURL: Required protocol/feature '${_item}' is not found")
+ else()
+ message(WARNING "CURL: Protocol/feature '${_item}' is not found")
+ endif()
+ endif()
+endforeach() |
Apologies in advance for the long response--you've given me a lot of good discussion :)
Yes, on reflection this does not need to be explicitly set by
My mistake, With regards to the config mode search issue, I think I've narrowed down why exactly curl/CMake/curl-config.cmake.in Line 42 in 8a66c11
Since find_package(CURL 7.68 CONFIG REQUIRED COMPONENTS HTTPS)
Yes, the In my mind I've seen an issue before where libcurl was mistakenly rebuilt without libz support and that caused HTTPS GET requests to an in-house REST endpoint to fail because that particular endpoint always sends data in gzip compressed chunks.
Yes, as I mentioned the
Yes, that would be ideal. I looked at your patch and have made some changes to better conform to CMake conventions, e.g. Here's a sample diff --git a/CMake/curl-config.cmake.in b/CMake/curl-config.cmake.in
index aa9eb51ff..6b4ce7f57 100644
--- a/CMake/curl-config.cmake.in
+++ b/CMake/curl-config.cmake.in
@@ -39,7 +39,33 @@ if("@HAVE_LIBZ@")
endif()
include("${CMAKE_CURRENT_LIST_DIR}/@TARGETS_EXPORT_NAME@.cmake")
-check_required_components("@PROJECT_NAME@")
+
+# set located components if requested
+# for each user-requested component <comp>, CURL_<comp>_FOUND is set to TRUE
+foreach(_req_comp ${CURL_FIND_COMPONENTS})
+ # first check features
+ list(FIND CURL_SUPPORTED_FEATURES ${_req_comp} CURL_${_req_comp}_POS)
+ # if not found, try again
+ if(CURL_${_req_comp}_POS EQUAL -1)
+ list(FIND CURL_SUPPORTED_PROTOCOLS ${_req_comp} CURL_${_req_comp}_POS)
+ endif()
+ # failed
+ if(CURL_${_req_comp}_POS EQUAL -1)
+ # if REQUIRED was given + component is not optional, hard error
+ if(CURL_FIND_REQUIRED_${_req_comp} AND CURL_FIND_REQUIRED)
+ message(FATAL_ERROR "CURL: missing component: ${_req_comp}")
+ # otherwise, just notify
+ else()
+ message(STATUS "CURL: missing component: ${_req_comp}")
+ endif()
+ # success, so mark component as found. this is necessary for correct
+ # interop with find_package_handle_standard_args and optional components
+ else()
+ set(CURL_${_req_comp}_FOUND TRUE)
+ endif()
+ # unset unused variable
+ unset(CURL_${_req_comp}_POS)
+endforeach()
# Alias for either shared or static library
if(NOT TARGET @PROJECT_NAME@::libcurl) Do note this is assuming I've tested this with the following set of
This seems to work well, as the optional components are correctly marked as not found, and even if If this ok, hopefully this can be accepted--from a CMake standpoint this is ideal behavior. |
Thanks for your inputs and research! In the meantime I also made some changes to the patch proposal It drops One difference is that it sets Can you give it a test? |
No problem, thank you for the great discussion. It's valuable to have your perspective as someone more actively involved in the project; I am an "outsider" so my knowledge on how proposed changes might interact with the CURL project is limited. I will take a look at #15858 and I hope you won't mind if I leave some comments when appropriate.
From what I understand from the CMake documentation, for any With regards to indicating all the supported features + protocols, I think downstream users can use
Of course, would be my pleasure. I'll comment later on my findings. |
Looks like the changes in #15858 are working nicely. I tested multiple Left some comments on the PR; let me know your thoughts. And happy New Year's Eve :) |
Via these variables, as lists: - `CURL_SUPPORTED_PROTOCOLS` - `CURL_SUPPORTED_FEATURES` As individual flags: - `CURL_SUPPORTS_<protocol/feature>` = `TRUE` Also: - set `CURL_VERSION_STRING` which was missing when using `find_package(CURL CONFIG)` or `find_package(CURL NO_MODULE)`. - set `CURL_<prototol/feature>_FOUND` for compatibility. - show full list of missing but required `COMPONENTS`. Assisted-by: Derek Huang Fixes #15854 Closes #15858
Motivation
To support a project on Windows I built libcurl from the Git repo checked out to 8.11.2-DEV with CMake as follows:
cmake -S . -B build_windows_x64 -A x64 -DCURL_USE_LIBPSL=OFF -DCURL_USE_SCHANNEL=ON ^ -DCMAKE_INSTALL_PREFIX=install_win cmake --build build_windows_x64 --config Debug -j cmake --build build_windows_x64 --config Release -j
After setting
CURL_ROOT
in the environment to allow CMake to locate the libcurl installation, I then ran my CMake build for the other project, and noticed thatfind_package(CURL 7.68 REQUIRED COMPONENTS HTTPS)
would get me this:Digging into the CMake
FindCURL
module I noticed that if config mode search fails (which it is failing),curl-config
would be then used to determine the available features and protocols. Unfortunately,curl-config
is a shell script, and thus unusable natively on Windows. I figured, however, that it should be possible to writecurl-config
as a (portable!) C program, in which casecurl-config
can be used on both Windows or POSIX-like systems equally.PR summary
This PR therefore adds a
curl-config.c.in
file that will be run through CMake'sconfigure_file
using the same variables that are used to configurecurl-config.in
, although some variables that use relative variable expansions (e.g.execprefix
,libdir
, andincludedir
) are supplanted with full-path-expandedlit_
-prefixed variables. For Windows, we also normalize theLIBCURL_PC_LIBS_PRIVATE
list of libraries into Windows-format CMake listLIBCURL_WIN32_PC_LIBS_PRIVATE
.The
curl-config.c.in
gets converted intocurl-config.c
byconfigure_file
, with all the@
patterns fully expanded into absolute paths, and is then compiled and installable withcmake --install
in place of thecurl-config
shell script.To not break existing behavior, I have kept the
curl-config
behavior identical where possible, only hardening--checkfor
to report if input is invalid, i.e. not inmajor[.minor[.patch[-info]]]
format, or if the expected argument is missing. However, the help output will report thatcurl-config
was configured using a particular CMake version.Testing results
Currently, using a local 8.11.2-DEV installation of libcurl built from this branch, the
find_package
call is working correctly, and CMake is able to correctly invoke thecurl-config
C program to determine what components are available. In particular, the following outputs seem to be "as expected" on Windows (build + install same as in the motivation section):Addendum
Since this my first PR I would appreciate any suggestions for improvements. I wasn't able to find any other reports on using
find_package(CURL)
on Windows when I searched the mailing list archive so I read the contribution guidelines and decided to put out this PR as in my mind, it would be better ifcurl-config
could be portable to Windows too, In the past I've successfully built libcurl with zlib support and Schannel as the TLS implementation and used it in Visual Studio projects (via hardcoding include/library paths and names), so I was a bit surprised to find CMake choking.