GHA/http3-linux: add CI reproducer for --with-ngtcp2=<path> regression#20926
GHA/http3-linux: add CI reproducer for --with-ngtcp2=<path> regression#20926vszakats wants to merge 11 commits intocurl:masterfrom
--with-ngtcp2=<path> regression#20926Conversation
--with-ngtcp2= regression--with-ngtcp2= regression
--with-ngtcp2= regression--with-ngtcp2= regression
e438d85 to
91d7cfa
Compare
|
BoringSSL job doesn't like |
8277ff3 to
a5bd52a
Compare
|
After a To make this even more confusing AWS-LC does provide an openssl.pc, and |
|
after: pkgconf, fail: https://github.com/curl/curl/actions/runs/23121500311/job/67156293555?pr=20926#step:18:328 pkgconf, fail: https://github.com/curl/curl/actions/runs/23121500329/job/67156283797?pr=20926#step:38:302 pkgconf, success: https://github.com/curl/curl/actions/runs/23121500329/job/67156283797?pr=20926#step:38:251 pkg-config, fail: https://github.com/curl/curl/actions/runs/23121500323/job/67156272080?pr=20926#step:21:347 pkg-config, success: https://github.com/curl/curl/actions/runs/23121500323/job/67156272080?pr=20926#step:21:291 |
To aid debugging cases when dependency detection acts unexpectedly. Sprung from spending days trying to figure out behavior of ngtcp2 crypto modules and their dependencies. You can enable via by setting env `CURL_TRACE_PKG_CONFIG` to a non-empty value. When enabled detection details are logged for both successful and unsuccessful detections. Logging of unsuccessful ones is automatically enabled when `CURL_CI` env is set, which is the case for all CI jobs. It works by asking for `--debug` output and grepping for lines that seem useful for this specific purpose. Output is different for classic pkg-config and pkgconf, and may depending on tool version. If there is a built-in feature like this in classic pkg-config or pkgconf, I could not find it. Also: - GHA/http3-linux: set `CURL_TRACE_PKG_CONFIG` to log detection details. H3 builds are a recurring hot-spot for hard-to-debug issues. Ref: curl#20920 Follow-up to 3c64ffa curl#18415 curl#18188 Follow-up to 9950066 curl#18028 curl#18022 Cherry-picked from curl#20926 Closes curl#20926
To aid debugging cases when dependency detection acts unexpectedly. Sprung from spending days trying to figure out behavior of ngtcp2 crypto modules and their dependencies. You can enable by setting env `CURL_TRACE_PKG_CONFIG` to a non-empty value. When enabled, details are logged for both successful and unsuccessful detections. Logging of unsuccessful ones is automatically enabled when `CURL_CI` env is set, which is the case for all CI jobs. It works by asking for `--debug` output and grepping for lines that seem useful for this purpose. Output is different for classic pkg-config and pkgconf, and may depending on tool version. Also append `--print-errors` output if any. Examples (with pkgconf): Fail, before: ``` checking for libngtcp2_crypto_boringssl options with pkg-config... no configure: error: --with-ngtcp2 was specified but could not find ngtcp2_crypto_boringssl pkg-config file. ``` Fail, after: ``` checking for libngtcp2_crypto_boringssl options with pkg-config... no configure: pkg-config --exists libngtcp2_crypto_boringssl trace: ---- begin trying path: /home/runner/nghttp3/build/lib/pkgconfig for libngtcp2_crypto_boringssl trying path: /home/runner/ngtcp2-boringssl/build/lib/pkgconfig for libngtcp2_crypto_boringssl trying path: /home/runner/nghttp3/build/lib/pkgconfig for libngtcp2 trying path: /home/runner/ngtcp2-boringssl/build/lib/pkgconfig for libngtcp2 trying path: /home/runner/nghttp3/build/lib/pkgconfig for openssl trying path: /home/runner/ngtcp2-boringssl/build/lib/pkgconfig for openssl trying path: /home/runner/nghttp2/build/lib/pkgconfig for openssl ==== error: Package openssl was not found in the pkg-config search path. Perhaps you should add the directory containing `openssl.pc' to the PKG_CONFIG_PATH environment variable Package 'openssl', required by 'libngtcp2_crypto_boringssl', not found ---- end configure: error: --with-ngtcp2 was specified but could not find ngtcp2_crypto_boringssl pkg-config file. ``` Success, after: ``` checking for libngtcp2_crypto_boringssl options with pkg-config... found configure: pkg-config --exists libngtcp2_crypto_boringssl trace: ---- begin trying path: /home/runner/awslc/build/lib/pkgconfig for libngtcp2_crypto_boringssl trying path: /home/runner/nghttp3/build/lib/pkgconfig for libngtcp2_crypto_boringssl trying path: /home/runner/nghttp2/build/lib/pkgconfig for libngtcp2_crypto_boringssl trying path: /home/runner/ngtcp2/build/lib/pkgconfig for libngtcp2_crypto_boringssl trying path: /home/runner/awslc/build/lib/pkgconfig for libngtcp2 trying path: /home/runner/nghttp3/build/lib/pkgconfig for libngtcp2 trying path: /home/runner/nghttp2/build/lib/pkgconfig for libngtcp2 trying path: /home/runner/ngtcp2/build/lib/pkgconfig for libngtcp2 trying path: /home/runner/awslc/build/lib/pkgconfig for openssl trying path: /home/runner/awslc/build/lib/pkgconfig for libssl trying path: /home/runner/awslc/build/lib/pkgconfig for libcrypto ---- end ``` More examples: #20926 (comment) If there is an externally enablable, built-in feature like this in classic pkg-config or pkgconf, I could not find it. Also: - GHA/http3-linux: set `CURL_TRACE_PKG_CONFIG` to log detection details. H3 builds are prone to hard-to-debug dependency issues. Ref: #20920 Follow-up to 3c64ffa #18415 #18188 Follow-up to 9950066 #18028 #18022 Cherry-picked from #20926 Closes #20931
BoringSSL does not provide pc files, as of v0.20260211.0 (and its latest main branch.) It also did not provide them in the past. Its sibling fork, AWS-LC does provide them since v1.18.0 (2023-12-04): aws/aws-lc@7e6aef8 aws/aws-lc#1310 Introduce internal variable `OPENSSL_IS_AWSLC` to make this possible. Bug: #20926 (comment) Ref: ngtcp2/ngtcp2#1689 (comment) Closes #20938
This reverts commit ec60fe4.
d45876e to
71e98e1
Compare
--with-ngtcp2= regression--with-ngtcp2=<path> regression
- document building curl with CMake. - mention all supported forks in the OpenSSL section. Delete dedicated quictls section. - add TLS-backend pkgconfig dir to `PKG_CONFIG_PATH` for correctness. OpenSSL-based ones often work without this, by finding system `openssl.pc`. For GnuTLS and wolfSSL this has a lesser chance. Best to point to them explicitly. (configure may technically be able to do this automatically, but it isn't implemented.) - use `--with-ngtcp2=<path>` again, where possible. GnuTLS is the exception, pending fix in #20910. Same for BoringSSL, but not documented in `HTTP3.md`. - replace `<somewhereN>` with `/path/to/depname` for clarity. - move `LDFLAGS` after `./configure` for curl, to match dep builds. - move `--with-ngtcp2` next to the TLS-backend option. Follow-up to 1e3c2be #20926 Follow-up to 3c64ffa #18415 #18188 Cherry-picked from #20920 Closes #20947
Configure LibreSSL autotools job with
--with-ngtcp=<path>instead ofadding ngtcp2 to
PKG_CONFIG_PATH. To test this way of configuration inCI and test for the regression reported in #20889.
Turns out this way of configuration isn't affected by the detection
issue in this particular case.
It also works for other backends except for these two, subject to
separate fixes:
--with-ngtcp2=<path>option for crypto libs #20920Follow-up to 666db80 #20891
Follow-up to 8db0e28 #18189
Follow-up to 9950066 #18028 #18022
H3 Linux: https://github.com/curl/curl/actions/runs/23121500311/job/67156293555?pr=20926#step:18:330 (fail, pkgconf)
Linux: https://github.com/curl/curl/actions/runs/23121500329/job/67156283850?pr=20926#step:39:232 (slackware is pkg-config)
Windows: https://github.com/curl/curl/actions/runs/23120741088/job/67154236531?pr=20926
Old Linux: https://github.com/curl/curl/actions/runs/23121500323/job/67156272080?pr=20926#step:21:224 (pkg-config)
non-native: https://github.com/curl/curl/actions/runs/23120741100/job/67154216142?pr=20926
H3 Linux: https://github.com/curl/curl/actions/runs/23120741073/job/67154235113?pr=20926
Linux: https://github.com/curl/curl/actions/runs/23121129085/job/67155256452?pr=20926 (pkg-config all lines)
Old Linux: https://github.com/curl/curl/actions/runs/23121129078/job/67155255953?pr=20926#step:21:203 (pkg-config all lines)