Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmake: add pre-fill for Unix, enable in GHA/macos, verify pre-fills #15841

Closed
wants to merge 54 commits into from

Conversation

vszakats
Copy link
Member

@vszakats vszakats commented Dec 27, 2024

TL;DR: Save 10 minutes of CI time for GHA/macos jobs using pre-fills and
add pre-fill verification for Apple and Windows. Also restores Xcode job
and saves 1.5-10 minutes configuring iOS jobs.

Pre-filling feature detection results can bring down the CMake configure
step to ~5 seconds on most GHA runners, ~10 seconds in slow envs like
Cygwin/MSYS2.

The potential savings per job are:

  • 5-40 (average 19) seconds on GHA/macos (33 jobs)
  • ~10 seconds on GHA for iOS GNU Makefile (1 job)
  • 1.5-10 minutes on GHA for iOS Xcode generator (1 job)
  • 10 seconds on GHA/linux with native Ubuntu (12 jobs)
  • 40 seconds for Cygwin/MSYS2 (2 jobs)
  • 5-10 seconds for virtualized BSDs, native CPU (3 jobs)
  • ~60 seconds for virtualized BSDs, emulated CPU (1 job)

On native Windows pre-filling has been in place for a long time and
saving 8 minutes (VS2019-VS2015) to 1.5-2 minutes (VS2022), 3 minutes
(VS2022 UWP), and 30-60 seconds (MinGW), per CI job.

The downside is that detection results need to be manually collected and
filtered to those that universally apply to all platforms that they are
enabled on. Another downside is that by using a cache, we're not running
the actual detections, and thus won't catch regressions in them. It
means we must make sure that the cache is solid and matches with actual
detections results. An upside is that it gives a rough overview of which
features are available on which platforms. Another upside is pre-filled
values do work for feature detections skipped for cross-builds, e.g.
HAVE_WRITABLE_ARGV.

This PR adds a pre-fill cache that supports all Unixes (except OmniOS)
used in CI, and makes it usable with an internal option. It also enables
it for GHA/macos CI jobs, where the maximum savings are. And also for
the two iOS [1] and two Cygwin/MSYS2 jobs. The latters don't have
pre-fill checks and we can drop them if they turn into a hassle.

Saving:

  • 10 minutes of CI time per GHA/macos workflow run. [2]
  • ~80 seconds per GHA/windows workflow run with Cygwin/MSYS2.
    (offsetting the cost of pre-fill verifications)
  • 1.5-10 minutes per GHA/non-native runs with iOS jobs. [3]

You can enable pre-fill locally with -D_CURL_PREFILL=ON. It's
experimental, and if you experience a problem, file a PR or an Issue.

This PR also adds a pre-fill checker for macOS and MinGW/MSVC Windows
GHA jobs to catch if the cache diverges from real detections. It also
adds this logic to AppVeyor, but doesn't enable it due to the perf
penalty of 2 minutes mininum.

The pre-fill checker works by configuring out-of-tree with and without
pre-fill, then diffing their lib/curl_config.h outputs.

Exceptions are 3 detection results exposed indirectly [4], and missing
to expose 2, of which one is the C89 header stddef.h. While we assume
the C99 stdint.h available outside iOS. We can expose them in the
future, if necessary.

The pre-fill checks cost in total:

  • ~20 seconds for macOS
  • ~40 seconds for MinGW on GHA
  • ~80 seconds for MSVC on GHA (UWP would be 2x this)

An extra time saving potential is caching type sizes. They are
well-known, and seldom change, esp. in CI. GHA/Windows jobs spend 8-17
seconds per job on these ~12 feature checks. ~5s on Cygwin/MSYS2. Couple
of seconds on other platforms. (This PR doesn't make this optimization.)

Another opportunity is doing the same for autotools, which typically
spends more time in the configuration step than cmake.

[1] Xcode job restored as a
follow-up to be5f202 #16302

[2] GHA/macos cmake configure times in seconds:

Job Bef. After Gain
CM clang GnuTLS !ldap krb5 21.2 4.5 16.7
CM clang LibreSSL !ldap heimdal c-ares +examples 13.3 3.9 9.4
CM clang OpenSSL +static libssh +examples 20.0 4.6 15.4
CM clang OpenSSL IDN clang-tidy~ (w/chkprefill) 15.7 18.6 -2.9
CM clang OpenSSL gsasl rtmp AppleIDN 25.0 4.7 20.3
CM clang OpenSSL torture !FTP 15.3 4.5 10.8
CM clang OpenSSL torture FTP 25.0 5.9 19.1
CM clang SecureTransport debug 18.0 3.8 14.2
CM clang macos-13 SecureTransport 45.8 12.4 33.4
CM clang macos-14 SecureTransport 15.8 4.6 11.2
CM clang macos-15 SecureTransport 26.8 6.1 20.7
CM clang mbedTLS openldap brotli zstd 15.1 6.5 8.6
CM clang wolfSSL !ldap brotli zstd 27.0 4.4 22.6
CM gcc-12 GnuTLS !ldap krb5 39.1 8.7 30.4
CM gcc-12 LibreSSL !ldap heimdal c-ares +examples 23.8 7.2 16.6
CM gcc-12 OpenSSL +static libssh +examples 20.7 8.5 12.2
CM gcc-12 OpenSSL gsasl rtmp AppleIDN 23.1 10.1 13.0
CM gcc-12 SecureTransport debug 21.1 4.8 16.3
CM gcc-12 mbedTLS openldap brotli zstd 21.4 5.8 15.6
CM gcc-12 wolfSSL !ldap brotli zstd 21.1 6.9 14.2
CM gcc-14 macos-13 SecureTransport 61.9 18.7 43.2
CM gcc-14 macos-14 SecureTransport 30.5 6.4 24.1
CM gcc-14 macos-15 SecureTransport 32.7 8.4 24.3
CM llvm@15 GnuTLS !ldap krb5 21.1 7.5 13.6
CM llvm@15 LibreSSL !ldap heimdal c-ares +exampl~ 24.6 6.8 17.8
CM llvm@15 OpenSSL +static libssh +examples 19.0 6.4 12.6
CM llvm@15 OpenSSL gsasl rtmp AppleIDN 19.0 8.2 10.8
CM llvm@15 SecureTransport debug 18.0 5.4 12.6
CM llvm@15 macos-13 SecureTransport 66.2 25.7 40.5
CM llvm@15 macos-14 SecureTransport 31.9 6.1 25.8
CM llvm@15 mbedTLS openldap brotli zstd 19.5 8.9 10.6
CM llvm@15 wolfSSL !ldap brotli zstd 24.3 5.9 18.4
CM llvm@18 macos-15 SecureTransport 33.8 6.4 27.4
Total 856.8 257.3 599.5
Before: https://github.com/curl/curl/actions/runs/13311042735/job/37173478424
After: https://github.com/curl/curl/actions/runs/13313927119/job/37183206426?pr=15841

[3] iOS:
Before: https://github.com/curl/curl/actions/runs/13326401704?pr=15841
After: https://github.com/curl/curl/actions/runs/13332177764?pr=15841

[4] detection results exposed indirectly in curl_config.h:

  • HAVE_FILE_OFFSET_BITS via _FILE_OFFSET_BITS
  • HAVE_GETHOSTBYNAME_R_*_REENTRANT via NEED_REENTRANT
  • HAVE_SOCKADDR_IN6_SIN6_ADDR via USE_IPV6

w/o ws: https://github.com/curl/curl/pull/15841/files?w=1

  • add a CI check comparing the macOS cached values with actual detection results (possibly to configure-vs-cmake).
  • leave a real macOS job as a canary that doesn't use caching. [SKIP, comparison is enough]
  • omit caching if any of the CMake sources changes. [SKIP]
  • keep caching only on macOS, where the vast majority of time saving is. leave the rest disable by default in CI. (but keep the logic, because it can still be beneficial locally)
  • consider uploading caches for MS-DOS/AmigaOS. [FUTURE PR maybe, useful for local development, but it's probably not of wide interest.]
  • try detecting "old" Linux, via glibc version, automatically? [FUTURE PR if this looks interesting, or perhaps affects more than a single detection result]
  • add script to generate a cache from CI log output (or CMakeCache.txt). [SKIP, the pre-fill checker output should be enough.]
  • enable pre-fill checker in MinGW and MSVC GHA/windows jobs.

@vszakats vszakats added cmake performance CI Continuous Integration appleOS specific to an Apple operating system labels Dec 27, 2024
@vszakats vszakats marked this pull request as draft December 27, 2024 23:28
@vszakats vszakats changed the title GHA/macos: cache cmake detection results GHA/macos: preload cmake detection results Dec 28, 2024
@vszakats vszakats force-pushed the gha-mac-cm-conf-cache branch 4 times, most recently from d226ce9 to b850285 Compare December 29, 2024 00:15
@testclutch

This comment was marked as outdated.

@vszakats vszakats force-pushed the gha-mac-cm-conf-cache branch 8 times, most recently from e87f86e to 3468cd0 Compare December 31, 2024 13:26
@vszakats vszakats force-pushed the gha-mac-cm-conf-cache branch 4 times, most recently from 823ac56 to 64c70a7 Compare January 23, 2025 00:51
@vszakats vszakats force-pushed the gha-mac-cm-conf-cache branch 4 times, most recently from 7826f74 to b97db64 Compare February 12, 2025 17:14
@vszakats vszakats changed the title GHA/macos: preload cmake detection results GHA/macos: pre-fill cmake detection results Feb 12, 2025
@vszakats vszakats force-pushed the gha-mac-cm-conf-cache branch 2 times, most recently from 62a6057 to bd577bb Compare February 13, 2025 14:23
@vszakats vszakats force-pushed the gha-mac-cm-conf-cache branch from 2a5d65d to 805d23c Compare February 15, 2025 23:15
@vszakats vszakats closed this in e7adf3e Feb 16, 2025
@vszakats vszakats deleted the gha-mac-cm-conf-cache branch February 16, 2025 01:02
vszakats added a commit to vszakats/curl that referenced this pull request Feb 17, 2025
To help debugging builds where the actual feature check is broken.

Follow-up to e7adf3e curl#15841
vszakats added a commit that referenced this pull request Feb 17, 2025
To help debugging builds where the actual feature check is broken.

Follow-up to e7adf3e #15841
Closes #16369
vszakats added a commit that referenced this pull request Feb 24, 2025
- appveyor: restore VS2008 job, after fixing its issues.
  Enable OpenSSL in it. It takes 1 minute.
  Follow-up to 9b0467b #16453
  Follow-up to edfa537 #16456
- appveyor: make a copy of OpenSSL DLLs to have them picked up as an
  artifact (disabled by default) to aid local tests.
- appveyor: dump CMake configuration logs on failure.
- appveyor: tidy up job parameter defaults.
- GHA/windows: add pre-fill check option for dl-mingw jobs.
- GHA/windows: fix pre-fill check option for MSYS jobs by installing
  `diffutils`.
  Follow-up to e7adf3e #15841
- GHA/windows: de-duplicate to `PATH` commands for Cygwin.
- GHA/windows: drop `$SYSTEMROOT/System32` from `PATH` for Cygwin
  configure. It's not needed.
  Follow-up to 36fd2dd #13599
- list `.pdb` files in curl version step for MSVC.
  Ref: #16439

Cherry-picked from #16394
Closes #16458
icing pushed a commit to icing/curl that referenced this pull request Feb 25, 2025
- appveyor: restore VS2008 job, after fixing its issues.
  Enable OpenSSL in it. It takes 1 minute.
  Follow-up to 9b0467b curl#16453
  Follow-up to edfa537 curl#16456
- appveyor: make a copy of OpenSSL DLLs to have them picked up as an
  artifact (disabled by default) to aid local tests.
- appveyor: dump CMake configuration logs on failure.
- appveyor: tidy up job parameter defaults.
- GHA/windows: add pre-fill check option for dl-mingw jobs.
- GHA/windows: fix pre-fill check option for MSYS jobs by installing
  `diffutils`.
  Follow-up to e7adf3e curl#15841
- GHA/windows: de-duplicate to `PATH` commands for Cygwin.
- GHA/windows: drop `$SYSTEMROOT/System32` from `PATH` for Cygwin
  configure. It's not needed.
  Follow-up to 36fd2dd curl#13599
- list `.pdb` files in curl version step for MSVC.
  Ref: curl#16439

Cherry-picked from curl#16394
Closes curl#16458
vszakats added a commit that referenced this pull request Mar 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
appleOS specific to an Apple operating system CI Continuous Integration cmake performance Windows Windows-specific
Development

Successfully merging this pull request may close these issues.

2 participants