Move most CI builds away from Cirrus

[dfandrich]

Cirrus is drastically reducing their free tier on Sept. 1, so they will no
longer perform all these builds for us. Most builds are moved other services in
this PR, with some caveats:

• Windows builds have not yet been replaced
• macOS ARM builds will be running on x86_64 instead of ARM for a few months

Doing this should preserve the ability to continue to run a FreeBSD build on
Cirrus without disruption. I'm not aware of another CI service offering FreeBSD
builds (for free), so keeping it going is important for test coverage.

[dfandrich]

The libressl build failure looks legit. There are dyld[NNN]: missing symbol called errors on the failing tests, which points to some kind of link problem. Is it linking against the wrong library at run-time? Is there a system LibreSSL library available on macOS that could be causing this confusion?

[icing]

macOS older versions used libressl, yes.

[bagder]

macOS older versions used libressl, yes.

Don't they still build their curl versions to use libressl?

But if this is a link problem, why does it only happen in a few tests?

[bagder]

Nice work!

[vszakats]

As of 12.x Monterey, macOS ships with LibreSSL and builds curl with it:

curl 7.87.0 (x86_64-apple-darwin21.0) libcurl/7.87.0 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.11 nghttp2/1.45.1
Release-Date: 2022-12-21
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL threadsafe UnixSockets

AFAIK this continues with Ventura.

[pheiduck]

Yep @vszakats

curl 8.1.2 (x86_64-apple-darwin22.0) libcurl/8.1.2 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.11 nghttp2/1.51.0
Release-Date: 2023-05-30
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL threadsafe UnixSockets

[dfandrich]

But if this is a link problem, why does it only happen in a few tests?

These errors are happening at run time, so I'm guessing those tests are the only ones that enter code paths that need those symbols. This would make sense if macOS is doing lazy symbol binding. The error message doesn't give the missing symbol name so it might not be a LibreSSL one, too; I recall a discussion a couple of months ago about accessing some functions on macOS using some kind of run-time determination.

As the comment in the config file notes, a previous run of this test failed even harder during compilation with --enable-debug on unit1303, so clearly there's some kind of odd linking issue a work here.

[dfandrich]

I wasn't able to reproduce this on a macOS box I have access to; LibreSSL from Brew worked just fine. I've just disabled the problematic tests until someone is able to figure it out.

[dfandrich]

Disabling the failing tests on the LibreSSL build worked, but looking at the version output in the banner it appears that the system isn't even using LibreSSL at run-time but rather trying to use OpenSSL (the version banner says OpenSSL/not available instead of LibreSSL/3.7.3). It compiles, links and runs fine when I try it on a macOS box manually. Since it looks like it's not even using LibreSSL, and who knows, possibly not even building with it, I'm going to drop that build altogether since it doesn't seem to be adding anything of value.

[MarcelRaad]

Looks like moving the macOS builds to CircleCI is unfortunate for forks. On every push to my fork, I'm now spammed with error e-mails from CirceCI because:

Resource class macos for macos.m1.medium.gen1, image xcode:15.0.0 is not available for your project, or is not a valid resource class. This message will often appear if the pricing plan for this project does not support macos use.

Do others also get that or am I doing something wrong?

[MarcelRaad]

Hmm, https://discuss.circleci.com/t/macos-intel-support-deprecation-in-january-2024/48718 says that "Xcode 14.2 is the last supported version for Gen1". How does this work here at all? 😕
Looks like Gen2 is used even though the config file explicitly specifies Gen1:
https://app.circleci.com/pipelines/github/curl/curl/6037/workflows/fd44dd35-d306-4c5e-9b31-1661dfc4476b/jobs/39563

[dfandrich]

@vszakats if you're running into local build issues and not CI problems please open a new issue.

@MarcelRaad I haven't been getting spammed by CircleCI, but I also know that there is an issue with the resource class. Their ARM machines aren't available to users in the free tier yet so it's been silently (from my perspective, anyway) replacing that with x86_64 machines. I didn't bother changing that because it seemed harmless and the whole reason for moving builds there was for ARM support, but if it's causing issues I'll change that to reflect reality and hopefully stop the spam.

[vszakats]

@dfandrich: I just run into this and felt relevant given that a similar issue was raised here that might be related. I'll link to this post from the commit I fixed this on the build side. It's not my local build but curl-for-win builds (now supporting macOS, Linux) that I'm at the moment testing locally. A little bit of goodwill would be appreciated.

[dfandrich]

I just didn't want to lose a valid issue in the weeds of another one.

[dfandrich]

I've switched the Circle macOS builds to an x86 class where those things should not be issues. Please say something if you still get those warning e-mails.

[MarcelRaad]

I can confirm that the builds on my fork run again, thanks Dan!