ci: Future of macOS and Windows MSVC CI tasks

[maflcko]

Cirrus CI will cap the community cluster, see https://cirrus-ci.org/blog/2023/07/17/limiting-free-usage-of-cirrus-ci/ .

Thus, someone would have to sponsor an amount of roughly 5kUSD/mo for those two tasks. Not sure if this is worth it, given that appveyor CI used to cost 50 USD/mo. Also, I am not sure if a sponsor can be found. Personally, I don't use macOS nor Windows, and I can't recommend it to anyone, so I don't mind if the tasks are removed. Though, I am opening this issue to gather feedback and alternatives.

cc @jonasnick @real-or-random (You'll probably have to do something about this as well)

The alternatives I can see (someone would have to implement them):

• Someone sets up Windows+macOS VMs and connects them to the bitcoin and bitcoin-core Cirrus CI account as a persistent worker. (I've only tested this with Linux, so no idea if it works)
• Go back to appveyor, which was 50 USD/mo. (See removal commit e7c6ed6)
• Use GitHub Actions CI?
• Something else?

[real-or-random]

Why are Linux tasks not affected? As far as I understand the blog post, there will be a monthly limit of 40 compute credits (= 40 USD) for all tasks, including Linux tasks. (Or are we already using a persistent worker for Linux?)

[maflcko]

We already use credits for some Linux tasks, so nothing will change for them. The others (or all) can be moved to persistent workers easily, I suspect.

Edited OP to mention GitHub Actions CI as alternative. Last time we looked it required write access, but GitHub already has full write access, so I am not sure if this is still a concern or if it even applies?

[real-or-random]

Someone sets up Windows+macOS VMs and connects them to the bitcoin and bitcoin-core Cirrus CI account as a persistent worker. (I've only tested this with Linux, so no idea if it works)

If we can make this happen, this will probably be ideal in the short term. (Heck, even if we convince a sponsor to hire someone, this is much cheaper than paying for compute credits...)

Last time we looked it required write access,

See the discussion here: #17803 You had brought up strong arguments against it, but it may indeed be worth reconsidering. As you point out, we trust GitHub anyway. (Or we don't trust them anyway, in the sense that it's not the end of the world if they become hostile.) And GitHub Actions is hopefully more mature now, so I don't think the risk of vulnerabilities should be a showstopper.

---

In secp256k1, we currently rely on Cirrus CI's "Dockerfile as a CI environment" feature for the Linux tasks. Moving away from it could require a bit of work. GitHub Actions apparently has a similar feature, but I haven't tried that one.

[maflcko]

Maybe Azure Pipelines can be used as an alternative to GitHub CI? Likely they use the same virtual machine pools, but the permissions and write access might be better handled?
Also, apparently all of Circle CI, Appveyor CI, and Travis CI now support macOS+Windows, so maybe they can be considered as alternative as well?

Cirrus CI's "Dockerfile as a CI env

Same here. But Linux builds should be trivial to keep on Cirrus CI with persistent workers, which can run podman/docker themselves to cache the result.
Ideally we figure out what to do with macOS/Windows within the next 30 days, then dealing with Linux should be trivial.

[hebasto]

Among free options, CircleCI looks better than others, IMO:

• https://circleci.com/docs/plan-free/
• https://circleci.com/pricing/

Personally, I don't use macOS nor Windows, and I can't recommend it to anyone, so I don't mind if the tasks are removed.

We are responsible for quality of every release we are shipping. And the user is free to make their own choice.

[maflcko]

CircleCI looks better than others

Can you elaborate on this? The free plan would give less than 10 hours of macOS and Windows per month. And the paid plan doesn't look cheaper than Cirrus CI, does it?

[hebasto]

CircleCI looks better than others

Can you elaborate on this? The free plan would give less than 10 hours of macOS and Windows per month. And the paid plan doesn't look cheaper than Cirrus CI, does it?

I've made comparison among available free plans. I did not consider paid plans in my comment.

[maflcko]

If the goal is to stay on a free plan, I think the only option is GitHub Actions CI.
If the goal is to pay a similar amount as with the previous appveyor task, I think the only options are appveyor or travis.

[hebasto]

If the goal is to stay on a free plan, I think the only option is GitHub Actions CI.

GitHub Actions: 2000 minutes/month

CircleCI: 6000 minutes/month

[hebasto]

Another consideration. AppVeyor still has:

60 minutes quota per build job.

That makes running functional tests questionable.

[real-or-random]

GitHub Actions: 2000 minutes/month

AFAIU public repos are free (no limit) for public repos: https://docs.github.com/en/billing/managing-billing-for-github-actions/about-billing-for-github-actions

[hebasto]

If the goal is to stay on a free plan, I think the only option is GitHub Actions CI.

I've tried to port our MSVC build from Cirrus CI to GitHub Actions CI and faced an issue: actions/runner-images#7832. A suggested workaround did not help. I'm going to investigate it shortly.

UPD. A workaround has been found.