AL2022 ARM64 packages not available after 2.0.4

[danybmx]

Bug Report

Describe the bug
There are no packages available for Amazon Linux 2022 arm64 in the repository after version 2.0.4

To Reproduce
Listing packages on the packages.fluentbit.com we can't see the fluent-bit 2.0.5, 2.0.6, 2.0.7, 2.0.8 and 2.0.9 for the arm64/aarch64.
https://packages.fluentbit.io/amazonlinux/2022/

Expected behavior
The packages should be there as for other distributions.

Screenshots
N/A

Your Environment
N/A

Additional context
We want to be able to install latest versions to avoid security issues and include bug fixes.

[patrick-stephens]

Yeah we stopped building them as there is some issue with the underlying base image - builds were timing out after ridiculous build times.

See #6300 and it seems to be a common issue for other OSS projects. Since then I have not had a chance to work on investigating why so any help would be appreciated.

[alexellis]

@patrick-stephens asked me to drop a link to what we're working on, it would solve this problem:

Two case-studies:

https://actuated.dev/blog/case-study-bring-your-own-bare-metal-to-actions
https://actuated.dev/blog/native-arm64-for-github-actions

[patrick-stephens]

Thanks @alexellis let's see if we can get this going as a PR POC to verify it is doable and we can then merge.

[alexellis]

I've emailed you instructions for next steps.

[patrick-stephens]

I will aim to integrate and test this in the next few weeks if I can.

[karolpiatek-sidero]

@patrick-stephens we were able to build directly on Amazon Linux 2023, EC2 host, Graviton 3, packages for Amazon Linux 2 and Amazon Linux 2023 (AL2023) in the reasonable time,
under 10 minutes, which aligns with times of X86.

Also AL2023 packages are served using S3 buckets internal by Amazon,
same goes for official Ubuntu images, which speeds up build as well.

Same goes for CentOS 7 build times improvements,
which slashes time to 5 minutes or less, on ARM64.

Additionally we used optimized tool for container builds,
which reduces image sizes for CentOS 7, AL2.

[patrick-stephens]

Sure, unfortunately self hosted runners are not recommended for public repositories. As you say it works fine, I'm just not sure why the speed drops so much for emulation to the point it's unusable. It used to be ok but something has obviously changed somewhere and only affecting the AL2023 build.

I do have a POC with actuated.dev that shows massive improvements though so hopefully we can integrate that soon. I want to ensure we can still build locally and on any infra too.

However, any PRs to improve are greatly appreciated so please contribute if you can.