[CI:BUILD] Packit: Enable Copr builds on PR and commit to main

[lsm5]

This commit adds Packit configuration files which will trigger rpm builds on copr:rhcontainerbot/packit-builds on every PR as well as on copr:rhcontainerbot/podman-next on every commit to main branch.

This commit will ensure main branch is always buildable on all supported Fedora and CentOS Stream versions for aarch64 and x86_64.
TODO: enable build checks for s390x and ppc64le while ensuring they don't take too long to build.

The packit builds reuse buildah.spec.rpkg and are thus independent of Fedora / CentOS dist-git.

This change will remove the need for the current webhook based triggering
of rpm builds on rhcontainerbot/podman-next after commit to main.
That will be instead handled by the trigger: commit action added in this
PR. New builds will continue to get posted to the same link so users
don't need to change any existing copr repo configuration.

What type of PR is this?

/kind other

What this PR does / why we need it:

Checks for downstream build issues prior to merge.

How to verify it

There will be additional packit tasks in CI.

Which issue(s) this PR fixes:

None

Special notes for your reviewer:

None

Does this PR introduce a user-facing change?

None

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lsm5

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [lsm5]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[edsantiago]

Mostly I just diffed against the aardvark-dns packit files. A few questions and nits.

[edsantiago]

Is buildah.spec generated from buildah.spec.rpkg in this directory? If so, it does not seem to include either %setup nor %autosetup...?

[lsm5]

@edsantiago try rpkg spec --outdir ./ . That will leave you with a buildah.spec that has:

%prep
%setup -T -b 0 -q -n buildah

[lsm5]

I can add a comment about rpkg spec ... in .packit.sh if that helps.

[edsantiago]

I don't speak packit, so maybe this is correct, maybe not. Just want to point out that it says "podman" in a buildah PR.

[lsm5]

So, that specifies the copr project where the build should be triggered after the commit is added to main branch upstream. It inherits most values from the block above it, but since the builds after merge should be in rhcontainerbot/podman-next, hence the project: podman-next.

[lsm5]

Btw, this change will remove the need for the current webhook based triggering of rpm builds on rhcontainerbot/podman-next after commit to main (https://copr.fedorainfracloud.org/coprs/rhcontainerbot/podman-next/package/buildah/) . That will be instead handled by the trigger: commit action added in this PR. New builds will continue to get posted to the same link so users don't need to change any existing copr repo configuration.

[lsm5]

Btw, this change will remove the need for the current webhook based triggering of rpm builds on rhcontainerbot/podman-next after commit to main (https://copr.fedorainfracloud.org/coprs/rhcontainerbot/podman-next/package/buildah/) . That will be instead handled by the trigger: commit action added in this PR. New builds will continue to get posted to the same link so users don't need to change any existing copr repo configuration.

I'll add this in the commit message..

[edsantiago]

Thanks for the explanations. Most of it is above my head, because I don't use these COPRs or repos or anything like this, but LGTM

[lsm5]

Thanks for the explanations. Most of it is above my head, because I don't use these COPRs or repos or anything like this, but LGTM

Alrighty :) . Thank you!

[edsantiago]

Hold on a second - I see four pack-it jobs running in CI right now. Shouldn't these run only when the PR merges? (Again, I don't grok packit, so maybe this is desired, but this looks like a lot of churn for very little benefit)

[TomSweeneyRedHat]

I too am not a packit person. From what I can see, LGTM

[edsantiago]

Before this merges, I'd like @containers/buildah-maintainers to confirm that they're OK with having these jobs run on every PR, for many many hours. (I have no timing data, because all packit jobs show "completed in 0s", but I know that I looked at this PR page several times today, and packit jobs were spinning for at least two hours, probably longer).

[lsm5]

Hold on a second - I see four pack-it jobs running in CI right now. Shouldn't these run only when the PR merges? (Again, I don't grok packit, so maybe this is desired, but this looks like a lot of churn for very little benefit)

In .packit.yaml, there's a trigger: pull_request which will trigger copr builds on rhcontainerbot/packit-builds copr, which is what you see in the gating task list on the PR.

After the PR is merged (IOW, commit to main branch), the trigger: commit part in .packit.yaml will trigger builds on rhcontainerbot/podman-next copr.

So, different sets of tasks for different coprs, one to check if every PR builds, and the other for users to install after upstream merge.

RE: benefit, I'd rather build issues get caught in PR than after upstream tag release. Also, this doesn't depend on the official distro spec file anymore and the config file is a lot simpler too with a lot of the configuration handled in rhcontainerbot/packit-builds itself.

RE: completed in 0s, I can check with the packit devs.

[lsm5]

@edsantiago I filed packit/packit-service#1970 for succeeded in 0s issue.

[rhatdan]

@lsm5 do you expect this PR to add any time to the build, or do you expect the new tests to pass within the ~1 hour that tests currently take to run?

[lsm5]

@lsm5 do you expect this PR to add any time to the build, or do you expect the new tests to pass within the ~1 hour that tests currently take to run?

Right now, s390x seems to be taking the longest and sometimes also ppc64le. I can remove those for now (from the copr itself which would remove those for the other repos too) and limit all our packit jobs to x86_64 and aarch64.

[lsm5]

Disabled s390x and ppc64le in the copr. Let's give this another try.

/packit copr-build

[rhatdan]

SGTM, since those are most likely to be down or busy. And I think your tests on x86 will catch the bulk of errors.
/lgtm
/hold

[lsm5]

Here's the latest copr_build that got triggered https://copr.fedorainfracloud.org/coprs/rhcontainerbot/packit-builds/build/5698195/

[lsm5]

let me update the commit message again to show we are only doing x86_64 and aarch64 for now.

[rhatdan]

/lgtm

[lsm5]

https://copr.fedorainfracloud.org/coprs/rhcontainerbot/packit-builds/build/5698195/ .. got done in 8 mins :)

[lsm5]

@edsantiago copr build times should be drastically reduced now and the 0s issue is on packit's radar. Please let me know if you still got any other concerns. I'll remove the slash-hold only after you green light this :)

[edsantiago]

@lsm5 it sgtm in theory, I just really know nothing about packit and am uncomfortable with reliance on something that can add hours to a CI run. (Thank you for addressing the rare-arches slowness, it's just that I have zero sense for the packit farm and what other unreliabilities it may have). But, in the spirit of optimism:

/hold cancel

[lsm5]

@lsm5 it sgtm in theory, I just really know nothing about packit and am uncomfortable with reliance on something that can add hours to a CI run. (Thank you for addressing the rare-arches slowness, it's just that I have zero sense for the packit farm and what other unreliabilities it may have). But, in the spirit of optimism:

/hold cancel

Ack, gotcha. Please let me know the moment you see it causing any further annoyances. I've filed a bunch of RFEs with the packit team, so things should only get better :) .