Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.github: Pin docker buildx version to v0.9.1 (v2) #23220

Merged
merged 2 commits into from
Jan 20, 2023

Conversation

joestringer
Copy link
Member

GitHub recently rolled out Docker buildx version v0.10.0 on their
builders, which transparently changed the MediaType of docker images to
OCI v1 and added provenance attestations.

Unfortunately, various tools we use in CI like SBOM tooling and docker
manifest inspect do not properly support some aspect of the new image
formats. This resulted in breaking CI, with some messages like this:

level=fatal msg="generating doc: creating SPDX document: generating
SPDX package from image ref quay.io/cilium/docker-plugin-ci:XXX:
generating image package"

This could also lead CI to fail while waiting for image builds to
complete, because the command we use to test whether the image is
available did not support the image types.

This commit attempts to revert buildx back to v0.9.1 to prevent it from
generating the images in a format that other tooling doesn't expect.
Over time we can work on migrating to buildx v0.10, testing various
parts of our CI as we do so.

This is a quick-and-dirty hack to stabilize CI for the short term, then
we can figure out over time how to properly resolve the conflict between
these systems.

Reverts #23206

This reverts commit 30b036d.

Chance pointed out on Slack that these changes were ineffective because
they were placed in the "with" statement for the wrong action. Revert
the original commit so we can then cleanly apply the change in the right
place.

Signed-off-by: Joe Stringer <joe@cilium.io>
GitHub recently rolled out Docker buildx version v0.10.0 on their
builders, which transparently changed the MediaType of docker images to
OCI v1 and added provenance attestations.

Unfortunately, various tools we use in CI like SBOM tooling and docker
manifest inspect do not properly support some aspect of the new image
formats. This resulted in breaking CI, with some messages like this:

    level=fatal msg="generating doc: creating SPDX document: generating
    SPDX package from image ref quay.io/cilium/docker-plugin-ci:XXX:
    generating image package"

This could also lead CI to fail while waiting for image builds to
complete, because the command we use to test whether the image is
available did not support the image types.

This commit attempts to revert buildx back to v0.9.1 to prevent it from
generating the images in a format that other tooling doesn't expect.
Over time we can work on migrating to buildx v0.10, testing various
parts of our CI as we do so.

This is a quick-and-dirty hack to stabilize CI for the short term, then
we can figure out over time how to properly resolve the conflict between
these systems.

Signed-off-by: Joe Stringer <joe@cilium.io>
@joestringer joestringer added needs-backport/1.11 needs-backport/1.13 This PR / issue needs backporting to the v1.13 branch release-note/misc This PR makes changes that have no direct user impact. labels Jan 20, 2023
@joestringer joestringer requested review from a team as code owners January 20, 2023 18:57
@maintainer-s-little-helper maintainer-s-little-helper bot added dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. and removed dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. labels Jan 20, 2023
@maintainer-s-little-helper maintainer-s-little-helper bot added this to Needs backport from master in 1.12.6 Jan 20, 2023
@maintainer-s-little-helper maintainer-s-little-helper bot added this to Needs backport from master in 1.11.13 Jan 20, 2023
Copy link
Contributor

@ldelossa ldelossa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

v3's the charm ;)

@joestringer
Copy link
Member Author

Thanks to @chancez for pointing out why the version shouldn't have worked on the previous PR, originally #23206 was trying to update the docker/build-push-action configuration for provenance (which didn't worK). When I switched to configuring version, I thought it was the same action. However, version is only configured on the docker/setup-buildx-action. So hopefully this actually pins the version this time.

Copy link
Contributor

@chancez chancez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep looks like the right spot!

Copy link
Member

@aanm aanm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we revert the SBOM revert PR here?

@joestringer
Copy link
Member Author

@aanm I was hoping to follow up on that separately once we confirmed that this is switching CI back to generating images in the older / supported format

@aanm aanm merged commit 9ab03d0 into master Jan 20, 2023
@aanm aanm deleted the pr/joe/buildx-provenance-v2 branch January 20, 2023 19:46
@joestringer
Copy link
Member Author

@aanm #23221

@joestringer joestringer added release-note/ci This PR makes changes to the CI. and removed release-note/misc This PR makes changes that have no direct user impact. labels Jan 20, 2023
@aanm aanm mentioned this pull request Jan 23, 2023
19 tasks
@aanm aanm added backport-pending/1.13 The backport for Cilium 1.13.x for this PR is in progress. and removed needs-backport/1.13 This PR / issue needs backporting to the v1.13 branch labels Jan 23, 2023
@aanm aanm added the backport-done/1.13 The backport for Cilium 1.13.x for this PR is done. label Jan 23, 2023
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Needs backport from master to Backport pending to v1.12 in 1.12.6 Jan 23, 2023
This was referenced Jan 23, 2023
@ldelossa ldelossa added backport-done/1.12 The backport for Cilium 1.12.x for this PR is done. and removed backport-pending/1.12 labels Jan 24, 2023
@tommyp1ckles
Copy link
Contributor

@joestringer FYI, you can also work around this by doing docker buildx imagetools inspect --raw quay.io/cilium/cilium-ci:10e55d6053762d098cce06cd97abd51afc6faf0f (credit goes to @pippolo84)

Are we ok to backport this to v1.11, CI is broken over there as well.

@ldelossa ldelossa mentioned this pull request Jan 24, 2023
18 tasks
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Backport pending to v1.12 to Backport done to v1.12 in 1.12.6 Jan 24, 2023
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Needs backport from master to Backport pending to v1.10 in 1.11.13 Jan 24, 2023
@maintainer-s-little-helper maintainer-s-little-helper bot added this to Backport pending to v1.10 in 1.10.19 Jan 24, 2023
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Backport pending to v1.10 to Backport done to v1.10 in 1.10.19 Jan 24, 2023
@joestringer joestringer added backport-done/1.11 The backport for Cilium 1.11.x for this PR is done. and removed backport-pending/1.11 labels Jan 25, 2023
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Backport pending to v1.10 to Backport done to v1.11 in 1.11.13 Jan 25, 2023
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Backport pending to v1.10 to Backport done to v1.11 in 1.11.13 Jan 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-done/1.11 The backport for Cilium 1.11.x for this PR is done. backport-done/1.12 The backport for Cilium 1.12.x for this PR is done. backport-done/1.13 The backport for Cilium 1.13.x for this PR is done. release-note/ci This PR makes changes to the CI.
Projects
No open projects
1.10.19
Backport done to v1.10
1.11.13
Backport done to v1.11
1.12.6
Backport done to v1.12
Development

Successfully merging this pull request may close these issues.

None yet

6 participants