Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Publish debian and rpm packages to Buildkite Packages #2824

Merged
merged 6 commits into from
Jun 12, 2024

Conversation

sj26
Copy link
Member

@sj26 sj26 commented Jun 12, 2024

We've added support for OIDC tokens to the package create endpoint built into Buildkite Packages now, so let's try it out! This adds another step similar to the Packagecloud task which uses an OIDC token to upload the agent packages to Buildkite Packages.

Testing this is a little tricky. I think we kinda just need to try it. All the changes are isolated to steps which are allowed to soft fail. The scripts are mostly copy/pasted from the proven Packagecloud scripts and then the differences I've tested in my terminal (e.g. https://github.com/buildkite/buildkite/pull/17083#issuecomment-2161873567).

Fixes PKG-7120.

@sj26 sj26 requested a review from a team June 12, 2024 01:49
@sj26 sj26 self-assigned this Jun 12, 2024
Copy link
Contributor

@moskyb moskyb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yo! this looks good and i'm psyched to be publishing things to packages. i have a couple notes, and would also be keen for you to run shellcheck across the newly-added script files just to catch any low-hanging fruit

queue: "deploy"
plugins:
- docker#v5.8.0:
image: "public.ecr.aws/docker/library/ruby:3.0"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

afaict we aren't actually using ruby at all - can we make this alpine or debian:$WHATEVER_DEBIAN_VERSION_PEOPLE_LIKE_USING_THESE_DAYS or something?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, happy to use whatever. I was using the same image so that it might already be warm in the cache, and it has curl installed. Neither alpine nor debian have curl installed.

032379705303.dkr.ecr.us-east-1.amazonaws.com/deploytools:2022.07 seems to be used by other steps and has curl, I'll swap to that.

Copy link
Member Author

@sj26 sj26 Jun 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth nothing, what these script are currently doing should be folded into bk or a plugin or something over time, hopefully without needing docker involved at all.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool, looking forward to it!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these two files (publish-debian-buildkite-packages.sh and publish-rpm-buildkite-packages.sh) seem to have almost the same functionality - are you able to combine them into one and parameterise them?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

@sj26 sj26 requested review from moskyb and a team June 12, 2024 04:13
@sj26
Copy link
Member Author

sj26 commented Jun 12, 2024

Cheers for the feedback @moskyb. shellcheck runs clean. Mind taking another look?

Copy link
Contributor

@moskyb moskyb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

@sj26 sj26 merged commit 18e81b8 into main Jun 12, 2024
1 check passed
@sj26 sj26 deleted the publish-buildkite-packages branch June 12, 2024 04:47
@sj26
Copy link
Member Author

sj26 commented Jun 12, 2024

Looks like the deploy tools container is unhappy with me, not sure why yet

image

https://buildkite.com/buildkite/agent-release-edge/builds/1275#01900ad3-e46c-4f19-8301-8b9f8f62f0c5

@sj26
Copy link
Member Author

sj26 commented Jun 12, 2024

(I was using the wrong queue!)

@yob
Copy link
Contributor

yob commented Jun 12, 2024

@sj26 buildkite/agent isn't on the allowlist for the deploy queue any more - hopefully elastic-runners works great though

@sj26
Copy link
Member Author

sj26 commented Jun 12, 2024

@yob yeah thanks, I think I originally copy/pasted the Packagecloud steps from an older branch which was still using the deploy queue — PEBKAC.

@yob
Copy link
Contributor

yob commented Jun 12, 2024

change change, everywhere, 🧀

@sj26
Copy link
Member Author

sj26 commented Jun 13, 2024

https://buildkite.com/buildkite/agent-release-edge/builds/1286#01900f34-e625-4497-895d-b3315aefb8ed/197

Screenshot 2024-06-13 at 11 51 41 am

nichsolson-yes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants