Update procedures for Copr workflow

theforeman · Nov 15, 2023 · 79b05de · 79b05de
1 parent 8ebeb8f
commit 79b05de
Show file tree

Hide file tree

Showing 6 changed files with 56 additions and 27 deletions.
diff --git a/procedures/foreman/branch.md.erb b/procedures/foreman/branch.md.erb
@@ -60,12 +60,6 @@
   - [ ] Use <%= rel_eng_script('upload_yum_gpg') %> to create [releases/<%= release %>/RPM-GPG-KEY-foreman](https://yum.theforeman.org/releases/<%= release %>/RPM-GPG-KEY-foreman) on yum.theforeman.org
   - [ ] Commit the [settings file](https://github.com/theforeman/theforeman-rel-eng/blob/master/releases/foreman/<%= release %>/settings) to the `theforeman-rel-eng` repository
 - [ ] Create new settings files for [client](https://github.com/theforeman/theforeman-rel-eng/blob/master/releases/client/<%= release %>/settings), ensure it has the rights `OSES` list and commit it too.
-- [ ] Create mash configs in [tool_belt](https://github.com/theforeman/tool_belt/tree/master/configs/foreman) using <%= rel_eng_script('create_mash_configs') %> and submit this as a PR.
-- [ ] Once merged, deploy mash configuration on Koji using <%= rel_eng_script('deploy_mash_configs') %>
-- [ ] Update [collection-mash-split.py](https://github.com/theforeman/foreman-infra/blob/master/koji/collection-mash-split.py)
-  - Add a Katello version mapping
-  - Make any changes if needed (usually happens when there are tag changes)
-  - Deploy using `scp collection-mash-split.py root@koji.katello.org:/usr/local/bin`
 - [ ] Open a PR with the result of [jenkins-jobs](https://github.com/theforeman/jenkins-jobs) branching: `./branch-foreman <%= release %> KATELLO_VERSION`
 - [ ] Open PR to remove any jobs that are related to end of life versions
 - [ ] Open PR to add <%= release %> to [Forklift versions config](https://github.com/theforeman/forklift/blob/master/vagrant/config/versions.yaml). Once the PR is merged, upgrade pipelines will fail, so do not merge it before packaging has been branched.
@@ -74,10 +68,12 @@
 
 ## Release Engineer
 
-- Branch RPM packaging using [tool_belt](https://github.com/theforeman/tool_belt)
-  - [ ] Clone tags and create build targets in Koji: `bundle exec ./tools.rb koji --commit configs/foreman/<%= release %>.yaml`
-    Note that you must use koji client < 1.30 since clone-tag was changed and our Koji server is too old. See https://docs.pagure.org/koji/release_notes/release_notes_1.30/#system-changes for details.
-  - [ ] Create a `rpm/<%= release %>` branch in [foreman-packaging](https://github.com/theforeman/foreman-packaging) based on `rpm/develop`
+- Branch RPM packaging
+  - [ ] Create a `rpm/<%= release %>` branch in [foreman-packaging](https://github.com/theforeman/foreman-packaging) based on `rpm/develop`: `git checkout rpm/develop && git pull && git checkout -b rpm/<%= release %>`
+  - [ ] Update `foreman_version` in `package_manifest.yaml` on the `rpm/<%= release %>` branch: `sed -i '/foreman_version:/ s/nightly/<%= release %>/' package_manifest.yaml`
+  - [ ] Update `katello_version` in `package_manifest.yaml` on the `rpm/<%= release %>` branch: `sed -i '/katello_version:/ s/nightly/KATELLO_VERSION_HERE/' package_manifest.yaml
+  - [ ] Create release repositories in Copr by forking nightly repositories `obal copr-project copr_projects` on the `rpm/<%= release %>` branch
+  - [ ] Push the `rpm/<%= release %>` branch
 - Branch Debian packaging
   - [ ] Clone Debian nightly repos to <%= release %> using [copy/freight instructions](https://projects.theforeman.org/projects/foreman/wiki/Debian_Packaging#Branching-for-release)
   - [ ] Create a `deb/<%= release %>` branch in [foreman-packaging](https://github.com/theforeman/foreman-packaging) based on `deb/develop`

diff --git a/procedures/foreman/release.md.erb b/procedures/foreman/release.md.erb
@@ -64,22 +64,38 @@ Note: If for some reason there was an issue with the tarballs that required uplo
 - Update [foreman-packaging](https://github.com/theforeman/foreman-packaging) branches
   - [ ] rpm/<%= short_version %> using <%= rel_eng_script('bump_rpm_packaging') %>
   - [ ] deb/<%= short_version %> using <%= rel_eng_script('bump_deb_packaging') %>
+<% if Gem::Version.new(foreman_version) < Gem::Version.new('3.9') -%>
 - [ ] Wait for packages to be built using <%= rel_eng_script('wait_packaging') %> (checks [rpm/<%= short_version %>](https://ci.theforeman.org/job/foreman-packaging-rpm-<%= short_version %>-release/) and [deb/<%= short_version %>](https://ci.theforeman.org/job/foreman-packaging-deb-<%= short_version %>-release/)). This takes a while so you can already start on the manual updates.
+<% else -%>
+- [ ] Wait for packages to be built using <%= rel_eng_script('wait_copr_packaging') %> (checks [rpm/<%= short_version %>](https://ci.theforeman.org/job/foreman-packaging-rpm-copr-<%= short_version %>-release/) and [deb/<%= short_version %>](https://ci.theforeman.org/job/foreman-packaging-deb-<%= short_version %>-release/)). This takes a while so you can already start on the manual updates.
+<% end -%>
 - Check for outstanding PRs against <%= short_version %> packaging, and merge if possible:
   - [ ] [rpm/<%= short_version%>](https://github.com/theforeman/foreman-packaging/pulls?q=is%3Apr+is%3Aopen+base%3Arpm%2F<%= short_version %>)
   - [ ] [deb/<%= short_version%>](https://github.com/theforeman/foreman-packaging/pulls?q=is%3Apr+is%3Aopen+base%3Adeb%2F<%= short_version %>)
 - Sign the RPMs in the release
+<% if Gem::Version.new(foreman_version) < Gem::Version.new('3.9') -%>
   - [ ] <%= rel_eng_script('download_rpms') %>
   - [ ] <%= rel_eng_script('sign_rpms') %>
   - [ ] <%= rel_eng_script('upload_rpm_signatures') %>
   - [ ] <%= rel_eng_script('upload_rpms') %>
+<% else -%>
+  - [ ] <%= rel_eng_script('generate_stage_repository') %>
+  - [ ] <%= rel_eng_script('sign_stage_rpms') %>
+  - [ ] <%= rel_eng_script('upload_stage_rpms') %>
+<% end -%>
 - Sign RPMs for client repos (call scripts with `PROJECT=client`)
+<% if Gem::Version.new(foreman_version) < Gem::Version.new('3.9') -%>
   - [ ] <%= rel_eng_script('download_rpms') %>
   - [ ] <%= rel_eng_script('sign_rpms') %>
   - [ ] <%= rel_eng_script('upload_rpm_signatures') %>
   - [ ] <%= rel_eng_script('upload_rpms') %>
-- [ ] Kick off the [release pipeline](https://ci.theforeman.org/job/foreman-<%= short_version %>-release-pipeline/) by calling <%= rel_eng_script('release_pipeline') %>
-- [ ] Kick off the [client pipeline](https://ci.theforeman.org/job/foreman-client-<%= short_version %>-rpm-pipeline/) by calling `PROJECT=client ./release_pipeline`
+<% else -%>
+  - [ ] <%= rel_eng_script('generate_stage_repository') %>
+  - [ ] <%= rel_eng_script('sign_stage_rpms') %>
+  - [ ] <%= rel_eng_script('upload_stage_rpms') %>
+<% end -%>
+- [ ] Kick off the [release pipeline](https://ci.theforeman.org/job/foreman-<%= short_version %>-release-pipeline/) by calling `<%= rel_eng_script('release_pipeline') %>`
+- [ ] Kick off the [client pipeline](https://ci.theforeman.org/job/foreman-client-<%= short_version %>-rpm-pipeline/) by calling `PROJECT=client <%= rel_eng_script('release_pipeline') %>`
 - [ ] Kick off the [plugins pipeline](https://ci.theforeman.org/job/foreman-plugins-<%= short_version %>-rpm-pipeline/) by calling <%= rel_eng_script('plugins_pipeline') %>
 
 # Manual updates: <%= target_date %>

diff --git a/procedures/katello/branch.md.erb b/procedures/katello/branch.md.erb
@@ -1,7 +1,6 @@
 ## Roles
 
 * Release Owner: <%= owner %>
-* Release Engineer: <%= engineer %>
 
 # One Month Prior to Branch Date
 
@@ -39,10 +38,8 @@
   - [ ] Manually update the following sections:
     - [ ] releases: update to current release.  Move the previous 'current' release to prior_releases below.
     - [ ] prior_releases: Remove the oldest prior_release (check with that release owner first to see if there's a reason it should stay)
-    - [ ] mash_scripts: update Katello version number in all values
     - [ ] repos: Update branch names to current versions, including any new releases that need to happen
     - [ ] ignores: Delete all items from this list and start fresh (this will be used for cherry-picks later)
-    - [ ] gpg-key: When it becomes available, get the new Foreman GPG key for the corresponding Foreman version ([example here](https://github.com/theforeman/theforeman-rel-eng/blob/master/releases/foreman/2.5/settings) and put the last 8 characters here
     - [ ] tags: update Katello version number in all values.  Check the nightly config to see if any tags/repos need to be updated
   - [ ] Open a PR to [tool_belt](https://github.com/theforeman/tool_belt) with the new config file
 - [ ] Coordinate with installer maintainers that expected changes are completed.
@@ -52,10 +49,6 @@
   - [ ] [katello](https://github.com/theforeman/puppet-katello)
 - [ ] Review the Foreman schedule and planning ([example](https://community.theforeman.org/t/foreman-2-5-schedule-and-planning/22219)) and note the date of the first scheduled release candidate.
 
-## Release Engineer
-
-- [ ] Ensure tool_belt config is merged and output from `./tools.rb koji configs/katello/<%= release %>.yaml` matches expectations
-
 # On Branch Date
 
 ## Release Owner
@@ -74,10 +67,3 @@
   - [ ] Rename the "Katello Next" release to Katello <%= develop %>.0
   - [ ] Recreate the "Katello Next" release and indicate that it is a placeholder for issues belonging to the next version of Katello
   - [ ] Create the "Katello <%= release %>.1 TODO" custom query using the last one as a template
-
-
-## Release Engineer
-
-- [ ] Run `./tools.rb koji configs/katello/<%= release %>.yaml --confirm` from [tool_belt](https://github.com/theforeman/tool_belt) to create Koji tags
-- [ ] Run `./tools.rb mash-scripts configs/katello/<%= release %>.yaml` from [tool_belt](https://github.com/theforeman/tool_belt) to create Koji mash configs and open PR to tool_belt to commit
-- [ ] Copy mash configs to Koji using <%= rel_eng_script('deploy_mash_configs') %>
diff --git a/procedures/katello/release.md.erb b/procedures/katello/release.md.erb
@@ -65,11 +65,20 @@
 - [ ] Wait for Foreman <%= foreman_version %>.0 GA to be packaged and built before proceeding
 <% end -%>
 - [ ] Update release version similar to [here](https://github.com/theforeman/theforeman-rel-eng/commit/2029a9688da00d9c385c3438dd71b594ba5f728e)
-- [ ] Update `katello`, `katello-repos` and `rubygem-katello` 
+- [ ] Update `katello`, `katello-repos` and `rubygem-katello`
   - [ ] Use <%= rel_eng_script('bump_rpm_packaging') %>: `PROJECT=katello VERSION=<%= short_version %> ./bump_rpm_packaging`
 - [ ] Merge packaging PR once job is green
+<% if Gem::Version.new(foreman_version) < Gem::Version.new('3.9') -%>
 - [ ] Use <%= rel_eng_script('wait_packaging') %> to wait for [Jenkins to build the packages](https://ci.theforeman.org/job/foreman-packaging-rpm-<%= foreman_version %>-release/)
 - [ ] <%= rel_eng_script('download_rpms') %>, <%= rel_eng_script('sign_rpms') %>, <%= rel_eng_script('upload_rpm_signatures') %>, <%= rel_eng_script('upload_rpms') %>
+<% end -%>
+<% if Gem::Version.new(foreman_version) >= Gem::Version.new('3.9') -%>
+- [ ] Use <%= rel_eng_script('wait_copr_packaging') %> to wait for [Jenkins to build the packages](https://ci.theforeman.org/job/foreman-packaging-rpm-copr-<%= foreman_version %>-release/)
+- Sign the RPMs in the release
+  - [ ] <%= rel_eng_script('generate_stage_repository') %>
+  - [ ] <%= rel_eng_script('sign_stage_rpms') %>
+  - [ ] <%= rel_eng_script('upload_stage_rpms') %>
+<% end -%>
 - [ ] Use <%= rel_eng_script('release_pipeline') %> to kick off the [release pipeline](https://ci.theforeman.org/job/katello-<%= short_version %>-rpm-pipeline/)
 
 # Once release is out

diff --git a/procedures/pulpcore/update.md.erb b/procedures/pulpcore/update.md.erb
@@ -36,6 +36,16 @@ The `python-pulp_2to3_migration` package is named differently due to a tito bug
 
 Any change to packaging requires signing, as otherwise the repo can't be published.
 
+
+If working with Pulpcore 3.29+:
+
+* [ ] Sign the RPMs in the release, pass `PROJECT=pulpcore` and `VERSION=<%= release %>` to every invocation.
+  * [ ] <%= rel_eng_script('generate_stage_repository') %>
+  * [ ] <%= rel_eng_script('sign_stage_rpms') %>
+  * [ ] <%= rel_eng_script('upload_stage_rpms') %>
+
+If working with Pulpcore 3.28 or less:
+
 * [ ] Sign the RPMs in the release, pass `PROJECT=pulpcore` and `VERSION=<%= release %>` to every invocation.
   * [ ] [Download](https://github.com/theforeman/theforeman-rel-eng/blob/master/download_rpms)
   * [ ] [Sign](https://github.com/theforeman/theforeman-rel-eng/blob/master/sign_rpms)

diff --git a/wait_copr_packaging b/wait_copr_packaging
@@ -0,0 +1,12 @@
+#!/bin/bash -e
+
+. settings
+
+if [[ $PROJECT == pulpcore ]] ; then
+	./ci-jobs poll "pulpcore-packaging-rpm-copr-${VERSION}-release"
+else
+	./ci-jobs poll "foreman-packaging-deb-${FOREMAN_VERSION}-release" &
+	./ci-jobs poll "foreman-packaging-rpm-copr-${FOREMAN_VERSION}-release" &
+
+	wait
+fi