Fixes #35058 - upgrade scenario for Foreman and Katello nightly

[upadhyeammit]

Depends on #626

The flow for the Foreman and Katello upgrade should be as below,

1. Have the correct repositories as per the version of the scenario. If scenario is for .z then update the release packages, in this case if package is missing update should install it and make relevant repositories available.
2. After getting the repositories, run the pre upgrade checks, procedures.
3. Enable correct modules if those are not enabled already.
4. Update the packages.
5. Run the installer
6. Execute any post upgrade procedures and checks.

[theforeman-bot]

Issues: #35058

[ehelms]

Ought the commit message only mention Foreman and not Katello?

[ehelms]

Could you include on here a comment describing what the expected workflow for upgrades will be?

[upadhyeammit]

Current status:

• Tried Foreman upgrades

  • Foreman 3.2 to 3.3 upgrade worked successfully, cmd,
  • # foreman-maintain upgrade run --target-version 3.3 --disable-self-upgrade
  • Foreman 3.3 to nightly upgrade worked successfully, cmd,
  • # foreman-maintain upgrade run --target-version nightly --disable-self-upgrade

• Tried Katello upgrades

  • Katello 4.4 to 4.5 worked successfully, cmd,
  • # foreman-maintain upgrade run --target-version 4.5 --disable-self-upgrade
  • Katello 4.5 to nightly worked successfully, cmd,
  • # foreman-maintain upgrade run --target-version nightly --disable-self-upgrade

• I tried EL upgrades only, on deb we are yet to solve foreman-release.deb situation.

• There is no nightly.z as users can just use nightly version to keep updating the system.

• Nightly upgrades are not bound to current version of Foreman or Katello, which means can be tried from older versions as well.

• The repositories we care for are only of Foreman, Katello and the current implementation expects other repositories are in place and configured correctly.

[upadhyeammit]

A couple of thoughts to help get in an initial version of this work:

1. Unless we plan to backport this work to 3.3 and 3.4 release streams I think this should for now only focus on nightly scenarios

2. If we want to add 3.4 or 3.3 scenarios, I'd layer those in in a follow up

3. The pre_upgrade issue for the tuning profile / system requirements is something we need to tackle, but as has been noted this requires doing an additional feature to the installer. That feature would correlate well with the nightly goal. I think the order could be: merge this PR with nightly scenario and no handling for failing installer checks, add installer feature for checks, update the scenario here with that check handling

I have removed the version based scenarios of Foreman and Katello and kept nightly. The tests were updated as per the change in installer feature.

[upadhyeammit]

@ekohl @evgeni like to know if you are good with this for initial start and later on we improve as per requirements?

[evgeni]

No comments besides the outstanding one around dnf module enable

[ekohl]

I played a bit with refactoring things so you don't repeat the target version all the time. In 3f99cc4 I restructured the directory layout a bit (it may break other things). Then in f0520bf I introduced a target_version method. In ab755a9 I tried to reduce duplication by introducing a base class for various upgrade scenarios.

While writing this, I realized I could apply the same changes in a more minimal patch to this one: dab29a9 does that. Then 1013221 makes the diff even smaller. Now it's:

--- definitions/scenarios/upgrade_to_foreman_nightly.rb	2022-09-08 13:05:33.742733736 +0200
+++ definitions/scenarios/upgrade_to_katello_nightly.rb	2022-09-08 13:08:45.405873682 +0200
@@ -1,4 +1,4 @@
-module Scenarios::Foreman_Nightly
+module Scenarios::Katello_Nightly
   class Abstract < ForemanMaintain::Scenario
     def self.target_version
       'nightly'
@@ -10,7 +10,7 @@
       metadata do
         tags :upgrade_scenario
         confine do
-          feature(:foreman_install) || ForemanMaintain.upgrade_in_progress == target_version
+          feature(:katello_install) || ForemanMaintain.upgrade_in_progress == target_version
         end
 
         @target_version = target_version
@@ -19,7 +19,7 @@
         end
 
         def target
-          "Foreman #{target_version}"
+          "Katello #{target_version}"
         end
 
         instance_eval(&block)
@@ -67,7 +67,7 @@
     end
 
     def modules_to_enable
-      el? ? ["foreman:#{el_short_name}"] : []
+      ["katello:#{el_short_name}", "pulpcore:#{el_short_name}"]
     end
 
     def compose

Now the next step could be to enhance modules_to_enable to look at the features to determine it. Once you have that, we can simplify it to just upgrade_to_upstream_nightly and we would simply have an upstream scenario that does the right thing for all installations.

Please include my patches (feel free to squash them in) and unify to a single upstream scenario.

[upadhyeammit]

I played a bit with refactoring things so you don't repeat the target version all the time. In 3f99cc4 I restructured the directory layout a bit (it may break other things). Then in f0520bf I introduced a target_version method. In ab755a9 I tried to reduce duplication by introducing a base class for various upgrade scenarios.

While writing this, I realized I could apply the same changes in a more minimal patch to this one: dab29a9 does that. Then 1013221 makes the diff even smaller. Now it's:

--- definitions/scenarios/upgrade_to_foreman_nightly.rb	2022-09-08 13:05:33.742733736 +0200
+++ definitions/scenarios/upgrade_to_katello_nightly.rb	2022-09-08 13:08:45.405873682 +0200
@@ -1,4 +1,4 @@
-module Scenarios::Foreman_Nightly
+module Scenarios::Katello_Nightly
   class Abstract < ForemanMaintain::Scenario
     def self.target_version
       'nightly'
@@ -10,7 +10,7 @@
       metadata do
         tags :upgrade_scenario
         confine do
-          feature(:foreman_install) || ForemanMaintain.upgrade_in_progress == target_version
+          feature(:katello_install) || ForemanMaintain.upgrade_in_progress == target_version
         end
 
         @target_version = target_version
@@ -19,7 +19,7 @@
         end
 
         def target
-          "Foreman #{target_version}"
+          "Katello #{target_version}"
         end
 
         instance_eval(&block)
@@ -67,7 +67,7 @@
     end
 
     def modules_to_enable
-      el? ? ["foreman:#{el_short_name}"] : []
+      ["katello:#{el_short_name}", "pulpcore:#{el_short_name}"]
     end
 
     def compose

Now the next step could be to enhance modules_to_enable to look at the features to determine it. Once you have that, we can simplify it to just upgrade_to_upstream_nightly and we would simply have an upstream scenario that does the right thing for all installations.

Please include my patches (feel free to squash them in) and unify to a single upstream scenario.

I like and admire your suggestions however I would like to have completely separate pull request or even work to refactor the upgrade scenarios/procedures. I have forked a new Redmine for same : https://projects.theforeman.org/issues/35508

[upadhyeammit]

Can we merge this one?