Skip to content

Commit

Permalink
[RHELC-1033, RHELC-1036] Make (re-)registering the system optional (#869
Browse files Browse the repository at this point in the history
)

* Make subscription from within convert2rhel optional.

Previously, we would unregister a system if it was registered and then re-register with
subscription-manager.  The only time we didn't do this would be when the user specified --no-rhsm
but that does too much.  It causes convert2rhel to avoid the use of subscription-manager altogether
(it doesn't install the package or configure the RHEL repos) For our purposes, we just don't want to
replace the registration if the system already has one.

The strategy we're implementing here is:
* If the user specifies (username and password) or (activation_key and organization) then
  (re)register the system.
* If not, then attempt to do the conversion without registering the system first.

There are several distinct changes made to enable this functionality:
* subscription.{subscribe_system,rollback}() were moved ito a RestorableChange (called
  RestorableSystemSubscription).  This is mainly because deciding whether to unregister the system
  in case of a rollback now needs to have some saved state to know whether we registered it in the
  first place.
* replace_subscription_manager() no longer unregisters the system as part of uninstalling the
  current subscription-manager packages and installing the RHEL ones.
* A new property was added to toolopts, should_subscribe.  This property tells whether the user has
  specified options which ask us to (re)register the system or not.
* In pre_ponr_changes/subscription.py we now use should_subscribe to decide whether to (re)subscribe
  the system instead of only no_rhsm.  We also use RestorableSystemSubscribe instead of calling
  subscribe_system().
* In main.py we no longer have to call subscription.rollback() because the backup framework
  (backup.backup_control.pop_all()) will take care of rolling back the subscription.
* Reviewed places where tool_opts.no_rhsm was used and change places that were used only to
  subscribe the system (as opposed to installing subscription-manager, which is also used for other
  things) to subscription.should_subscribe() instead:

  * actions.pre_ponr_changes.subscription.PreSubscription: This action makes sure that
    /usr/bin/subscription-manager can run so it needs to run even if we don't subscribe the system.
  * actions.system_checks.dbus.DbusIsRunning: We currently only use DBus as part of subscribing the
    system so should_subscribe is appropriate here.
  * toolopts.CLI: use should_subscribe to tell the user we're ignoring the presence of --serverurl.

* Deprecate --keep-rhsm and use a heuristic to decide whether to register with subscription-manager.

  The heuristic is to check whether the user gave us the necessary credentials to register the system
  with subscription-manager.  If so, then we use rhsm to do so.  If not, then we do not attempt to
  register the system.

  Fixes: https://issues.redhat.com/browse/RHELC-1036

* cert.py:
  * make cert.SystemCert() work with the backup system.  Now it will only install the certificate if
    it isn't already present on the system and only remove the certificate if it was convert2rhel
    which installed it.
  * Give SystemCert the ability to install to different directories.  This allows us to use it for
    both the susbcription-manager product certificate and the yum repo certificate (needed for the
    convert2rhel yum repo).
* main.py: Use the backup system to uninstall the certificates if we rollback instead of making
  a manual call to remove it.
* subscription.py:
  * The RestorableChange for registering and unregistering the system no longer needs to check
    keep_rhsm.  We will always unsubscribe the system if we subsribed it but by default we are no
    longer subscribing it in the first place.
  * Remove replace_subscription_manager() and remove_original_subscription_manager() as we don't
    need anything that uninstalls the subscription-manager package from the vendor anymore.
  * Add detect_subscription_manager() to check whether subscription-manager is installed (the
    entrypoint packages that we need for conversion, basically '*subscription-manager*'.)
  * Move a check for whether subscription-manager packages for RHEL have been downloaded
    successfully from replace_subscription_manager() to install_rhel_subscription_manager().
    This entails calling logger.critical() if the rpms are not present.  In the past, we would fail
    in the same way, just at an earlier stage (in replace[..]() instead of install[..]().
  * Move the installation of the redhat-uep.pem for the yum repos into a pre_ponr_change
    instead of here.  It's graduating from a hack to something that we need to ensure is installed.
    (Remove _check_and_install_redhat_uep_pem()).
  * verify_rhsm_installed(): Remove the keep_rhsm check.  Now, we will always fail at this point if
    verify_rhsm_installed() was called and subscription-manager is not installed.  (The equivalent
    check is now done by the caller)
  * download_rhsm_pkgs(): Do not check for keep_rhsm.  Now, when this is called, we will always
    download the packages.  (The equivalent check is now done by the caller)
* toolopts.py: Remove the keep_rhsm toolopts attribute.  When parsing the CLI, warn if --keep-rhsm
  is passed and ignore the option.
* actions/pre_ponr_changes/subscription.py:
  * Create and action for installing yum repo certs with code migrated from
    subscription._check_and_install_redhat_uep_pem()
  * Move installation of the gpg key from the PreSubscription Action and make both Presubscription
    and pre_ponr_changes.transaction.ValidatePackageManagerTransaction depend on it. (TODO from
    r0x0d)
* Now only download and install subscription-manager if they are not already installed.
* Remove an unused namedtuple import
* Remove unused toolopts.credentials_thru_cli
  When the RegistrationCommand was refactored into a class, we stopped
  needing credentials_thru_cli.  Remove that now.
* Make sure subscription-manager knows about the product certificate.
  Add a function to call subscription-manager refresh.  Use it to make sure subscription-manager
  knows about any RHSM product certificate the user may have put on the system since the last time
  subscription-manager refreshed its cache.

* Add subscription-manager package installation to the backup controller.  This is needed so that the
  packages can be uninstalled after we have unregistered the system during rollback.
  Without this change thte tiing was wrong because:
  * some packages (those which were removed) need to be rolled back early (before file rollbacks
    happen)
  * The packages here (installed subscription-manager packages) need to be rolled back later, after
    unregistering the system.
  * Unregistering the system is now handled by the backup framework and the backup framework
    rollback is a single step.

  Since the backup framework rollback can't exist both before and after the other rollback steps,
  adding the subscription-manager package rollbacks to the framework is the only way to fix this.

* Use get_installed_pkg_information() instead of get_installed_pkg_objects() for
  subscription-manager installation. On RHEL8, those two functions deal with architecture
  specification differently.  Using get_installed_pkg_information everywhere gives the right answer
  on both RHEL7 and RHEL8.

* Make subscription-manager installation work with old versions of deps.

  We are trying to get convert2rhel to only install the subset of subscription-manager packages
  which it requires.  However, when we do have to install packages, we are getting them from the UBI
  repositories where the version of subscription-manager may need a newer vrsion of dependencies
  than the vendor has. For this reason, we may need to install some dependencies from the UBI
  repositories even though the vendor versions of them are already installed.

  Currently, python-syspurpose and json-c are the only problematic packages so make sure that they
  are added to the install set.

  .. seealso:: Bug report illustrating the version problem:
      #494

* syspurpose and jsonc should not be uninstalled on rollback if they were installed to begin with.

  Fix this by sending both packages to install and packages to update to RestorablePackageSet.  When
  we call restore(), packages which were updated are not uninstalled.  (In the future, we could
  backup and restore those packages).

  The PR fixing this in the past was: #393

Co-authored-by: Rodolfo Olivieri <rodolfo.olivieri3@gmail.com>
Co-authored-by: Michal Bocek <mbocek@redhat.com>
Co-authored-by: E Gustavsson <eric@spytec.se>
Co-authored-by: Martin 'kokesak' Litwora <mlitwora@redhat.com>

* Fix for poor error message when a cert has already been removed in rollback.

If something (for instance, package removal) removes a certificate that we installed, then we will
try to reove the cert during rollback but the file won't exist. Add some code to give a nice message
in this case.

Also fix the integration test test_rhsm_error_logged().  It was checking for a prior bad error
message (an exception).  That won't happen in the same circumstance with the new code so adapt it to fit the new reality.

* Hack to fix the order of rollbacks while we still have items that aren't ported to the backup framework.

* Fix package backup on a pre-registered system

When the system is registered to RHSM before running convert2rhel, the
redhat.repo file is not generated because of the missing RHSM product
cert on non-RHEL system. However convert2rhel installs this cert and
when we do a package backup, the subscription-manager yum plugin
generates the redhat.repo file. Having the RHEL repos defined and
enabled during the package backup causes a failure when accessing the
RHEL repos due to the $releasever being different on RHEL and
RHEL-clones. Besides that, when backing up packages we want to
be accessing only the RHEL-clone repos, not RHEL repos.

Also, fix the check whether the packages to be removed have been really
removed. We were comparing strings to objects which led to a false flag
warning that packages that were in fact removed haven't been removed.

Also, update dependencies of a few Actions to make sure:
- we backup system repos before pkg backup because we're
  using the backed up repos for the pkg backups
- we remove pkgs that contain repofiles only after installing
  sub-man pkgs because the sub-man pkg installation may require
  dependencies from the original vendor repos

---------

Signed-off-by: Daniel Diblik <ddiblik@redhat.com>
Co-authored-by: Rodolfo Olivieri <rodolfo.olivieri3@gmail.com>
Co-authored-by: Michal Bocek <mbocek@redhat.com>
Co-authored-by: E Gustavsson <eric@spytec.se>
Co-authored-by: Martin 'kokesak' Litwora <mlitwora@redhat.com>
Co-authored-by: Daniel Diblik <ddiblik@redhat.com>
  • Loading branch information
6 people committed Sep 7, 2023
1 parent fb6b4a9 commit 65f7c48
Show file tree
Hide file tree
Showing 45 changed files with 2,168 additions and 1,342 deletions.
25 changes: 13 additions & 12 deletions convert2rhel/actions/pre_ponr_changes/handle_packages.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,6 @@
logger = logging.getLogger(__name__)


def _get_pkg_names(packages):
return [pkghandler.get_pkg_nevra(pkg_obj, include_zero_epoch=True) for pkg_obj in packages]


class ListThirdPartyPackages(actions.Action):
id = "LIST_THIRD_PARTY_PACKAGES"

Expand Down Expand Up @@ -60,7 +56,7 @@ def run(self):
id="THIRD_PARTY_PACKAGE_DETECTED_MESSAGE",
title="Third party packages detected",
description="Third party packages will not be replaced during the conversion.",
diagnosis=warning_message + ", ".join(_get_pkg_names(third_party_pkgs)),
diagnosis=warning_message + ", ".join(pkghandler.get_pkg_nevras(third_party_pkgs)),
)
else:
logger.info("No third party packages installed.")
Expand All @@ -72,6 +68,7 @@ def extract_packages(self, pkg):

class RemoveExcludedPackages(actions.Action):
id = "REMOVE_EXCLUDED_PACKAGES"
dependencies = ("BACKUP_REPOSITORY",) # We use the backed up repos in remove_pkgs_unless_from_redhat()

def run(self):
"""
Expand Down Expand Up @@ -103,10 +100,9 @@ def run(self):
return

# shows which packages were not removed, if false, all packages were removed
pkgs_not_removed = sorted(frozenset(pkgs_to_remove).difference(pkgs_removed))
pkgs_not_removed = sorted(frozenset(pkghandler.get_pkg_nevras(pkgs_to_remove)).difference(pkgs_removed))
if pkgs_not_removed:
pkg_names = _get_pkg_names(pkgs_not_removed)
message = "The following packages were not removed: %s" % ", ".join(pkg_names)
message = "The following packages were not removed: %s" % ", ".join(pkgs_not_removed)
logger.warning(message)
self.add_message(
level="WARNING",
Expand All @@ -129,7 +125,13 @@ def run(self):

class RemoveRepositoryFilesPackages(actions.Action):
id = "REMOVE_REPOSITORY_FILES_PACKAGES"
dependencies = ("BACKUP_REDHAT_RELEASE",)
dependencies = (
"BACKUP_REDHAT_RELEASE",
# We use the backed up repos in remove_pkgs_unless_from_redhat()
"BACKUP_REPOSITORY",
# The installation of sub-man pkgs needs access to the original repofiles to get the sub-man deps from there
"PRE_SUBSCRIPTION",
)

def run(self):
"""
Expand Down Expand Up @@ -171,10 +173,9 @@ def run(self):
return

# shows which packages were not removed, if false, all packages were removed
pkgs_not_removed = sorted(frozenset(pkgs_to_remove).difference(pkgs_removed))
pkgs_not_removed = sorted(frozenset(pkghandler.get_pkg_nevras(pkgs_to_remove)).difference(pkgs_removed))
if pkgs_not_removed:
pkg_names = _get_pkg_names(pkgs_not_removed)
message = "The following packages were not removed: %s" % ", ".join(pkg_names)
message = "The following packages were not removed: %s" % ", ".join(pkgs_not_removed)
logger.warning(message)
self.add_message(
level="WARNING",
Expand Down
151 changes: 119 additions & 32 deletions convert2rhel/actions/pre_ponr_changes/subscription.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,21 +16,72 @@
__metaclass__ = type

import logging
import os.path

from convert2rhel import actions, cert, pkghandler, repo, subscription, toolopts
from convert2rhel import actions, backup, cert, pkghandler, repo, subscription, toolopts, utils


logger = logging.getLogger(__name__)

# Source and target directories for the cdn.redhat.com domain ssl ca cert that:
# - we tell customers to use when installing convert2rhel from that domain
# - is used by RHSM when accessing RHEL repos hosted on the Red Hat CDN
_REDHAT_CDN_CACERT_SOURCE_DIR = utils.DATA_DIR
_REDHAT_CDN_CACERT_TARGET_DIR = "/etc/rhsm/ca/"

# Source and target directories for the RHSM ssl cert that we tell customers need
# to use to access repositories managed by subscription-manager
_RHSM_PRODUCT_CERT_SOURCE_DIR = os.path.join(utils.DATA_DIR, "rhel-certs")
_RHSM_PRODUCT_CERT_TARGET_DIR = "/etc/pki/product-default"


class InstallRedHatCertForYumRepositories(actions.Action):
id = "INSTALL_RED_HAT_CERT_FOR_YUM"

def run(self):
super(InstallRedHatCertForYumRepositories, self).run()

# We need to make sure the redhat-uep.pem file exists since the
# various Red Hat yum repositories (including the convert2rhel
# repo) use it.
# The subscription-manager-rhsm-certificates package contains this cert but for
# example on CentOS Linux 7 this package is missing the cert due to intentional
# debranding. Thus we need to ensure the cert is in place even when the pkg is installed.
logger.task("Convert: Install cdn.redhat.com SSL CA certificate")
repo_cert = cert.PEMCert(_REDHAT_CDN_CACERT_SOURCE_DIR, _REDHAT_CDN_CACERT_TARGET_DIR)
backup.backup_control.push(repo_cert)


class InstallRedHatGpgKeyForRpm(actions.Action):
id = "INSTALL_RED_HAT_GPG_KEY"

def run(self):
super(InstallRedHatGpgKeyForRpm, self).run()

# Import the Red Hat GPG Keys for installing Subscription-manager
# and for later.
logger.task("Convert: Import Red Hat GPG keys")
pkghandler.install_gpg_keys()


class PreSubscription(actions.Action):
id = "PRE_SUBSCRIPTION"
dependencies = ("REMOVE_EXCLUDED_PACKAGES",)
dependencies = (
"INSTALL_RED_HAT_CERT_FOR_YUM",
"INSTALL_RED_HAT_GPG_KEY",
"REMOVE_EXCLUDED_PACKAGES",
)

def run(self):
super(PreSubscription, self).run()

if toolopts.tool_opts.no_rhsm:
# Note: we don't use subscription.should_subscribe here because we
# need the subscription-manager packages even if we don't subscribe
# the system. It's only if --no-rhsm is passed (so we rely on
# user configured repos rather than system-manager configured repos
# to get RHEL packages) that we do not need subscription-manager
# packages.
logger.warning("Detected --no-rhsm option. Skipping.")
self.add_message(
level="WARNING",
Expand All @@ -41,27 +92,37 @@ def run(self):
return

try:
# TODO(r0x0d): Check later if we can move this piece to be an independant
# check, rather than one step in the pre-subscription, as this is
# not only used for subscription-manager, but for installing the
# packages later with yum transaction.

# Import the Red Hat GPG Keys for installing Subscription-manager
# and for later.
logger.task("Convert: Import Red Hat GPG keys")
pkghandler.install_gpg_keys()

logger.task("Convert: Subscription Manager - Download packages")
subscription.download_rhsm_pkgs()

logger.task("Convert: Subscription Manager - Replace")
subscription.replace_subscription_manager()
logger.task("Convert: Subscription Manager - Check for installed packages")
subscription_manager_pkgs = subscription.needed_subscription_manager_pkgs()
if not subscription_manager_pkgs:
logger.info("Subscription Manager is already present")
else:
logger.task("Convert: Subscription Manager - Install packages")
# Hack for 1.4: if we install subscription-manager from the UBI repo, it
# may require newer versions of packages than provided by the vendor.
# (Note: the function is marked private because this is a hack
# that should be replaced when we aren't under a release
# deadline.
update_pkgs = subscription._dependencies_to_update(subscription_manager_pkgs)

# Part of another hack for 1.4 that allows us to rollback part
# of the backup control, then do old rollback items that
# haven't been ported into the backup framework yet, and then
# do the rest.
# We need to do this here so that subscription-manager packages
# that we install are uninstalled before other packages which
# we may install during rollback.
backup.backup_control.push(backup.backup_control.partition)

subscription.install_rhel_subscription_manager(subscription_manager_pkgs, update_pkgs)

logger.task("Convert: Subscription Manager - Verify installation")
subscription.verify_rhsm_installed()

logger.task("Convert: Install RHEL certificates for RHSM")
cert.SystemCert().install()
logger.task("Convert: Install a RHEL product certificate for RHSM")
product_cert = cert.PEMCert(_RHSM_PRODUCT_CERT_SOURCE_DIR, _RHSM_PRODUCT_CERT_TARGET_DIR)
backup.backup_control.push(product_cert)

except SystemExit as e:
# TODO(r0x0d): Places where we raise SystemExit and need to be
# changed to something more specific.
Expand Down Expand Up @@ -102,19 +163,45 @@ class SubscribeSystem(actions.Action):
def run(self):
super(SubscribeSystem, self).run()

if toolopts.tool_opts.no_rhsm:
logger.warning("Detected --no-rhsm option. Skipping.")
self.add_message(
level="WARNING",
id="SUBSCRIPTION_CHECK_SKIP",
title="Subscription check skip",
description="Detected --no-rhsm option. Skipping.",
)
return
if not subscription.should_subscribe():
if toolopts.tool_opts.no_rhsm:
logger.warning("Detected --no-rhsm option. Skipping subscription step.")
self.add_message(
level="WARNING",
id="SUBSCRIPTION_CHECK_SKIP",
title="Subscription check skip",
description="Detected --no-rhsm option. Skipping.",
)
return

logger.task("Convert: Subscription Manager - Reload configuration")
# We will use subscription-manager later to enable the RHEL repos so we need to make
# sure subscription-manager knows about the product certificate. Refreshing
# subscription info will do that.
try:
subscription.refresh_subscription_info()
except subscription.RefreshSubscriptionManagerError as e:
if "not yet registered" in str(e):
self.set_result(
level="ERROR",
id="SYSTEM_NOT_REGISTERED",
title="Not registered with RHSM",
description="This system must be registered with rhsm in order to get access to the RHEL rpms. In this case, the system was not already registered and no credentials were given to convert2rhel to register it.",
remediation="You may either register this system via subscription-manager before running convert2rhel or give convert2rhel credentials to do that for you. The credentials convert2rhel would need are either activation_key and organization or username and password. You can set these in a config file and then pass the file to convert2rhel with the --config-file option.",
)
return
raise

logger.warning("No rhsm credentials given to subscribe the system. Skipping the subscription step.")

try:
logger.task("Convert: Subscription Manager - Subscribe system")
subscription.subscribe_system()
# In the future, refactor this to be an else on the previous
# condition or a separate Action. Not doing it now because we
# have to disentangle the exception handling when we do that.
if subscription.should_subscribe():
logger.task("Convert: Subscription Manager - Subscribe system")
restorable_subscription = subscription.RestorableSystemSubscription()
backup.backup_control.push(restorable_subscription)

logger.task("Convert: Get RHEL repository IDs")
rhel_repoids = repo.get_rhel_repoids()
Expand All @@ -126,8 +213,8 @@ def run(self):
# we don't get backups to restore from on a rollback
logger.task("Convert: Subscription Manager - Enable RHEL repositories")
subscription.enable_repos(rhel_repoids)
except IOError as e:
# TODO(r0x0d): Places where we raise IOError and need to be
except OSError as e:
# TODO(r0x0d): Places where we raise OSError and need to be
# changed to something more specific.
# - Could fail in invoking subscription-manager to get repos (get_avail_repos)
# - Could fail in invoking subscirption-manager to disable repos (disable_repos)
Expand Down
1 change: 1 addition & 0 deletions convert2rhel/actions/pre_ponr_changes/transaction.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
class ValidatePackageManagerTransaction(actions.Action):
id = "VALIDATE_PACKAGE_MANAGER_TRANSACTION"
dependencies = (
"INSTALL_RED_HAT_GPG_KEY",
"REMOVE_EXCLUDED_PACKAGES",
# This package can cause problems during the validation. Since no one
# is depending on this action, it may run whenever it wants to, which
Expand Down
5 changes: 2 additions & 3 deletions convert2rhel/actions/system_checks/dbus.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,8 @@

import logging

from convert2rhel import actions
from convert2rhel import actions, subscription
from convert2rhel.systeminfo import system_info
from convert2rhel.toolopts import tool_opts


logger = logging.getLogger(__name__)
Expand All @@ -33,7 +32,7 @@ def run(self):
super(DbusIsRunning, self).run()
logger.task("Prepare: Check that DBus Daemon is running")

if tool_opts.no_rhsm:
if not subscription.should_subscribe():
logger.info("Skipping the check because we have been asked not to subscribe this system to RHSM.")
self.add_message(
level="INFO",
Expand Down

0 comments on commit 65f7c48

Please sign in to comment.