Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IPU 9 -> 10: Initial preparation: upgrade paths and leapp packages def #1169

Merged
merged 18 commits into from
Jul 18, 2024

Commits on Jul 15, 2024

  1. IPU 9 -> 10: Initial preparation: upgrade paths and leapp packages def

    Introducing initial changes needed for IPU 9 -> 10 to prevent abouvious
    crashes and inhibitors:
    * Extend the list of supported source versions for the upgrade by el9
    * Define upgrade paths for IPU 9 -> 10
    * Add Python path for el 10 (py 3.12)
    * Introducing el9toel10 repository
      created .gitkeep files inside empty dirs
    
    NOTE:
        Enable upgrade from RHEL 9.4 & 9.5 to RHEL 10.0 for now. For the
        experiment purposes. These will not be supported for the upgrade at all,
        but to make testing and experimentations easier, allow it for now.
    
    TODO:
    * add RHEL 10 product certificates
    * add RHEL 10 GPG keys
    pirat89 committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    889dc35 View commit details
    Browse the repository at this point in the history

Commits on Jul 17, 2024

  1. Packaging: Introduce EL 9 builds for IPU 9 -> 10

    SPEC files has been updated and polished to be more friendly when
    keeping suppoer for building in EL 7, 8, and 9. Also Makefile
    contains some initial changes But keeping the full solution for a
    separate commit.
    
    Packit configuration has been updated to start building for
    epel-9-x86_64 buildroot.
    pirat89 committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    11e8c81 View commit details
    Browse the repository at this point in the history
  2. Add RHEL9 container for building and testing

    Adding RHEL 9 containers so we can start to test & build packages
    in RHEL 9 environemnt.
    
    Since `python3-virtualenv` package is not available in the RHEL 9 UBI,
    a workaround is introduced `make install-deps-fedora`, which is used by
    the testing containers. If not able to install the package, `pip install
    virtualenv` is used as a fallback as this is working in the image.
    
    The relatively recent Python version used in the container means there
    is also a relative recent Pylint verision, which introduces 2+ new
    checks.
    matejmatuska authored and pirat89 committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    6885a9f View commit details
    Browse the repository at this point in the history
  3. tests: update pytest requirements to Py3.6+ compat

    Currently we use frozen python modules due to compatibility with
    python2. As we want to keep now still Py2 support due to RHEL 7
    for a while, we do not want to simply use the newest versions of
    current modules. However, with Python 3.12 we see it's really old
    and some of old versions (like pytest) does not work as expected.
    
    For that reason, adding conditional requirements for pytest. I chose
    now to stick with pytest 6.2.5 which is compatible with Python3.6+
    (due to RHEL 8).
    
    Most likely we will need to revise other requirements too, but I focused
    now just on obvious issues.
    pirat89 committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    57486cd View commit details
    Browse the repository at this point in the history
  4. IPU 9 -> 10: Add (FAKE) RHEL 10 certs and GPG keys

    * Add RHEL 10.0 Beta product certificates
    * Add FAKE RHEL 10.0 product certificates
      * productions certificates for RHEL 10 has not been generated yet,
        so adding faked certificates for now. In this part development
        it's expected to test upgrades using internal custom repositories
        only. Until the testing with RHSM certs are not needed yet.
        But they will have to be updated for sure before the release.
    * Add RHEL 10 (+Beta) RPM-GPG-KEY-redhat-release
      * updated key from RHELBLD-15138 attached
    
    * TODO: replace faked RHEL 10 product certificates by real ones when
      available
    
    Co-authored-by: Michal Reznik (mreznik) <mreznik@redhat.com>
    Co-authored-by: Matej Matuska <mmatuska@redhat.com>
    3 people committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    a82b471 View commit details
    Browse the repository at this point in the history
  5. Install util-linux into the target container

    The util-linug RPM is not installed automatically as a dependency
    when:
      * upgrading 9 -> 10
      * or when dnf is configured to: install_weak_deps=False
        (affecting all upgrade paths)
    
    Missing util-linux package leads to error due to missing `su` command
    inside the container.
    
    jira: https://issues.redhat.com/browse/RHEL-43978
    Rezney authored and pirat89 committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    af1ca27 View commit details
    Browse the repository at this point in the history
  6. Fix LEAPP_IPU_IN_PROGRESS (9to1 -> 9to10)

    Implementation of processing versions inside commands/command_utils.py
    has been bugy. One of function returned string instread of list
    which led to the situation that only the first character has been
    returned instead of the first chunk. This has been ok for releases
    where major version is just one decimal. However since el 10 it has
    been buggy and generated set envar:
        LEAPP_IPU_IN_PROGRESS=9to1
    instead of
        LEAPP_IPU_IN_PROGRESS=9to10
    
    Now it should be working correctly again.
    
    jira: OAMG-11260
    matejmatuska authored and pirat89 committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    5049986 View commit details
    Browse the repository at this point in the history
  7. Add unit tests for config.version.get_major_version()

    Based on the bug fixed in the previuos commit, we are rather adding
    unit-tests for get_major_version() function in common libraries,
    to prevent possible issues in future there too.
    pirat89 committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    0597aea View commit details
    Browse the repository at this point in the history
  8. Add artificial 9->10 repomaping data

    Note that the data is artificial, meaning it's not generated through
    repomapping generator and therefore not verified by it against the
    source of truth. The assumption is that the repos will be similar to how
    they are on RHEL 9 and are basically copied (redundant channels and rhui
    repos are dropped).
    pirat89 committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    72d52a9 View commit details
    Browse the repository at this point in the history
  9. Update PES data

    Rezney authored and pirat89 committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    f7e4744 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    d212b42 View commit details
    Browse the repository at this point in the history
  11. Skip "directory-hash" files

    They are not owned by any package and can dynamically grow to
    a huge amount of files causing hitting open files limit
    Rezney authored and pirat89 committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    b9ed093 View commit details
    Browse the repository at this point in the history
  12. IPU 9 -> 10: Migrate & rebuild RPM DB

    The RPM DB has been moved from /var/lib/rpm to /usr/lib/sysimage/rpm
    in RHEL 10. Apply the change and create symlink in the original
    path to the new one as expected.
    
    Also rebuild the RPM DB to ensure it's compatible with the new RPM
    version. Previously the RPM DB was being rebuilt during IPU 8 -> 9.
    However after discussion with RPM SMEs it has been decided that this
    is actually very reasonable to do always. So applying for any upgrade
    path but IPU 7 -> 8 (let's do not change this one so much when it's
    kind of finished).
    
    Co-authored-by: Matej Matuska <mmatuska@redhat.com>
    2 people authored and pirat89 committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    a92a1f1 View commit details
    Browse the repository at this point in the history
  13. Define empty obsoleted RPM GPG keys for RHEL 10

    No keys will be obosoleted, however it's expected to define the list.
    matejmatuska authored and pirat89 committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    c70d6e2 View commit details
    Browse the repository at this point in the history
  14. Add 9to10 to GH unit test action

    matejmatuska authored and pirat89 committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    1427bb2 View commit details
    Browse the repository at this point in the history
  15. Configuration menu
    Copy the full SHA
    cfe9e31 View commit details
    Browse the repository at this point in the history
  16. Replace deprecated shutil.rmtree onerror kwarg

    The `onerror` argument of shutil.rmtree is deprecated and replaced by
    `onexc` since Python 3.12. It accepts callback with the same arguments
    except for the last one which is now a subclass of `BaseException`. Our
    callback `libraries.utils.report_and_ignore_shutil_rmtree_error` is thus
    modified accordingly.
    matejmatuska authored and pirat89 committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    4d51850 View commit details
    Browse the repository at this point in the history
  17. UpgradeInitramfsGenerator: drop distutils dependency

    Originally the actor used distutils.version.LooseVersion to detect
    newest version of installed kernel package inside target userspace
    container. But the distutils python module has become deprecated
    and we should not use it anymore in Python 3.12+.
    
    However, we do not expect to see multiple versions of kernel present
    inside the target userspace container as the container is created
    during the IPU process from scratch (always). Considering the presence
    of multiple kernels to be sign of error, which could negatively affect
    also additional actions later.
    
    Updated the solution, raising an error if multiple kernels are detected
    inside the container. As we expect one kernel only, no need to compare
    versions of particular possible kernel packages.
    
    Note there are now just these cases when this could happen:
      * a third party package is required to be installed inside the
        container
      * an explicit requirement to install particular version of container
        is made by a custom actor
    pirat89 committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    32171f2 View commit details
    Browse the repository at this point in the history