IPU 9 -> 10: Initial preparation: upgrade paths and leapp packages def #1169
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Thank you for contributing to the Leapp project!Please note that every PR needs to comply with the Leapp Guidelines and must pass all tests in order to be mergeable.
Packit will automatically schedule regression tests for this PR's build and latest upstream leapp build. If you need a different version of leapp from PR#42, use It is possible to schedule specific on-demand tests as well. Currently 2 test sets are supported,
[Deprecated] To launch on-demand regression testing public members of oamg organization can leave the following comment:
Please open ticket in case you experience technical problem with the CI. (RH internal only) Note: In case there are problems with tests not being triggered automatically on new PR/commit or pending for a long time, please contact leapp-infra. |
fernflower
reviewed
Jan 23, 2024
|
/packit copr-build |
Rezney
force-pushed
the
poc-ipu-9-10
branch
2 times, most recently
from
9c512d4
to
1e42b2b
Compare
|
/packit copr-build |
matejmatuska
force-pushed
the
poc-ipu-9-10
branch
from
708e812
to
c8898da
Compare
|
Rebased to include new device deprecation data and PES events. |
matejmatuska
force-pushed
the
poc-ipu-9-10
branch
4 times, most recently
from
40e2010
to
0e24ae2
Compare
matejmatuska
force-pushed
the
poc-ipu-9-10
branch
from
8261889
to
dd8070f
Compare
pirat89
commented
Jul 15, 2024
There was a problem hiding this comment.
Before it could be merged, we need to:
- we need to polish commits - there is number of commits that are basically dead - on top of sqash & fixup commits
- partially done. keeping the final round when tests are green again
- update repomap.json
- it needs to be rebased against
masterto resolve conflict
repos/system_upgrade/common/actors/checksaphana/libraries/checksaphana.py
Show resolved
Hide resolved
repos/system_upgrade/common/actors/opensshpermitrootlogincheck/actor.py
Outdated
Show resolved
Hide resolved
repos/system_upgrade/common/actors/removeobsoletegpgkeys/libraries/removeobsoleterpmgpgkeys.py
Outdated
Show resolved
Hide resolved
pirat89
force-pushed
the
poc-ipu-9-10
branch
4 times, most recently
from
5b19dcd
to
ce4e56b
Compare
Introducing initial changes needed for IPU 9 -> 10 to prevent abouvious
crashes and inhibitors:
* Extend the list of supported source versions for the upgrade by el9
* Define upgrade paths for IPU 9 -> 10
* Add Python path for el 10 (py 3.12)
* Introducing el9toel10 repository
created .gitkeep files inside empty dirs
NOTE:
Enable upgrade from RHEL 9.4 & 9.5 to RHEL 10.0 for now. For the
experiment purposes. These will not be supported for the upgrade at all,
but to make testing and experimentations easier, allow it for now.
TODO:
* add RHEL 10 product certificates
* add RHEL 10 GPG keys
|
@matejmatuska resolved the problem with distutils. keeping it in a separate commit for possible future investigations of bugs I introduced inside :-D (there is always bug) |
@pirat89 The code looks alright. I suggest adding a unit test(s) as there currently aren't any IIRC. (or I can add some) |
Calling it EOD. Some tests would be nice really, but as they are missing all the time, considering it nice to have in this situation. But it would be still better to have written at least TODO or something that they are actually missing. If you want, feel free to add them. Hmm...or maybe we could try to use AI :-P I've realized it's not bad when speaking about unit-tests. |
vojtechsokol
previously requested changes
Jul 16, 2024
SPEC files has been updated and polished to be more friendly when keeping suppoer for building in EL 7, 8, and 9. Also Makefile contains some initial changes But keeping the full solution for a separate commit. Packit configuration has been updated to start building for epel-9-x86_64 buildroot.
Adding RHEL 9 containers so we can start to test & build packages in RHEL 9 environemnt. Since `python3-virtualenv` package is not available in the RHEL 9 UBI, a workaround is introduced `make install-deps-fedora`, which is used by the testing containers. If not able to install the package, `pip install virtualenv` is used as a fallback as this is working in the image. The relatively recent Python version used in the container means there is also a relative recent Pylint verision, which introduces 2+ new checks.
Currently we use frozen python modules due to compatibility with python2. As we want to keep now still Py2 support due to RHEL 7 for a while, we do not want to simply use the newest versions of current modules. However, with Python 3.12 we see it's really old and some of old versions (like pytest) does not work as expected. For that reason, adding conditional requirements for pytest. I chose now to stick with pytest 6.2.5 which is compatible with Python3.6+ (due to RHEL 8). Most likely we will need to revise other requirements too, but I focused now just on obvious issues.
* Add RHEL 10.0 Beta product certificates
* Add FAKE RHEL 10.0 product certificates
* productions certificates for RHEL 10 has not been generated yet,
so adding faked certificates for now. In this part development
it's expected to test upgrades using internal custom repositories
only. Until the testing with RHSM certs are not needed yet.
But they will have to be updated for sure before the release.
* Add RHEL 10 (+Beta) RPM-GPG-KEY-redhat-release
* updated key from RHELBLD-15138 attached
* TODO: replace faked RHEL 10 product certificates by real ones when
available
Co-authored-by: Michal Reznik (mreznik) <mreznik@redhat.com>
Co-authored-by: Matej Matuska <mmatuska@redhat.com>
The util-linug RPM is not installed automatically as a dependency
when:
* upgrading 9 -> 10
* or when dnf is configured to: install_weak_deps=False
(affecting all upgrade paths)
Missing util-linux package leads to error due to missing `su` command
inside the container.
jira: https://issues.redhat.com/browse/RHEL-43978
Implementation of processing versions inside commands/command_utils.py
has been bugy. One of function returned string instread of list
which led to the situation that only the first character has been
returned instead of the first chunk. This has been ok for releases
where major version is just one decimal. However since el 10 it has
been buggy and generated set envar:
LEAPP_IPU_IN_PROGRESS=9to1
instead of
LEAPP_IPU_IN_PROGRESS=9to10
Now it should be working correctly again.
jira: OAMG-11260
Based on the bug fixed in the previuos commit, we are rather adding unit-tests for get_major_version() function in common libraries, to prevent possible issues in future there too.
Note that the data is artificial, meaning it's not generated through repomapping generator and therefore not verified by it against the source of truth. The assumption is that the repos will be similar to how they are on RHEL 9 and are basically copied (redundant channels and rhui repos are dropped).
They are not owned by any package and can dynamically grow to a huge amount of files causing hitting open files limit
The RPM DB has been moved from /var/lib/rpm to /usr/lib/sysimage/rpm in RHEL 10. Apply the change and create symlink in the original path to the new one as expected. Also rebuild the RPM DB to ensure it's compatible with the new RPM version. Previously the RPM DB was being rebuilt during IPU 8 -> 9. However after discussion with RPM SMEs it has been decided that this is actually very reasonable to do always. So applying for any upgrade path but IPU 7 -> 8 (let's do not change this one so much when it's kind of finished). Co-authored-by: Matej Matuska <mmatuska@redhat.com>
No keys will be obosoleted, however it's expected to define the list.
The `onerror` argument of shutil.rmtree is deprecated and replaced by `onexc` since Python 3.12. It accepts callback with the same arguments except for the last one which is now a subclass of `BaseException`. Our callback `libraries.utils.report_and_ignore_shutil_rmtree_error` is thus modified accordingly.
Originally the actor used distutils.version.LooseVersion to detect
newest version of installed kernel package inside target userspace
container. But the distutils python module has become deprecated
and we should not use it anymore in Python 3.12+.
However, we do not expect to see multiple versions of kernel present
inside the target userspace container as the container is created
during the IPU process from scratch (always). Considering the presence
of multiple kernels to be sign of error, which could negatively affect
also additional actions later.
Updated the solution, raising an error if multiple kernels are detected
inside the container. As we expect one kernel only, no need to compare
versions of particular possible kernel packages.
Note there are now just these cases when this could happen:
* a third party package is required to be installed inside the
container
* an explicit requirement to install particular version of container
is made by a custom actor
pirat89
dismissed
vojtechsokol’s stale review
Required changes have been addressed
pirat89
changed the title
|
@Rezney @matejmatuska great job guys!! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Introducing initial changes needed for IPU 9 -> 10 to prevent abouvious crashes and inhibitors:
get_leapp_packagesandget_leapp_dep_packagesLEAPP_IPU_IN_PROGRESSfor IPU 9->10 which set originally just9to1TODO (required bare minimum changes for the IPU 9 -> 10):