Add CentOS / RHEL 8 support #2997

alexandre-allard · 2020-12-22T13:55:45Z

Component: salt, build, packages

Context:
Support for CentOS 8 / RHEL 8

Summary:

Acceptance criteria:
Installation on RHEL 8 works (single-node-install-rhel-8) and on other dists also.

Closes: #2164

bert-e · 2020-12-22T13:55:47Z

Hello alexandre-allard-scality,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Status report is not available.

bert-e · 2020-12-22T13:55:52Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

the author
one peer

Peer approvals must include at least 1 approval from the following list:

buildchain/buildchain/packaging.py

buildchain/buildchain/versions.py

salt/metalk8s/map.jinja

salt/metalk8s/repo/redhat.sls

NicolasT · 2020-12-22T16:40:13Z

scripts/bootstrap.sh.in

+)
+# shellcheck disable=SC2034
+# Used through an indirection
+declare -a PACKAGES_REDHAT_7=(
    python36-rpm


Whenever I see python36-rpm for EL7 and python36 for EL8 I'm very confused. python36-rpm is not alike python36, right?

No, python36-rpm is the python36 lib to interact with rpm
Maybe it's not needed in rhel8 (and btw it's maybe also no longer needed with salt 3002.2 we have now, I didn't checked, but it's not related to this PR btw saltstack/salt#57972)
I agree it's really different stuff and to be consistent we may want to install "python36-rpm" on rhel8 as well (if this package exists, otherwise maybe we should add also "python36" in the list for rhel7 even if it's not really needed since it's a dependency of "python36-rpm")

Exactly. So, if we need python36-rpm on RHEL7 systems and python36 (when not using the RPM bindings), we should declare python36 as a dependency, and not rely on it being implicitly available because we install python36-rpm which has python36 as a dep.
If on RHEL8 we don't need python36-rpm, then it shouldn't be listed and only include python36 (or similar).

Let's try to keep the differences between the OSes as minimal as possible.

scripts/common.sh

bert-e · 2021-01-05T10:51:01Z

Integration data created

I have created the integration data for the additional destination branches.

this pull request will merge feature/2164-support-centos-8 into
development/2.7
w/2.8/feature/2164-support-centos-8 will be merged into development/2.8

The following branches will NOT be impacted:

development/1.0
development/1.1
development/1.2
development/1.3
development/2.0
development/2.1
development/2.2
development/2.3
development/2.4
development/2.5
development/2.6

You can set option create_pull_requests if you need me to create
integration pull requests in addition to integration branches, with:

@bert-e create_pull_requests

bert-e · 2021-01-05T10:51:02Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

the author
one peer

Peer approvals must include at least 1 approval from the following list:

bert-e · 2021-01-05T10:51:06Z

History mismatch

Merge commit #1b51240e40a36dca6f7908a4ccd9686ef89c2895 on the integration branch
w/2.8/feature/2164-support-centos-8 is merging a branch which is neither the current
branch feature/2164-support-centos-8 nor the development branch
development/2.8.

It is likely due to a rebase of the branch feature/2164-support-centos-8 and the
merge is not possible until all related w/* branches are deleted or updated.

Please use the reset command to have me reinitialize these branches.

bert-e · 2021-01-05T22:19:53Z

History mismatch

Merge commit #376abd07345365be5aa2dbbfe7b5a8ce3de3776d on the integration branch
w/2.8/feature/2164-support-centos-8 is merging a branch which is neither the current
branch feature/2164-support-centos-8 nor the development branch
development/2.8.

It is likely due to a rebase of the branch feature/2164-support-centos-8 and the
merge is not possible until all related w/* branches are deleted or updated.

Please use the reset command to have me reinitialize these branches.

alexandre-allard · 2021-01-06T08:49:54Z

/reset

bert-e · 2021-01-06T08:50:01Z

Reset complete

I have successfully deleted this pull request's integration branches.

bert-e · 2021-01-06T08:50:10Z

Integration data created

I have created the integration data for the additional destination branches.

this pull request will merge feature/2164-support-centos-8 into
development/2.7
w/2.8/feature/2164-support-centos-8 will be merged into development/2.8

The following branches will NOT be impacted:

development/1.0
development/1.1
development/1.2
development/1.3
development/2.0
development/2.1
development/2.2
development/2.3
development/2.4
development/2.5
development/2.6

You can set option create_pull_requests if you need me to create
integration pull requests in addition to integration branches, with:

@bert-e create_pull_requests

bert-e · 2021-01-06T08:50:11Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

the author
one peer

Peer approvals must include at least 1 approval from the following list:

bert-e · 2021-01-06T08:50:15Z

History mismatch

Merge commit #ebfb1157ae2a9ca5c87b54362ce1607a7e6bf3d8 on the integration branch
w/2.8/feature/2164-support-centos-8 is merging a branch which is neither the current
branch feature/2164-support-centos-8 nor the development branch
development/2.8.

It is likely due to a rebase of the branch feature/2164-support-centos-8 and the
merge is not possible until all related w/* branches are deleted or updated.

Please use the reset command to have me reinitialize these branches.

bert-e · 2021-01-06T09:34:39Z

History mismatch

Merge commit #0879315b1d49ca58f6aa6bc03f76bd0bd7860ca5 on the integration branch
w/2.8/feature/2164-support-centos-8 is merging a branch which is neither the current
branch feature/2164-support-centos-8 nor the development branch
development/2.8.

It is likely due to a rebase of the branch feature/2164-support-centos-8 and the
merge is not possible until all related w/* branches are deleted or updated.

Please use the reset command to have me reinitialize these branches.

bert-e · 2021-01-06T09:35:03Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

the author
one peer

Peer approvals must include at least 1 approval from the following list:

bert-e · 2021-01-08T09:51:40Z

Conflict

There is a conflict between your branch feature/2164-support-centos-8 and the
destination branch development/2.8.

Please resolve the conflict on the feature branch (feature/2164-support-centos-8).

 $ git fetch
 $ git checkout origin/feature/2164-support-centos-8
 $ git merge origin/development/2.8
 $ # <intense conflict resolution>
 $ git commit
 $ git push origin HEAD:feature/2164-support-centos-8

bert-e · 2021-01-11T09:21:12Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

the author
one peer

Peer approvals must include at least 1 approval from the following list:

Create a tree under packages/redhat to handle multiple releases. Adapt paths in the buildchain to the new scheme. Refs: #2164

Create a tree under `packages/debian` to handle multiple releases. Adapt paths in the buildchain to the new scheme. Refs: #2164

This is no longer needed as it has been fixed in Salt 3002, python36-rpm is now a direct dependency of salt-minion.

Since we now support multiple release, we need to append the release in the path to retrieve the right repository. Refs: #2164

Apt is not made to be used in a non-interactive way, but we do in the script used to fetch and build ubuntu packages. We deactivate the warning as it pollutes log output when there is a need to debug.

Use match to retrieve matched values instead of doing a match and then a gensub.

It looks like on CentOS 8, repoquery returns some SRPMs when looking for a package dependency. So, let's filter them out since we don't care about these. Refs: #2164

btrfs is no longer supported on RHEL 8, so we disable btrfs support for containerd on RHEL 8. Refs: #2164

Since RHEL 8 does not ship Python 2, we build this package for Python 3 instead 2. Refs: #2164

This is the definition to build the container image used to download and build MetalK8s packages and dependencies for RHEL 8. Refs: #2164

The purpose of this builder is to download and build RHEL 8 packages. Refs: #2164

Download and build packages for CentOS 8 Refs: #2164

In RHEL 8, the package name in repoquery command is no longer a positional argument, but the argument of the `--whatrequires` option, so we need to move this option right after the package name. Refs: #2164

On CentOS/RHEL 8, ensure Python3 is installed prior to do anything else. This is needed because there is no Python installed otherwise and we need it for some scripts (e.g. to check packages presence, before proceeding to bootstrap) Refs: #2164

Refs: #2164

We need this option on RHEL 8, otherwise we end up with RHSM information polluting stdout which then breaks the module waiting for a specific output format `<package-name> <version>`. Refs: #2164

Since we now support RHEL 8, we want to test it in pre-merge on a single-node instead of RHEL 7. Refs: #2164

Since all code in MetalK8s is APL2 licensed, let's put our sosreport plugin under the same license

Clean up entries that are no longer used anywhere. These are leftovers when we were using fromrepo to install embedded packages.

Refs: #2164

bert-e · 2021-01-15T13:11:05Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

the author
one peer

Peer approvals must include at least 1 approval from the following list:

alexandre-allard · 2021-01-18T13:19:49Z

/approve

bert-e · 2021-01-18T13:19:54Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

the author
one peer

Peer approvals must include at least 1 approval from the following list:

The following options are set: approve

bert-e · 2021-01-18T13:24:44Z

In the queue

The changeset has received all authorizations and has been added to the
relevant queue(s). The queue(s) will be merged in the target development
branch(es) as soon as builds have passed.

The changeset will be merged in:

✔️ development/2.8

The following branches will NOT be impacted:

development/1.0
development/1.1
development/1.2
development/1.3
development/2.0
development/2.1
development/2.2
development/2.3
development/2.4
development/2.5
development/2.6
development/2.7

There is no action required on your side. You will be notified here once
the changeset has been merged. In the unlikely event that the changeset
fails permanently on the queue, a member of the admin team will
contact you to help resolve the matter.

IMPORTANT

Please do not attempt to modify this pull request.

Any commit you add on the source branch will trigger a new cycle after the
current queue is merged.
Any commit you add on one of the integration branches will be lost.

If you need this pull request to be removed from the queue, please contact a
member of the admin team now.

The following options are set: approve

bert-e · 2021-01-18T14:51:28Z

I have successfully merged the changeset of this pull request
into targetted development branches:

✔️ development/2.8

The following branches have NOT changed:

development/1.0
development/1.1
development/1.2
development/1.3
development/2.0
development/2.1
development/2.2
development/2.3
development/2.4
development/2.5
development/2.6
development/2.7

Please check the status of the associated issue None.

Goodbye alexandre-allard-scality.

alexandre-allard requested a review from a team as a code owner December 22, 2020 13:55

NicolasT reviewed Dec 22, 2020

View reviewed changes

alexandre-allard force-pushed the feature/2164-support-centos-8 branch from 63fc71b to e399ea7 Compare December 22, 2020 17:15

TeddyAndrieux reviewed Jan 4, 2021

View reviewed changes

scripts/common.sh Outdated Show resolved Hide resolved

alexandre-allard force-pushed the feature/2164-support-centos-8 branch from e399ea7 to aad4ed1 Compare January 5, 2021 22:19

alexandre-allard force-pushed the feature/2164-support-centos-8 branch from aad4ed1 to 8c3ce93 Compare January 6, 2021 08:49

alexandre-allard force-pushed the feature/2164-support-centos-8 branch from 8c3ce93 to 9549335 Compare January 6, 2021 09:34

alexandre-allard changed the base branch from development/2.7 to development/2.8 January 6, 2021 09:34

alexandre-allard force-pushed the feature/2164-support-centos-8 branch 3 times, most recently from 6fe85bd to 130ce88 Compare January 8, 2021 09:51

alexandre-allard force-pushed the feature/2164-support-centos-8 branch from 130ce88 to b2c2deb Compare January 11, 2021 09:21

alexandre-allard force-pushed the feature/2164-support-centos-8 branch 3 times, most recently from 8991be5 to 1bba292 Compare January 13, 2021 08:52

alexandre-allard added 20 commits January 15, 2021 14:09

build, packages: handle different RedHat release

b3d29a2

Create a tree under packages/redhat to handle multiple releases. Adapt paths in the buildchain to the new scheme. Refs: #2164

build, packages: handle different Ubuntu releases

3e9de01

Create a tree under `packages/debian` to handle multiple releases. Adapt paths in the buildchain to the new scheme. Refs: #2164

salt,scripts: remove python36-rpm install

0d02228

This is no longer needed as it has been fixed in Salt 3002, python36-rpm is now a direct dependency of salt-minion.

scripts: Add dist release in repositories path

1cd9480

Since we now support multiple release, we need to append the release in the path to retrieve the right repository. Refs: #2164

packages: Remove warning when using apt in script

61df2c2

Apt is not made to be used in a non-interactive way, but we do in the script used to fetch and build ubuntu packages. We deactivate the warning as it pollutes log output when there is a need to debug.

packages: Small fix of awk script

673b4dd

Use match to retrieve matched values instead of doing a match and then a gensub.

packages: Filter out SRPMs from repoquery output

75e5871

It looks like on CentOS 8, repoquery returns some SRPMs when looking for a package dependency. So, let's filter them out since we don't care about these. Refs: #2164

packages: Disable containerd btrfs support

c5675bf

btrfs is no longer supported on RHEL 8, so we disable btrfs support for containerd on RHEL 8. Refs: #2164

packages: Build metalk8s-sosreport for Python3

a5c0710

Since RHEL 8 does not ship Python 2, we build this package for Python 3 instead 2. Refs: #2164

packages: Add RHEL 8 package builder Dockerfile

ef3ae10

This is the definition to build the container image used to download and build MetalK8s packages and dependencies for RHEL 8. Refs: #2164

build: Add builder for RHEL 8

5fcb709

The purpose of this builder is to download and build RHEL 8 packages. Refs: #2164

build, packages: Add support for CentOS 8

5c24b1d

Download and build packages for CentOS 8 Refs: #2164

salt: Fix repoquery option order for RHEL 8

e8e9e32

In RHEL 8, the package name in repoquery command is no longer a positional argument, but the argument of the `--whatrequires` option, so we need to move this option right after the package name. Refs: #2164

scripts: install python3 during bootstrap

35028c7

On CentOS/RHEL 8, ensure Python3 is installed prior to do anything else. This is needed because there is no Python installed otherwise and we need it for some scripts (e.g. to check packages presence, before proceeding to bootstrap) Refs: #2164

scripts, salt: Add support for RHEL 8

a193eaa

Refs: #2164

salt: Add --quiet option to repoquery

b2e62c9

We need this option on RHEL 8, otherwise we end up with RHSM information polluting stdout which then breaks the module waiting for a specific output format `<package-name> <version>`. Refs: #2164

eve: Replace RHEL 7 single node with RHEL 8

57d01ab

Since we now support RHEL 8, we want to test it in pre-merge on a single-node instead of RHEL 7. Refs: #2164

packages: Change metalk8s-sosreport licensing

2bef308

Since all code in MetalK8s is APL2 licensed, let's put our sosreport plugin under the same license

salt: Remove useless entries in map

1dfc7e7

Clean up entries that are no longer used anywhere. These are leftovers when we were using fromrepo to install embedded packages.

Add CHANGELOG entry for CentOS/RHEL 8 support

ae58ee1

Refs: #2164

alexandre-allard dismissed TeddyAndrieux’s stale review via ae58ee1 January 15, 2021 13:10

alexandre-allard force-pushed the feature/2164-support-centos-8 branch from 04ed5d2 to ae58ee1 Compare January 15, 2021 13:10

TeddyAndrieux approved these changes Jan 18, 2021

View reviewed changes

bert-e merged commit 7944925 into development/2.8 Jan 18, 2021

bert-e deleted the feature/2164-support-centos-8 branch January 18, 2021 14:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add CentOS / RHEL 8 support #2997

Add CentOS / RHEL 8 support #2997

alexandre-allard commented Dec 22, 2020 •

edited

Loading

bert-e commented Dec 22, 2020

bert-e commented Dec 22, 2020

NicolasT Dec 22, 2020

TeddyAndrieux Jan 4, 2021

NicolasT Jan 4, 2021

bert-e commented Jan 5, 2021

bert-e commented Jan 5, 2021

bert-e commented Jan 5, 2021

bert-e commented Jan 5, 2021

alexandre-allard commented Jan 6, 2021

bert-e commented Jan 6, 2021

bert-e commented Jan 6, 2021

bert-e commented Jan 6, 2021

bert-e commented Jan 6, 2021

bert-e commented Jan 6, 2021

bert-e commented Jan 6, 2021

bert-e commented Jan 8, 2021

bert-e commented Jan 11, 2021

bert-e commented Jan 15, 2021

alexandre-allard commented Jan 18, 2021

bert-e commented Jan 18, 2021

bert-e commented Jan 18, 2021

bert-e commented Jan 18, 2021

Add CentOS / RHEL 8 support #2997

Add CentOS / RHEL 8 support #2997

Conversation

alexandre-allard commented Dec 22, 2020 • edited Loading

bert-e commented Dec 22, 2020

Hello alexandre-allard-scality,

bert-e commented Dec 22, 2020

Waiting for approval

NicolasT Dec 22, 2020

Choose a reason for hiding this comment

TeddyAndrieux Jan 4, 2021

Choose a reason for hiding this comment

NicolasT Jan 4, 2021

Choose a reason for hiding this comment

bert-e commented Jan 5, 2021

Integration data created

bert-e commented Jan 5, 2021

Waiting for approval

bert-e commented Jan 5, 2021

History mismatch

bert-e commented Jan 5, 2021

History mismatch

alexandre-allard commented Jan 6, 2021

bert-e commented Jan 6, 2021

Reset complete

bert-e commented Jan 6, 2021

Integration data created

bert-e commented Jan 6, 2021

Waiting for approval

bert-e commented Jan 6, 2021

History mismatch

bert-e commented Jan 6, 2021

History mismatch

bert-e commented Jan 6, 2021

Waiting for approval

bert-e commented Jan 8, 2021

Conflict

bert-e commented Jan 11, 2021

Waiting for approval

bert-e commented Jan 15, 2021

Waiting for approval

alexandre-allard commented Jan 18, 2021

bert-e commented Jan 18, 2021

Waiting for approval

bert-e commented Jan 18, 2021

In the queue

bert-e commented Jan 18, 2021

alexandre-allard commented Dec 22, 2020 •

edited

Loading