Server migrator to 15.4 #4704

mbussolotto · 2022-01-18T14:19:37Z

What does this PR change?

Upgrade server migrator to upgrade to 15.4.

GUI diff

No difference.

DONE

Documentation

Documentation required, issue will be created
DONE

Test coverage

No tests
DONE

Links

Fixes https://github.com/SUSE/spacewalk/issues/17546

DONE

Changelogs

Make sure the changelogs entries you are adding are compliant with https://github.com/uyuni-project/uyuni/wiki/Contributing#changelogs and https://github.com/uyuni-project/uyuni/wiki/Contributing#uyuni-projectuyuni-repository

If you don't need a changelog check, please mark this checkbox:

No changelog needed

If you uncheck the checkbox after the PR is created, you will need to re-run changelog_test (see below)

Re-run a test

If you need to re-run a test, please mark the related checkbox, it will be unchecked automatically once it has re-run:

Re-run test "changelog_test"
Re-run test "backend_unittests_pgsql"
Re-run test "java_pgsql_tests"
Re-run test "schema_migration_test_oracle"
Re-run test "schema_migration_test_pgsql"
Re-run test "susemanager_unittests"
Re-run test "javascript_lint"
Re-run test "spacecmd_unittests"

susemanager/bin/server-migrator.sh

… the magic

juliogonzalez · 2022-05-25T13:17:35Z

spacewalk/package/spacewalk.spec

@@ -118,7 +118,7 @@ Requires:       postgresql13-contrib
 Conflicts:      postgresql-implementation >= 14
 Conflicts:      postgresql-contrib-implementation >= 14
 %endif # if sle_version >= 150400
-%else # not suse_version
+%else # not suse_version or opensuse


Not for this PR, but I am wondering about this part... @sbluhm does it mean the installations on top of EL8 are still using PostgreSQL12?

EL is using 13.5. Does the logic force a different version? (if it does, then it's not working :) )

the code forced to use a postgres version between 12 and 14, I just changed it from 13 to 15 (15 is not released yet).

ACK. Thanks.

@sbluhm well, on paper EL should use the same version Uyuni uses for openSUSE.

Until now it was OK, because we were based on Leap 15.3 and PostgreSQL 13. But Uyuni 2022.06 will change to Leap 15.4 and PostgreSQL 14.

EL8 should be working, as I don't recall we are using anything specific that only works for PostgreSQL 14. But now that we have it on Uyuni for Leap, that could change.

In fact, it could be Uyuni on EL won't start again if it's not switched to PostgreSQL 14.

From the SUSE Manager 4.3 release notes I am copying this to Uyuni 2022.06:

To prevent inconsistent configurations and data on upgrade or update, Uyuni 2022.06 refuse to start until the database migration from PostgreSQL 13 to PostgreSQL 14 has been completed successfully.

Not sure if that check works for EL as well, but FMPOV, this should be reviewed and EL should use PosgreSQL 14 as well.

ok, let me check what I can do.

we will have postgresql14

update postgres version for no suse product

mbussolotto · 2022-05-26T14:32:44Z

Quick update. Last changes allow to install pg14, but we still have an issue

INFO: Migrating password encryption mechanism to scram-sha-256
× uyuni-check-database.service - Uyuni check database
     Loaded: loaded (/usr/lib/systemd/system/uyuni-check-database.service; static)
     Active: failed (Result: exit-code) since Thu 2022-05-26 16:02:02 CEST; 19ms ago
    Process: 1984 ExecStart=/usr/sbin/spacewalk-startup-helper check-database (code=exited, status=1/FAILURE)
   Main PID: 1984 (code=exited, status=1/FAILURE)

May 26 16:02:02 uyuni systemd[1]: Starting Uyuni check database...
May 26 16:02:02 uyuni spacewalk-startup-helper[1995]: report_db_host = uyuni.tf.local
May 26 16:02:02 uyuni spacewalk-startup-helper[2003]: psql: error: connection to server at "uyuni.tf.local" (192.168.100.151), port 5432 failed: Connection refused
May 26 16:02:02 uyuni spacewalk-startup-helper[2003]:         Is the server running on that host and accepting TCP/IP connections?
May 26 16:02:02 uyuni spacewalk-startup-helper[1984]: Report Database version '' is not supported for SUSE Manager/Uyuni on openSUSE Leap 15.4. Perform database migration.
May 26 16:02:02 uyuni systemd[1]: uyuni-check-database.service: Main process exited, code=exited, status=1/FAILURE
May 26 16:02:02 uyuni systemd[1]: uyuni-check-database.service: Failed with result 'exit-code'.
May 26 16:02:02 uyuni systemd[1]: Failed to start Uyuni check database.
FAILED

I'm investigating on it (it might be something related to `pg_hba.conf so similar to this #5184 ...or simply something on my environment :) )

juliogonzalez · 2022-05-26T14:52:34Z

Just in case... check if the system time for your server is OK. Specially if you are using KVM snapshots.

mbussolotto · 2022-05-26T15:13:45Z

ok found it. /var/lib/pgsql/data/postgresql.conf miss the line listen_addresses = '*', which is now required for reportdb connection. I'll fix it asap

juliogonzalez

Some minor comments.

susemanager/bin/pg-migrate-x-to-y.sh

susemanager/bin/server-migrator.sh

Co-authored-by: Julio González Gil <juliogonzalez@users.noreply.github.com>

sbluhm · 2022-05-26T17:46:10Z

spacewalk/package/spacewalk.spec

@@ -118,7 +118,7 @@ Requires:       postgresql13-contrib
 Conflicts:      postgresql-implementation >= 14
 Conflicts:      postgresql-contrib-implementation >= 14
 %endif # if sle_version >= 150400
-%else # not suse_version
+%else # not suse_version or opensuse


ok, let me check what I can do.

sbluhm · 2022-05-27T06:55:31Z

spacewalk/package/spacewalk.spec

@@ -118,7 +118,7 @@ Requires:       postgresql13-contrib
 Conflicts:      postgresql-implementation >= 14
 Conflicts:      postgresql-contrib-implementation >= 14
 %endif # if sle_version >= 150400
-%else # not suse_version
+%else # not suse_version or opensuse


we will have postgresql14

sbluhm · 2022-05-27T08:17:13Z

spacewalk/package/spacewalk.spec

@@ -118,7 +118,7 @@ Requires:       postgresql13-contrib
 Conflicts:      postgresql-implementation >= 14
 Conflicts:      postgresql-contrib-implementation >= 14
 %endif # if sle_version >= 150400
-%else # not suse_version
+%else # not suse_version or opensuse
 Requires:       postgresql >= 12


Can you please put this and the line below to >= 14?

mcalmer

I approve this, we can can only really test it, if it is merged.

mcalmer · 2022-05-30T11:56:24Z

susemanager/bin/server-migrator.sh

-zypper -n dup --allow-vendor-change
-if [ $? -ne 0 ];then
-    echo "Migration went wrong. Please fix the issues and try again."
+zypper -n dup


I tried this script yesterday and I think we have conflicts which must be solved manual.
There is at least the "netty" problem, where we require an older version, but a patch require the newer version.

We also have a problem with the vendor of a product.

@juliogonzalez should we remove the -n and let the user make decissions?

I tried this script yesterday and I think we have conflicts which must be solved manual.

What do you mean? Can you provide details? AFAIK @mbussolotto tested the upgrade and worked for him.

There is at least the "netty" problem, where we require an older version, but a patch require the newer version.

Can you provide more details? What patch is requiring the newer versions? I remember you adjusted one spec to require our version, to fix a bug caused by a version bump at Leap 15.3

Wasn't that enough?

We also have a problem with the vendor of a product.

You mean the discussion I have with Adrian? That's true, but doesn't really break the issue if we allow vendor change.

@juliogonzalez should we remove the -n and let the user make decissions?

I don't see why? The script should just work, no questions asked.

And BTW: why are we removing --allow-vendor-change?

And BTW: why are we removing --allow-vendor-change?

I tested upgrade from JeOS image and it wasn't required, I will re-add it if you think it's safer

But did the vendor change happened or not?

If it happened, then removing is OK.

Keep in mind there are a few packages we removed from our codestream and will now come from Leap, so those must change the vendor.

juliogonzalez · 2022-05-31T09:42:13Z

susemanager/bin/server-migrator.sh

 echo "==================================================================="
-echo "If you did not yet migrate the database to postgresql13, do so now"
+echo "If you did not yet migrate the database to the new postgres version, do so now"


One minor thing about this: I don't think the user can do the DB migration before this script runs.

Even if the new postgres is installed pg-migrate-x-to-y.sh and the related scripts would come from the old Uyuni version, so they won't be adapte to Postgres 14.

Shouldn't this sentece tell the user to run the DB migration script without any "if"?

Correct, also if new postgres version is installed, pg-migrate-x-to-y.sh is still required

should be better now

github-actions bot added the susemanager_unittests label Jan 18, 2022

mbussolotto self-assigned this Mar 28, 2022

server migrator to 15.4

7250721

mbussolotto force-pushed the server-migrator_15_4 branch from c706850 to 7250721 Compare April 28, 2022 10:09

mcalmer and others added 2 commits April 28, 2022 13:11

force installation of uyuni version of httpcomponents-asyncclient (#3)

9e99bb3

Force netty from Uyuni, do not force httpcomponents-asyncclient anymore

e57bcfc

juliogonzalez reviewed May 16, 2022

View reviewed changes

susemanager/bin/server-migrator.sh Show resolved Hide resolved

juliogonzalez reviewed May 16, 2022

View reviewed changes

susemanager/bin/server-migrator.sh Outdated Show resolved Hide resolved

juliogonzalez and others added 2 commits May 16, 2022 10:36

The right netty version is already required at spacewalk-java, remove…

940ad8d

… the magic

fix spec

2261b47

mbussolotto force-pushed the server-migrator_15_4 branch from 62a37cd to 2261b47 Compare May 25, 2022 12:19

juliogonzalez reviewed May 25, 2022

View reviewed changes

mbussolotto changed the title ~~DO NOT MERGE: server migrator to 15.4~~ Server migrator to 15.4 May 25, 2022

Update spacewalk.spec

cfcd8a4

update postgres version for no suse product

mbussolotto added 2 commits May 26, 2022 17:48

pg-migrate re-use the previous postgres.conf

70a0d71

do not allow vendor changes

9f8a5fd

mbussolotto marked this pull request as ready for review May 26, 2022 15:49

mbussolotto requested review from jordimassaguerpla, juliogonzalez and mcalmer May 26, 2022 15:50

juliogonzalez reviewed May 27, 2022

View reviewed changes

susemanager/bin/pg-migrate-x-to-y.sh Outdated Show resolved Hide resolved

susemanager/bin/server-migrator.sh Outdated Show resolved Hide resolved

susemanager/bin/server-migrator.sh Outdated Show resolved Hide resolved

susemanager/bin/server-migrator.sh Outdated Show resolved Hide resolved

mbussolotto and others added 4 commits May 27, 2022 10:13

Update susemanager/bin/server-migrator.sh

09d35a3

Co-authored-by: Julio González Gil <juliogonzalez@users.noreply.github.com>

Update susemanager/bin/server-migrator.sh

62c2c41

Co-authored-by: Julio González Gil <juliogonzalez@users.noreply.github.com>

Update susemanager/bin/server-migrator.sh

770c1b5

Co-authored-by: Julio González Gil <juliogonzalez@users.noreply.github.com>

Update susemanager/bin/pg-migrate-x-to-y.sh

9599b8d

Co-authored-by: Julio González Gil <juliogonzalez@users.noreply.github.com>

sbluhm suggested changes May 27, 2022

View reviewed changes

Update spacewalk.spec

b6b1d67

sbluhm approved these changes May 27, 2022

View reviewed changes

jordimassaguerpla approved these changes May 27, 2022

View reviewed changes

mcalmer approved these changes May 30, 2022

View reviewed changes

Update server-migrator.sh

d17dbba

juliogonzalez requested changes May 31, 2022

View reviewed changes

Update server-migrator.sh

7f3ee0c

mbussolotto requested a review from juliogonzalez June 1, 2022 08:03

juliogonzalez approved these changes Jun 1, 2022

View reviewed changes

juliogonzalez merged commit 31f6a86 into uyuni-project:master Jun 1, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Server migrator to 15.4 #4704

Server migrator to 15.4 #4704

mbussolotto commented Jan 18, 2022 •

edited

Loading

juliogonzalez May 25, 2022

sbluhm May 25, 2022

mbussolotto May 26, 2022

juliogonzalez May 26, 2022

juliogonzalez May 26, 2022

sbluhm May 26, 2022

sbluhm May 27, 2022

mbussolotto commented May 26, 2022

juliogonzalez commented May 26, 2022

mbussolotto commented May 26, 2022

juliogonzalez left a comment

sbluhm May 26, 2022

sbluhm May 27, 2022

sbluhm May 27, 2022

mcalmer left a comment

mcalmer May 30, 2022

juliogonzalez May 30, 2022

mbussolotto May 30, 2022 •

edited

Loading

juliogonzalez May 30, 2022

juliogonzalez May 31, 2022

mbussolotto May 31, 2022

mbussolotto May 31, 2022

Server migrator to 15.4 #4704

Server migrator to 15.4 #4704

Conversation

mbussolotto commented Jan 18, 2022 • edited Loading

What does this PR change?

GUI diff

Documentation

Test coverage

Links

Changelogs

Re-run a test

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mbussolotto commented May 26, 2022

juliogonzalez commented May 26, 2022

mbussolotto commented May 26, 2022

juliogonzalez left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mcalmer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mbussolotto May 30, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mbussolotto commented Jan 18, 2022 •

edited

Loading

mbussolotto May 30, 2022 •

edited

Loading