Use version of erlang < 1.25 and fix EL8 passlib #313

amanda11 · 2022-05-31T08:48:07Z

Use the CloudSmith rabbitmq-erlang packages rather than the LaunchPad erlang packages for Debian installations, so that we can specify a 24.x version of the rabbitmq-erlang. (25.x and 22.x are only available in LaunchPad for Debian and fail with latest rabbitMQ).

We no-longer want to use the latest 1.25 that is available in the rabbitmq-erlang repository, as this means that rabbitmq fails to install, and instead reports error:

fatal: [localhost]: FAILED! => {"attempts": 5, "cache_update_time": 1653987046, "cache_updated": false, "changed": false, "msg": "'/usr/bin/apt-get -y -o \"Dpkg::Options::=--force-confdef\" -o \"Dpkg::Options::=--force-confold\"      install 'rabbitmq-server'' failed: E: Unable to correct problems, you have held broken packages.\n", "rc": 100, "stderr": "E: Unable to correct problems, you have held broken packages.\n", "stderr_lines": ["E: Unable to correct problems, you have held broken packages."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nSome packages could not be installed. This may mean that you have\nrequested an impossible situation or if you are using the unstable\ndistribution that some required packages have not yet been created\nor been moved out of Incoming.\nThe following information may help to resolve the situation:\n\nThe following packages have unmet dependencies:\n rabbitmq-server : Depends: erlang-base (< 1:25.0) but 1:25.0-1rmq1ppa1~ubuntu18.04.1 is to be installed or\n                            erlang-base-hipe (< 1:25.0) but it is not going to be installed or\n                            esl-erlang (< 1:25.0) but it is not installable\n                   Depends: erlang-crypto (< 1:25.0) but 1:25.0-1rmq1ppa1~ubuntu18.04.1 is to be installed or\n                            esl-erlang (< 1:25.0) but it is not installable\n                   Depends: erlang-eldap (< 1:25.0) but 1:25.0-1rmq1ppa1~ubuntu18.04.1 is to be installed or\n                            esl-erlang (< 1:25.0) but it is not installable\n                   Depends: erlang-inets (< 1:25.0) but 1:25.0-1rmq1ppa1~ubuntu18.04.1 is to be installed or\n                            esl-erlang (< 1:25.0) but it is not installable\n                   Depends: erlang-mnesia (< 1:25.0) but 1:25.0-1rmq1ppa1~ubuntu18.04.1 is to be installed or\n                            esl-erlang (< 1:25.0) but it is not installable\n                   Depends: erlang-os-mon (< 1:25.0) but 1:25.0-1rmq1ppa1~ubuntu18.04.1 is to be installed or\n                            esl-erlang (< 1:25.0) but it is not installable\n                   Depends: erlang-parsetools (< 1:25.0) but 1:25.0-1rmq1ppa1~ubuntu18.04.1 is to be installed or\n                            esl-erlang (< 1:25.0) but it is not installable\n                   Depends: erlang-public-key (< 1:25.0) but 1:25.0-1rmq1ppa1~ubuntu18.04.1 is to be installed or\n                            esl-erlang (< 1:25.0) but it is not installable\n                   Depends: erlang-runtime-tools (< 1:25.0) but 1:25.0-1rmq1ppa1~ubuntu18.04.1 is to be installed or\n                            esl-erlang (< 1:25.0) but it is not installable\n                   Depends: erlang-ssl (< 1:25.0) but 1:25.0-1rmq1ppa1~ubuntu18.04.1 is to be installed or\n                            esl-erlang (< 1:25.0) but it is not installable\n                   Depends: erlang-syntax-tools (< 1:25.0) but 1:25.0-1rmq1ppa1~ubuntu18.04.1 is to be installed or\n                            esl-erlang (< 1:25.0) but it is not installable\n                   Depends: erlang-tools (< 1:25.0) but 1:25.0-1rmq1ppa1~ubuntu18.04.1 is to be installed or\n                            esl-erlang (< 1:25.0) but it is not installable\n                   Depends: erlang-xmerl (< 1:25.0) but 1:25.0-1rmq1ppa1~ubuntu18.04.1 is to be installed or\n                            esl-erlang (< 1:25.0) but it is not installable\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "Some packages could not be installed. This may mean that you have", "requested an impossible situation or if you are using the unstable", "distribution that some required packages have not yet been created", "or been moved out of Incoming.", "The following information may help to resolve the situation:", "", "The following packages have unmet dependencies:", " rabbitmq-server : Depends: erlang-base (< 1:25.0) but 1:25.0-1rmq1ppa1~ubuntu18.04.1 is to be installed or", "                            erlang-base-hipe (< 1:25.0) but it is not going to be installed or", "                            esl-erlang (< 1:25.0) but it is not installable", "                   Depends: erlang-crypto (< 1:25.0) but 1:25.0-1rmq1ppa1~ubuntu18.04.1 is to be installed or", "                            esl-erlang (< 1:25.0) but it is not installable", "                   Depends: erlang-eldap (< 1:25.0) but 1:25.0-1rmq1ppa1~ubuntu18.04.1 is to be installed or", "                            esl-erlang (< 1:25.0) but it is not installable", "                   Depends: erlang-inets (< 1:25.0) but 1:25.0-1rmq1ppa1~ubuntu18.04.1 is to be installed or", "                            esl-erlang (< 1:25.0) but it is not installable", "                   Depends: erlang-mnesia (< 1:25.0) but 1:25.0-1rmq1ppa1~ubuntu18.04.1 is to be installed or", "                            esl-erlang (< 1:25.0) but it is not installable", "                   Depends: erlang-os-mon (< 1:25.0) but 1:25.0-1rmq1ppa1~ubuntu18.04.1 is to be installed or", "                            esl-erlang (< 1:25.0) but it is not installable", "                   Depends: erlang-parsetools (< 1:25.0) but 1:25.0-1rmq1ppa1~ubuntu18.04.1 is to be installed or", "                            esl-erlang (< 1:25.0) but it is not installable", "                   Depends: erlang-public-key (< 1:25.0) but 1:25.0-1rmq1ppa1~ubuntu18.04.1 is to be installed or", "                            esl-erlang (< 1:25.0) but it is not installable", "                   Depends: erlang-runtime-tools (< 1:25.0) but 1:25.0-1rmq1ppa1~ubuntu18.04.1 is to be installed or", "                            esl-erlang (< 1:25.0) but it is not installable", "                   Depends: erlang-ssl (< 1:25.0) but 1:25.0-1rmq1ppa1~ubuntu18.04.1 is to be installed or", "                            esl-erlang (< 1:25.0) but it is not installable", "                   Depends: erlang-syntax-tools (< 1:25.0) but 1:25.0-1rmq1ppa1~ubuntu18.04.1 is to be installed or", "                            esl-erlang (< 1:25.0) but it is not installable", "                   Depends: erlang-tools (< 1:25.0) but 1:25.0-1rmq1ppa1~ubuntu18.04.1 is to be installed or", "                            esl-erlang (< 1:25.0) but it is not installable", "                   Depends: erlang-xmerl (< 1:25.0) but 1:25.0-1rmq1ppa1~ubuntu18.04.1 is to be installed or", "                            esl-erlang (< 1:25.0) but it is not installable"]}

Resolves #312

…rather than the sliced erlang packages

…of split seems to be 1:22.2.7

amanda11 · 2022-05-31T10:55:39Z

Seems to be a problem with dependencies. I have been unable to install erlang and rabbitMQ compatible verisons on U20 using the repositories we currently use (LaunchPad ones). Also affects U18.

Previously we installed version 1:24.3.4-1rmq1ppa1~ubuntu20.04.1 when it was working successfully - for erlang-base etc. But this version doesn't seem to be available any more on the LaunchPad apt repository.
The latest erlang-base etc version I can find is 1:22....
If you use the older erlang-base then you can't install the latest rabbitMQ as it requires erlang >= 1:23.2

So either:

we need to find the erlang-base, erlang-asn1 etc 1:24 versions we used before. But I've not managed to find it in the repos we set:

$ apt list erlang-base -a
Listing... Done
erlang-base/focal 1:25.0-1rmq1ppa1~ubuntu20.04.1 amd64 [upgradable from: 1:22.2.7+dfsg-1]
erlang-base/focal,now 1:22.2.7+dfsg-1 amd64 [installed,upgradable to: 1:25.0-1rmq1ppa1~ubuntu20.04.1]

Have managed to get compatible set of erlang repos from "erlang solutions" repos, see https://www.rabbitmq.com/which-erlang.html#erlang-repositories
If we use this repo and install version 1:24.3.3-1, then I was able to install rabbitmq-server

Also the older erlang repositories seem to be available at their cloudsmith repos:
https://cloudsmith.io/~rabbitmq/repos/rabbitmq-erlang/packages/?q=erlang-base

I've found the erlang group package 1:24.3.4-1rmq1ppa1~ubuntu22.04.1 is available at https://launchpad.net/~rabbitmq/+archive/ubuntu/rabbitmq-erlang/+packages?field.name_filter=&field.status_filter=published&field.series_filter=
But if I attempt to install that it complains about missing dependencies - so that doesn't seem to work
If we go back to the earlier erlang-base etc that is in the LaunchPad repository then we would have to rollback rabbitMQ version as well - and ideally we don't really want to rollback erlang and rabbitmq versions...

amanda11 · 2022-05-31T11:59:43Z

Using cloudsmith repos instead of LaunchPad allows us to install erlang version that works with rabbitmq
Using pip install of passlib on EL8 resolves the failing to find passlib for python3.8

arm4b · 2022-05-31T16:10:03Z

roles/StackStorm.st2/tasks/auth.yml

+- name: auth | Install pip pre-reqs (RedHat)
+  become: yes
+  pip:
+     name: passlib


Could we install the passlib via pip for all the OS flavors to get rid of this kind of mishmash?

ansible-st2/roles/StackStorm.st2/vars/debian.yml

Lines 1 to 3 in 02b5ffb

# List of python variables

---

passlib: python-passlib

ansible-st2/roles/StackStorm.st2/vars/redhat_8.yml

Lines 1 to 3 in 02b5ffb

# List of python3 variables

---

passlib: python3-passlib

ansible-st2/roles/StackStorm.st2/vars/redhat.yml

Lines 1 to 3 in 02b5ffb

# List of python2 variables

---

passlib: python-passlib

ansible-st2/roles/StackStorm.st2/vars/debian_20.yml

Lines 1 to 3 in 02b5ffb

# List of python3 variables

---

passlib: python3-passlib

Wondering if it'll make it any better or if we'll hit another corner case with the environment.

I think we'll still need to mess about a bit, as the pip executable differs. So we have to point to pip3 on EL8, but we'll need to point to different pip versions on some of the other installs. (I tried with just the default pip module and no executable name - but that failed :-( )
So we'll just be replacing the passlib variable with a pip variable.

But might make it a little tidier. I'll give it a quick run on focal and EL8....

Thanks for the efforts!
Approved the PR. Feel free to merge it with the fixes you'll find tidier.

@armab We don't install default pip at moment for U20 as part of ansible install. So we'd need to add that in as well.
I'm not sure its going to make it any cleaner, just mean we'll have another pip on the box to perhaps get confused with....

Either way looks like a rake.
Alright, thanks for checking.

LGTM 👍

arm4b

Thanks for the fixes! 👍

Use version of erlang < 1.25 and use the packaged rabbitmt mq erlang …

158c634

…rather than the sliced erlang packages

pull-request-size bot added the size/S PR that changes 10-29 lines. Very easy to review. label May 31, 2022

amanda11 marked this pull request as draft May 31, 2022 08:48

pull-request-size bot added size/M PR that changes 30-99 lines. Good size to review. and removed size/S PR that changes 10-29 lines. Very easy to review. labels May 31, 2022

amanda11 added 4 commits May 31, 2022 09:56

Missed task change

503d310

Fails to install erlang with just the grouped erlang. Latest version …

a53d55d

…of split seems to be 1:22.2.7

Fix wrong version number

6942684

Amend order of install

feb95ac

amanda11 and others added 3 commits May 31, 2022 12:26

Attempt install using CloudSmith erlang and rabbitMQ

5f2dadf

Use pip passlib for RH to resolve EL8 python38-passlib issues

76da6b8

Fix lint errors

d4ecc23

amanda11 added this to In progress in StackStorm v3.8.0 via automation May 31, 2022

Use python passlib on EL7 and pip on EL8

5586876

amanda11 and others added 3 commits May 31, 2022 12:15

Fix El7

b6ae985

Install of python3-passlib for EL8

1dde4d5

Fix lint

a8061bc

amanda11 marked this pull request as ready for review May 31, 2022 14:09

amanda11 changed the title ~~[WIP] Use version of erlang < 1.25~~ Use version of erlang < 1.25 and fix EL8 passlib May 31, 2022

arm4b reviewed May 31, 2022

View reviewed changes

arm4b approved these changes May 31, 2022

View reviewed changes

amanda11 merged commit e4cec0f into master May 31, 2022

StackStorm v3.8.0 automation moved this from In progress to Done May 31, 2022

amanda11 deleted the fix_ci branch May 31, 2022 22:21

arm4b mentioned this pull request Jun 7, 2022

TSC Meeting (07 Jun 2022): Exchange "!bootstrap pack" demo, AWS disaster plan, v3.7.0 retrospective, v3.8.0 planning StackStorm/community#102

Closed

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use version of erlang < 1.25 and fix EL8 passlib #313

Use version of erlang < 1.25 and fix EL8 passlib #313

amanda11 commented May 31, 2022 •

edited

amanda11 commented May 31, 2022 •

edited

amanda11 commented May 31, 2022

arm4b May 31, 2022 •

edited

amanda11 May 31, 2022 •

edited

arm4b May 31, 2022

amanda11 May 31, 2022

arm4b May 31, 2022

arm4b left a comment

Use version of erlang < 1.25 and fix EL8 passlib #313

Use version of erlang < 1.25 and fix EL8 passlib #313

Conversation

amanda11 commented May 31, 2022 • edited

amanda11 commented May 31, 2022 • edited

amanda11 commented May 31, 2022

arm4b May 31, 2022 • edited

Choose a reason for hiding this comment

amanda11 May 31, 2022 • edited

Choose a reason for hiding this comment

arm4b May 31, 2022

Choose a reason for hiding this comment

amanda11 May 31, 2022

Choose a reason for hiding this comment

arm4b May 31, 2022

Choose a reason for hiding this comment

arm4b left a comment

Choose a reason for hiding this comment

amanda11 commented May 31, 2022 •

edited

amanda11 commented May 31, 2022 •

edited

arm4b May 31, 2022 •

edited

amanda11 May 31, 2022 •

edited