Rocky Linux 9 IDR upgrade

[sbesson]

This PR contains the changes require to deploy an IDR system onto Rocky Linux 9. Most importantly, the Ansible requirements are updated to depend on the latest versions of the OME roles adjusted to supported Rocky Linux 9.

The following Ansible roles are upgraded

• ome.analysis_tools 1.0.1 -> 1.1.0 (diff)
• ome.anonymous_ftp 0.1.4 -> 0.2.0 (diff)
• ome.basedeps 1.1.0 -> 1.3.1 (diff)
• ome.cadvisor 0.3.3 -> 0.5.0 (diff)
• ome.cli_utils 1.1.1 -> 1.2.5 (diff)
• ome.deploy_archive 0.1.4 -> 0.2.0 (diff)
• ome.docker 3.1.1 -> 3.2.2 (diff)
• ome.hosts_populate 0.2.3 -> 0.3.0 (diff)
• ome.ice 4.3.0 -> 4.4.4 (diff)
• ome.iptables_raw 0.3.1 -> 0.4.0 (diff)
• ome.java 2.1.0 -> 2.2.0 (diff)
• ome.local_accounts 1.1.0 -> 1.2.0 (diff)
• ome.lvm_partition 1.1.1 -> 1.2.0 (diff)
• ome.network_cloud_interfaces 1.2.4 -> 1.3.0 (diff)
• ome.network 1.1.4 -> 1.2.0 (diff)
• ome.nfs_mount 1.3.0 -> 1.4.0 (diff)
• ome.nfs_share 1.0.4 -> 1.1.0 (diff)
• ome.nginx 2.1.2 -> 2.2.1 (diff)
• ome.nginx_proxy 1.15.1 -> 1.16.0 (diff)
• ome.omero_common 0.3.4 -> 0.4.0 (diff)
• ome.omero_logmonitor 3.0.1 -> 3.1.0 (diff)
• ome.omero_prometheus_exporter 0.3.6 -> 0.4.0 (diff)
• ome.omero_server 4.1.0 -> 6.0.0 (diff)
• ome.omero_user 0.3.1 -> 0.4.0 (diff)
• ome.omero_web 4.0.1 -> 5.1.1 (diff)
• ome.postgresql 5.2.0 -> 5.4.0 (diff)
• ome.postgresql_client 0.2.0 -> 0.4.3 (diff)
• ome.python3_virtualenv 0.1.2 -> 0.2.0 (diff)
• ome.reboot_server 0.1.3 -> 0.2.0 (diff)
• ome.redis 1.1.1 -> 1.3.0 (diff)
• ome.selinux_utils 2.0.2 -> 2.1.1 (diff)
• ome.ssl_certificate 0.3.3 -> 0.5.0 (diff)
• ome.storage_volume_initialise 1.0.2 -> 1.1.0 (diff)
• ome.sudoers 1.0.4 -> 1.1.0 (diff)
• ome.upgrade_distpackages 1.1.3 -> 1.2.0 (diff)
• ome.versioncontrol_utils 1.0.2 -> 1.1.0 (diff)
• ome.fluentd 0.2.3 -> 0.3.1 (diff)
• ome.prometheus 0.5.1 -> 0.5.0 (diff)
• ome.prometheus_jmx 0.3.1 -> 0.4.0 (diff)
• ome.prometheus_node 0.3.1 -> 0.4.0 (diff)
• ome.prometheus_postgres 0.4.2 -> 0.5.0 (diff)

A few additional changes were required:

• the ome.python_pydata role and its usage is removed
• the ome.minio_s3_gateway role and its usage is removed
• all remaining Python 3.6 references are cleaned up
• various OMERO plugins were upgraded
• omero_server_selfsigned_certificates is removed as it default to True
• omero_web_setup_redis_session is set to True
• the Molecule tests are updated to use Rocky Linux 9 images with systemd similar to the upstream roles and the FTP TCP tests are adjusted

As discussed during the IDR weekly meeting, this PR takes the advantage of the Python 3.9 deployment to also upgrades OMERO.web to 5.25.0 and the associated OMERO.web apps:

• OMERO.mapr to 0.5.2
• OMERO.iviewer to 0.13.0
• OMERO.figure to 6.2.0

A few additional role fixes still need to be reviewed, merged and released:

• Become root before importing the RPM key ome/ansible-role-cli-utils#11
• Become root before importing the RPM key ome/ansible-role-postgresql-client#16
• Become root ome/ansible-role-ice#28
• Import ome.deploy_archive role with privilege escalation ome/ansible-role-ice#29
• Become root ome/ansible-role-nginx#15
• Become root ome/ansible-role-omero-web#50
• Become root ome/ansible-role-docker#17
• No 'firewalld' on Rocky9 ome/ansible-role-iptables-raw#12
• Add support for Rocky Linux 9 ome/ansible-role-prometheus#14

[sbesson]

From the Molecule perspective everything looks to be working with the set of upgraded Ansible roles - thanks @pwalczysko @khaledk2 and @jburel for all the underlying maintenance work.

Next step will be to deploy these changes onto a dedicate pilot VM created with Rocky Linux 9.

[dominikl]

👍 I'll try to spin up a pilot with rocky9.

[dominikl]

Spun up pilot-idr0159 using this PR and https://github.com/openmicroscopy/management_tools/pull/1733 .
The pilot provisioning steps ran fine, but updating the pilot proxy (./playbooks/deploy-idr.sh pilot deploy) fails with

...
[WARNING]: Unhandled error in Python interpreter discovery for host 0c25d71f-5e26-49dc-9cb0-be3c3d85cb7c: exceptions.IOError: Unable to find a useable temporary directory. This
likely means no system-supplied TMP directory can be written to, or all directories were mounted on 'noexec' filesystems.  The following paths were tried:
/home/rocky/.ansible/tmp     /var/tmp     /tmp     /tmp     /var/tmp     /usr/tmp     /home/rocky  Please check '-vvv' output for a log of individual path errors.   File "<stdin>",
line 3706, in _dispatch_one   File "master:/Users/dom/PR_Testing/management_tools/mitogen/ansible_mitogen/target.py", line 376, in init_child     good_temp_dir =
find_good_temp_dir(candidate_temp_dirs)   File "master:/Users/dom/PR_Testing/management_tools/mitogen/ansible_mitogen/target.py", line 329, in find_good_temp_dir     'paths': '\n
'.join(paths),
An exception occurred during task execution. To see the full traceback, use -vvv. The error was:     'paths': '\n    '.join(paths),
0c25d71f-5e26-49dc-9cb0-be3c3d85cb7c | FAILED! => {
    "msg": "Unexpected failure during module execution.",
    "stdout": ""
}
Failed to ping 'pilot-*' hosts
Typical causes of failure include:
- mismatch or missing SSH key in .ssh/known_hosts for the proxy
- networking configuration that is incompatible with the dynamic inventory
Aborting

[pwalczysko]

I am testing this PR in unison with https://github.com/openmicroscopy/management_tools/pull/1733 - please see my comments there, thanks.

[sbesson]

As a brief summary of the deployment effort over the last few month:

• the updated Ansible roles were successfully used to deploy the next production IDR environment prod121 on Rocky Linux 9. No usability concern has been raised by the IDR team in the last few weeks
• @pwalczysko and @dominikl independently established they could deploy a pilot VM on Rocky Linux 9 with these changes
• the known outstanding upgrade issues are:
  • migrate the pilot infrastructure by upgrading the pilot proxy and management VMs to Rocky Linux
  • migrate the FTP service to Rocky Linux 9

Both remaining issues are outside the scope of this original PR and should rather build on top of this. I'll leave until tomorrow 10am for any last minute objection and then will integrate this PR.