Replace ceph-ansible by Cephadm

[dupremathieu]

Abstract

Cephadm is tools to configure and deploy SEAPATH: https://docs.ceph.com/en/quincy/cephadm/.
Cephadm relies on containers. With the cephadm-ansible collection, it replaces ceph-ansible.

Note that ceph-ansible we actually used to configure Ceph will no longer get new features and will not support the future Ceph version (but it is still maintained).

Current status

• Ceph is installed directly inside SEAPATH images.
• Ceph version differs between SEAPATH variants.
• In both variants, the Ceph version is quite old and will be deprecated soon.
• We use ceph-ansible to setup Ceph
• ceph-ansible version differs between SEAPATH variants (because it has to follow the Ceph version).
• ceph-ansible requires an old version of Ansible to be used.

Detailed Description

The biggest change of cephadm-ansible regarding ceph-ansible is cephadm only supports containers. The installation will be done in two steps:

• first, we install ceph only on one machine of the cluster
• after cephadm extends the installation to other machines We also need to setup a registry to store Ceph images containers. This registry can be one machine of the cluster or inside an external machine. It can be removed after the installation.

Benefit to SEAPATH

• This is the recommended way to configure and deploy Ceph.
• The Ceph version will no longer be linked to the distribution.
• The security will be enhanced with the container isolation used by Cephadm.
• Greatly reduce the Yocto variant generation times.
• The Ansible version will be no longer restricted by ceph-ansible
• Easier Ceph update which is now managed by Cephadm

Works to be done

• Remove Ceph from SEAPATH images.
• Modify our Ansible files to use Cephadm-ansible instead of ceph-ansible.
• Use Ansible to create a local registry to push Ceph container images on it to allow offline installation

[insatomcat]

Very interesting and we will definitely have to work on this at some point.
I have 2 questions:

Modify our Ansible files to use Cephadm-ansible instead of ceph-ansible

The description of the cephadm-ansible project states "cephadm-ansible is a collection of Ansible playbooks to simplify workflows that are NOT covered by cephadm"
Not quite sure what it means, but it looks like it's quite limited, and does not help create the cluster for example.

Use Ansible to create a local registry to push Ceph container images on it to allow offline installation

In our current philosophy, the build of the installation media is the only time we need to be online.
The ansible part does currently not require an internet connection.
But if I understand your idea, we would need to run ansible playbooks on an internet connected machine (so that it can create the registry), so it kinds of defeats the "offline deployment" objective, or at least complicates it a lot (we would need the ansible server to be online to get the docker images, and then offline for deployment ...?).

[dupremathieu]

For the first point, I do not understand what it means to.

For the offline connection, I see three solutions.

• The user have already an external registry accessible by SEAPATH machine
  • In this case, we can push ceph image on it
• Embedded containers images inside SEAPATH images during the build
  • it can be done with docker image save
  • During the Ansible step, we create a registry inside one SEAPATH machine and push these images on it
• Same as previous, except the fetching is done inside the Ansible machine just after fetching Ansible files, for instance.

Note that maybe if we are lucky, there is no need to have a registry if we already have the images on the machine. In that case, we can just use docker image load to install the images and skip the registry part.