diff --git a/README.rst b/README.rst index a2d0f0625..31e714af0 100644 --- a/README.rst +++ b/README.rst @@ -5,14 +5,9 @@ StackHPC Kayobe Configuration This repository provides a base Kayobe configuration for the Yoga release of StackHPC OpenStack. -Documentation for this project is provided in the ``doc`` directory. To build -it, run: - -.. code-block:: console - - tox -e docs - -The HTML will be written to ``doc/build``. +Documentation is hosted on `readthedocs.io +`__, +and includes release notes. Resources ========= diff --git a/doc/requirements.txt b/doc/requirements.txt index 842c709a9..650e0def0 100644 --- a/doc/requirements.txt +++ b/doc/requirements.txt @@ -2,6 +2,6 @@ # of appearance. Changing the order has an impact on the overall integration # process, which may cause wedges in the gate later. -reno>=3.1.0 # Apache-2.0 +reno>=3.4.0 # Apache-2.0 sphinx>=4.2.0 # BSD sphinxcontrib-svg2pdfconverter>=0.1.0 # BSD diff --git a/doc/source/_static/images/release-train.svg b/doc/source/_static/images/release-train.svg new file mode 100644 index 000000000..aaf2f2303 --- /dev/null +++ b/doc/source/_static/images/release-train.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/doc/source/conf.py b/doc/source/conf.py index f7214fed0..0c68c0a88 100644 --- a/doc/source/conf.py +++ b/doc/source/conf.py @@ -26,12 +26,29 @@ # See the License for the specific language governing permissions and # limitations under the License. +# -- StackHPC Kayobe configuration -------------------------------------- +# Variables to override + +current_series = "yoga" +branch = f"stackhpc/{current_series}" + +# Substitutions loader +rst_epilog = """ +.. |current_release| replace:: {current_release} +.. |current_release_git_branch_name| replace:: {current_release_git_branch_name} +""".format( # noqa: E501 + current_release_git_branch_name=branch, + current_release=current_series, +) + # -- General configuration ---------------------------------------------------- # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones. extensions = [ + 'reno.sphinxext', #'sphinx.ext.autodoc', + 'sphinx.ext.extlinks', #'sphinx.ext.intersphinx', 'sphinxcontrib.rsvgconverter', ] @@ -91,3 +108,13 @@ # Disable usage of xindy https://bugzilla.redhat.com/show_bug.cgi?id=1643664 latex_use_xindy = False +extlinks_projects = { + "kayobe", + "kolla", + "kolla-ansible", +} + +extlinks = { + f"{project}-doc": (f"https://docs.openstack.org/{project}/{current_series}/", "%s documentation") + for project in extlinks_projects +} diff --git a/doc/source/configuration/cephadm.rst b/doc/source/configuration/cephadm.rst index f966b4997..e207dd5b7 100644 --- a/doc/source/configuration/cephadm.rst +++ b/doc/source/configuration/cephadm.rst @@ -3,7 +3,7 @@ Cephadm & Kayobe ================ This section describes how to use the Cephadm integration included in StackHPC -Kayobe configuration since Yoga to deploy Ceph. +Kayobe configuration since Xena to deploy Ceph. The Cephadm integration takes the form of custom playbooks that wrap around the Ansible `stackhpc.cephadm collection diff --git a/doc/source/configuration/host-images.rst b/doc/source/configuration/host-images.rst new file mode 100644 index 000000000..cfa5fe678 --- /dev/null +++ b/doc/source/configuration/host-images.rst @@ -0,0 +1,66 @@ +.. _host-images: + +=========== +Host Images +=========== + +StackHPC Kayobe configuration provides configuration for some standard +overcloud host images, built using the :kayobe-doc:`overcloud DIB +` functionality of Kayobe. + +The overcloud DIB configuration is provided in +``etc/kayobe/stackhpc-overcloud-dib.yml``. It is not used by default, and must +be actively opted into. This can be done as follows: + +.. code-block:: yaml + :caption: ``etc/kayobe/overcloud-dib.yml`` + + overcloud_dib_build_host_images: true + + overcloud_dib_host_images: + - "{{ stackhpc_overcloud_dib_host_image }}" + +The image name is configured via ``stackhpc_overcloud_dib_name``, and is +``deployment_image`` by default. + +The list of DIB elements is configured via ``stackhpc_overcloud_dib_elements``. +The default value depends on the ``os_distribution`` variable. See the YAML +file for details. + +The DIB environment variables are configured via +``stackhpc_overcloud_dib_env_vars``. See the YAML file for details. + +A list of packages to install is configured via +``stackhpc_overcloud_dib_packages``. + +By default, a UEFI-compatible image is built that uses separate LVM volumes for +different mount points. This is done to pass Centre for Internet Security (CIS) +partition benchmarks. The block device YAML configuration is configured via +``stackhpc_overcloud_dib_block_device_config_uefi_lvm``. + +The 3 partitions are: + +* p0: EFI ESP bootloader +* p1: EFI BSP +* p2: LVM PV (``rootpv``) + +The LVM Logical Volumes are: + +============== ================== ========= +LV Mount point Size (GB) +============== ================== ========= +``lv_root`` ``/`` 5G +``lv_tmp`` ``/tmp`` 1G +``lv_var`` ``/var`` 1G +``lv_var_tmp`` ``/var/tmp`` 1G +``lv_log`` ``/var/log`` 1G +``lv_audit`` ``/var/log/audit`` 128M +``lv_home`` ``/home`` 128M +============== ================== ========= + +A compatible LVM configuration is provided, and covered in :ref:`lvm`. +The Logical Volumes in the image are defined with small sizes, with the +intention that they will be grown after provisioning. + +For RedHat family distributions, Dracut modules are configured via +``stackhpc_overcloud_dib_dracut_enabled_modules_default_config``. diff --git a/doc/source/configuration/index.rst b/doc/source/configuration/index.rst index 74d145e29..ebaab804e 100644 --- a/doc/source/configuration/index.rst +++ b/doc/source/configuration/index.rst @@ -10,5 +10,8 @@ the various features provided. walled-garden release-train + host-images + lvm + swap cephadm monitoring diff --git a/doc/source/configuration/lvm.rst b/doc/source/configuration/lvm.rst new file mode 100644 index 000000000..65b892aa8 --- /dev/null +++ b/doc/source/configuration/lvm.rst @@ -0,0 +1,98 @@ +.. _lvm: + +=== +LVM +=== + +StackHPC Kayobe configuration provides Logical Volume Manager (LVM) +configuration that is compatible with the included :ref:`host-images` +configuration. The configuration uses the :kayobe-doc:`LVM +` host configuration functionality of +Kayobe. + +The LVM configuration is provided in +``etc/kayobe/inventory/group_vars/all/stackhpc/lvm``. This allows configuration +variables to be overridden on a per-group or per-host basis (which would not be +possible for an "extra variable" in ``etc/kayobe/*.yml``). This configuration +is not used by default, and must be actively opted into. This can be done as +follows: + +.. code-block:: yaml + + controller_lvm_groups: + - "{{ stackhpc_lvm_group_rootvg }}" + +This will configure the standard set of logical volumes for the ``rootvg`` +volume group on controller hosts. + +The disks in this volume group are configured via +``stackhpc_lvm_group_rootvg_disks``, and by default this contains a single +disk, matched by a partition label of ``root`` (as used in the standard +:ref:`host-images`). + +The size of each LV is configurable via the following variables: + +.. code-block:: yaml + + # StackHPC LVM lv_swap LV size. + stackhpc_lvm_lv_swap_size: 16g + + # StackHPC LVM lv_root LV size. + stackhpc_lvm_lv_root_size: 50g + + # StackHPC LVM lv_tmp LV size. + stackhpc_lvm_lv_tmp_size: 10g + + # StackHPC LVM lv_var LV size. + stackhpc_lvm_lv_var_size: 20g + + # StackHPC LVM lv_var_tmp LV size. + stackhpc_lvm_lv_var_tmp_size: 2g + + # StackHPC LVM lv_log LV size. + stackhpc_lvm_lv_log_size: 20g + + # StackHPC LVM lv_audit LV size. + stackhpc_lvm_lv_audit_size: 10g + + # StackHPC LVM lv_home LV size. + stackhpc_lvm_lv_home_size: 10g + +Additional LVs may be configured via ``stackhpc_lvm_group_rootvg_lvs_extra``. A +common requirement is to have ``/var/lib/docker/`` mounted on a separate LV, +so this has been made convenient to achieve: + +.. code-block:: yaml + + stackhpc_lvm_group_rootvg_lvs_extra: + - "{{ stackhpc_lvm_lv_docker }}" + + # StackHPC LVM lv_docker LV size. + stackhpc_lvm_lv_docker_size: 100%FREE + +It may be desirable to use a lower percentage of the free space, in case +another LV needs to be grown at a later date. + +Growroot playbook +================= + +A ``growroot.yml`` custom playbook is provied that can be used to grow the +partition and LVM Physical Volume (PV) of the root Volume Group (VG). This +allows for expansion of Logical Volumes (LVs) in that VG. + +The following variables may be used to configure the playbook: + +``growroot_group`` + Host pattern against which to target the playbook. Default is ``overcloud``. +``growroot_vg`` + Name of the VG containing the PV to grow. Default is ``rootvg`` to match the + standard :ref:`host image configuration `. + +This playbook may be used as a host configure pre hook, e.g. for overcloud +hosts: + +.. code-block:: console + + mkdir -p ${KAYOBE_CONFIG_PATH}/hooks/overcloud-host-configure/pre.d + cd ${KAYOBE_CONFIG_PATH}/hooks/overcloud-host-configure/pre.d + ln -s ../../../ansible/growroot.yml 30-growroot.yml diff --git a/doc/source/configuration/monitoring.rst b/doc/source/configuration/monitoring.rst index f358ea084..6045f3c29 100644 --- a/doc/source/configuration/monitoring.rst +++ b/doc/source/configuration/monitoring.rst @@ -1,13 +1,29 @@ -=========== +========== Monitoring -=========== +========== Monitoring Configuration ======================== -StackHPC kayobe config includes a reference monitoring stack based on -Prometheus. Whilst this often works out of the box, there are some tunables -which can be customised to adapt the configuration to a particular deployment. +StackHPC kayobe config includes a reference monitoring and alerting stack based +on Prometheus, Alertmanager, Grafana, Fluentd, Elasticsearch & Kibana. These +services by default come enabled and configured. Central Elasticsearch cluster +collects OpenStack logs, with an option to receive operating system logs too. +In order to enable this, execute custom playbook after deployment: + +.. code-block:: console + + cd $KAYOBE_CONFIG_PATH + kayobe playbook run ansible/rsyslog.yml + +`Prometheus `__ comes with a comprehensive set of +metrics gathered from enabled exporters; every exporter's data is visualised +by at least one `Grafana `__ dashboard. Standard set of +alerting rules is present as well. + +While the default configuration often works out of the box, there +are some tunables which can be customised to adapt the configuration to a +particular deployment's needs. The configuration options can be found in ``etc/kayobe/stackhpc-monitoring.yml``: @@ -15,8 +31,19 @@ The configuration options can be found in .. literalinclude:: ../../../etc/kayobe/stackhpc-monitoring.yml :language: yaml +In order to enable stock monitoring configuration within a particular +environment, create the following symbolic links: + +.. code-block:: console + + cd $KAYOBE_CONFIG_PATH + ln -s kolla/config/grafana/ environments/$KAYOBE_ENVIRONMENT/kolla/config/ + ln -s kolla/config/prometheus/ environments/$KAYOBE_ENVIRONMENT/kolla/config/ + +and commit them to the config repository. + SMART Drive Monitoring -======================= +====================== StackHPC kayobe config also includes drive monitoring for spinning disks and NVME's. @@ -27,7 +54,7 @@ a cronjob, to output the metrics and we use node exporter's Textfile collector to report the metrics output by the scripts to Prometheus. These metrics can then be visualised in Grafana with the bundled dashboard. -After pulling in the latest changes into your local kayobe config, reconfigure +After pulling in the latest changes into your local kayobe config, reconfigure Prometheus and Grafana .. code-block:: console @@ -38,7 +65,7 @@ Prometheus and Grafana `this `__ bug and at present, the workaround is to go into each node running Grafana and manually restart the process with ``docker restart grafana`` and then try the reconfigure -command again.)  +command again.) Once the reconfigure has completed you can now run the custom playbook which copies over the scripts and sets up the cron jobs to start SMART monitoring @@ -50,6 +77,4 @@ on the overcloud hosts: (kayobe) [stack@node kayobe]$ kayobe playbook run ansible/smartmontools.yml SMART reporting should now be enabled along with a Prometheus alert for -unhealthy disks and a Grafana dashboard called ``Hardware Overview``.  - - +unhealthy disks and a Grafana dashboard called ``Hardware Overview``. diff --git a/doc/source/configuration/release-train.rst b/doc/source/configuration/release-train.rst index 20b43d674..af1e7b3f9 100644 --- a/doc/source/configuration/release-train.rst +++ b/doc/source/configuration/release-train.rst @@ -3,33 +3,52 @@ StackHPC Release Train ====================== StackHPC provides packages and container images for OpenStack via `Ark -`__. +`__. These artifacts are built and released using a +process known as the `Release Train +`__. Deployments should use a local `Pulp `__ repository -server to synchronise content from Ark and serve it locally. Access to the -repositories on Ark is controlled via user accounts issued by StackHPC. +server to synchronise content from Ark and serve it locally. This reduces +Internet bandwidth requirements for package and container downloads. Content is +synced on demand from Ark to the local Pulp, meaning that the local Pulp acts +like a pull-through cache. -This configuration is a base, and should be merged with any existing Kayobe -configuration. It currently provides the following: +Access to the repositories on Ark is controlled via user accounts issued by +StackHPC. -* Configuration to deploy a local Pulp service as a container on the seed -* Pulp repository definitions for CentOS Stream 8 -* Playbooks to synchronise a local Pulp service with Ark -* Configuration to use the local Pulp repository mirrors on control plane hosts -* Configuration to use the local Pulp container registry on control plane hosts +.. image:: /_static/images/release-train.svg + :width: 75% + +All content on Ark is versioned, meaning that a deployment may continue to use +older package repository snapshots and container images when newer content is +released. This allows for improved reliability & repeatability of deployments. This configuration defines two `Pulp distributions `__ for packages, ``development`` and ``production``. This allows packages to be updated and tested in a development or staging environment before rolling them -out to production. +out to production. Typically a given environment will always use the same +distribution, meaning that package repository configuration files do not need +to be updated on control plane hosts in order to consume a package update. Configuration ============= +This configuration provides the following: + +* Configuration to deploy a local Pulp service as a container on the seed +* Pulp repository definitions for CentOS Stream 8 and Rocky Linux 8 +* Playbooks to synchronise a local Pulp service with Ark +* Configuration to use the local Pulp repository mirrors on control plane hosts +* Configuration to use the local Pulp container registry on control plane hosts + Local Pulp server ----------------- +The Pulp container is deployed on the seed by default, but may be disabled by +setting ``seed_pulp_container_enabled`` to ``false`` in +``etc/kayobe/seed.yml``. + The URL and credentials of the local Pulp server are configured in ``etc/kayobe/pulp.yml`` via ``pulp_url``, ``pulp_username`` and ``pulp_password``. In most cases, the default values should be sufficient. @@ -38,6 +57,9 @@ An admin password must be generated and set as the value of a ``etc/kayobe/secrets.yml`` file. This password will be automatically set on Pulp startup. +If a proxy is required to access the Internet from the seed, ``pulp_proxy_url`` +may be used. + StackHPC Ark ------------ @@ -49,22 +71,60 @@ The Ark pulp credentials issued by StackHPC should be configured in stackhpc_release_pulp_username: stackhpc_release_pulp_password: +Package repositories +-------------------- + +Currently, Ark does not provide package repositories for Ubuntu - only +container images. For this reason, ``stackhpc_pulp_sync_ubuntu_focal`` in +``etc/kayobe/pulp.yml`` is set to ``false`` by default. + +CentOS Stream 8 and Rocky Linux 8 package repositories are synced based on the +value of ``os_distribution``. If you need to sync multiple distributions, +``stackhpc_pulp_sync_centos_stream8`` and ``stackhpc_pulp_sync_rocky_8`` in +``etc/kayobe/pulp.yml`` may be set to ``true``. + +On Ark, each package repository provides versioned snapshots using a datetime +stamp (e.g. ``20220817T082321``). The current set of tested versions is defined +in ``etc/kayobe/pulp-repo-versions.yml``. This file is managed by the StackHPC +Release Train and should generally not be modified by consumers of this +repository. + +Package managers +---------------- + +No configuration is provided for APT, since Ark does not currently provide +package repositories for Ubuntu - only container images. + +For CentOS and Rocky Linux based systems, package manager configuration is +provided by ``stackhpc_dnf_repos`` in ``etc/kayobe/dnf.yml``, which points to +package repositories on the local Pulp server. To use this configuration, the +``dnf_custom_repos`` variable must be set, and this is done for hosts in the +``overcloud`` group via the group_vars file +``etc/kayobe/inventory/group_vars/overcloud/stackhpc-dnf-repos``. Similar +configuration may be added for other groups, however there may be ordering +issues during initial deployment when Pulp has not yet been deployed. + The distribution name for the environment should be configured as either ``development`` or ``production`` via ``stackhpc_repo_distribution`` in ``etc/kayobe/stackhpc.yml``. +Ceph container images +--------------------- + +By default, Ceph images are not synced from quay.io to the local Pulp. To sync +these images, set ``stackhpc_sync_ceph_images`` to ``true``. + Usage ===== -The local Pulp service will be deployed as a `Seed custom container -`__ +The local Pulp service will be deployed as a :kayobe-doc:`Seed custom container +` on next ``kayobe seed service deploy`` or ``kayobe seed service upgrade``. The following custom playbooks are provided in ``etc/kayobe/ansible/``: -See the Kayobe `custom playbook documentation -`__ -for information on how to run them. +See the Kayobe :kayobe-doc:`custom playbook documentation +` for information on how to run them. * ``pulp-repo-sync.yml``: Pull packages from Ark to the local Pulp. This will create a new repository version (snapshot) for each repository in the local @@ -91,6 +151,39 @@ for information on how to run them. local Pulp. This will make synchonised container images available to cloud nodes. +Syncing content +--------------- + +A typical workflow to sync all packages and containers is as follows: + +.. code-block:: console + + kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/pulp-repo-sync.yml + kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/pulp-repo-publish.yml + kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/pulp-container-sync.yml + kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/pulp-container-publish.yml + +Once the content has been tested in a test/staging environment, it may be +promoted to production: + +.. code-block:: console + + kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/pulp-repo-promote-production.yml + +Initial seed deployment +----------------------- + +During the initial seed deployment, there is an ordering issue where the +Bifrost container will not yet have been synced, but the local Pulp container +has not yet been deployed. This can be avoided with the following workflow: + +.. code-block:: console + + kayobe seed service deploy --tags seed-deploy-containers --kolla-tags none + kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/pulp-container-sync.yml + kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/pulp-container-publish.yml + kayobe seed service deploy + Working with pulp ================= @@ -149,7 +242,7 @@ with the push repository using the pulp CLI: .Done. HTTP Error 404: Not Found -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +~~~~~~~~~~~~~~~~~~~~~~~~~ If your login credentials are incorrect, or lack the required permissions, you will see a 404 error during ``pulp-repo-sync.yml``: diff --git a/doc/source/configuration/swap.rst b/doc/source/configuration/swap.rst new file mode 100644 index 000000000..20bae501b --- /dev/null +++ b/doc/source/configuration/swap.rst @@ -0,0 +1,23 @@ +==== +Swap +==== + +StackHPC Kayobe configuration provides a ``swap.yml`` custom playbook that may +be used to configure a swap device. + +The following variables may be used to configure the playbook: + +``swap_group`` + Host pattern against which to target the playbook. Default is ``overcloud``. +``swap_device`` + Name of the swap device to configure. Default is ``/dev/rootvg/lv_swap`` to + match the standard :ref:`host image configuration `. + +This playbook may be used as a host configure post hook, e.g. for overcloud +hosts: + +.. code-block:: console + + mkdir -p ${KAYOBE_CONFIG_PATH}/hooks/overcloud-host-configure/post.d + cd ${KAYOBE_CONFIG_PATH}/hooks/overcloud-host-configure/post.d + ln -s ../../../ansible/swap.yml 10-swap.yml diff --git a/doc/source/contributor/documentation.rst b/doc/source/contributor/documentation.rst new file mode 100644 index 000000000..b8a263f6c --- /dev/null +++ b/doc/source/contributor/documentation.rst @@ -0,0 +1,20 @@ +.. _documentation: + +============= +Documentation +============= + +This documentation is provided in the ``doc`` directory. To build it, run: + +.. code-block:: console + + tox -e docs + +The HTML will be written to ``doc/build``, and may be viewed in a web browser. + +Documentation is hosted on readthedocs.io. Multiple maintainers can be added +for the project. GitHub integration ensures that each version is built when the +corresponding branch is pushed to. + +Each release is hosted as a separate version, and new versions will need to be +added manually. diff --git a/doc/source/contributor/environments/ci-aio.rst b/doc/source/contributor/environments/ci-aio.rst index ecd2b1214..de1d2d8fb 100644 --- a/doc/source/contributor/environments/ci-aio.rst +++ b/doc/source/contributor/environments/ci-aio.rst @@ -9,7 +9,7 @@ Prerequisites ============= * a CentOS Stream 8 or Ubuntu Focal 20.04 host -* access to the local Pulp server +* access to the Test Pulp server on SMS lab Setup ===== @@ -20,31 +20,31 @@ Install package dependencies. On CentOS: -.. code-block:: console +.. parsed-literal:: sudo dnf install -y python3-virtualenv On Ubuntu: -.. code-block:: console +.. parsed-literal:: sudo apt update sudo apt install -y python3-virtualenv Clone the Kayobe and Kayobe configuration repositories (this one): -.. code-block:: console +.. parsed-literal:: cd mkdir -p src pushd src - git clone https://github.com/stackhpc/kayobe.git -b stackhpc/yoga - git clone https://github.com/stackhpc/stackhpc-kayobe-config -b stackhpc/yoga kayobe-config + git clone https://github.com/stackhpc/kayobe.git -b |current_release_git_branch_name| + git clone https://github.com/stackhpc/stackhpc-kayobe-config -b |current_release_git_branch_name| kayobe-config popd Create a virtual environment and install Kayobe: -.. code-block:: console +.. parsed-literal:: cd mkdir -p venvs @@ -57,7 +57,7 @@ Create a virtual environment and install Kayobe: Add initial network configuration: -.. code-block:: console +.. parsed-literal:: sudo ip l add breth1 type bridge sudo ip l set breth1 up @@ -75,7 +75,7 @@ Acquire the Ansible Vault password for this repository, and store a copy at The following commands install Kayobe and its dependencies, and prepare the Ansible control host. -.. code-block:: console +.. parsed-literal:: export KAYOBE_VAULT_PASSWORD=$(cat ~/vault-pw) pushd ~/venvs/kayobe @@ -90,13 +90,13 @@ Deployment Next, configure the host OS & services. -.. code-block:: console +.. parsed-literal:: kayobe overcloud host configure Finally, deploy the overcloud services. -.. code-block:: console +.. parsed-literal:: kayobe overcloud service deploy @@ -107,7 +107,7 @@ Testing Run a smoke test: -.. code-block:: console +.. parsed-literal:: cd ~/kayobe ./dev/overcloud-test-vm.sh diff --git a/doc/source/contributor/environments/ci-builder.rst b/doc/source/contributor/environments/ci-builder.rst index 5979be716..17b73ad1c 100644 --- a/doc/source/contributor/environments/ci-builder.rst +++ b/doc/source/contributor/environments/ci-builder.rst @@ -10,7 +10,7 @@ Prerequisites ============= * a CentOS Stream 8 or Ubuntu Focal 20.04 host -* access to the local Pulp server +* access to the Test Pulp server on SMS lab Setup ===== @@ -21,31 +21,31 @@ Install package dependencies. On CentOS: -.. code-block:: console +.. parsed-literal:: sudo dnf install -y python3-virtualenv On Ubuntu: -.. code-block:: console +.. parsed-literal:: sudo apt update sudo apt install -y python3-virtualenv Clone the Kayobe and Kayobe configuration repositories (this one): -.. code-block:: console +.. parsed-literal:: cd mkdir -p src pushd src - git clone https://github.com/stackhpc/kayobe.git -b stackhpc/yoga - git clone https://github.com/stackhpc/stackhpc-kayobe-config -b stackhpc/yoga kayobe-config + git clone https://github.com/stackhpc/kayobe.git -b |current_release_git_branch_name| + git clone https://github.com/stackhpc/stackhpc-kayobe-config -b |current_release_git_branch_name| kayobe-config popd Create a virtual environment and install Kayobe: -.. code-block:: console +.. parsed-literal:: cd mkdir -p venvs @@ -58,7 +58,7 @@ Create a virtual environment and install Kayobe: Add initial network configuration: -.. code-block:: console +.. parsed-literal:: sudo ip l add breth1 type bridge sudo ip l set breth1 up @@ -76,7 +76,7 @@ Acquire the Ansible Vault password for this repository, and store a copy at The following commands install Kayobe and its dependencies, and prepare the Ansible control host. -.. code-block:: console +.. parsed-literal:: export KAYOBE_VAULT_PASSWORD=$(cat ~/vault-pw) pushd ~/venvs/kayobe @@ -91,7 +91,7 @@ Deployment Next, configure the host OS & services. -.. code-block:: console +.. parsed-literal:: kayobe seed host configure @@ -100,12 +100,12 @@ Building images At this point you are ready to build and push some container images. -.. code-block:: console +.. parsed-literal:: kayobe seed container image build --push kayobe overcloud container image build --push -The container images are tagged as ``yoga-``. This Kayobe +The container images are tagged as |current_release|-. This Kayobe configuration includes a hook that writes the tag to ``~/kolla_tag``, since it is not always simple to determine which tag was last applied to built images. diff --git a/doc/source/contributor/environments.rst b/doc/source/contributor/environments/index.rst similarity index 62% rename from doc/source/contributor/environments.rst rename to doc/source/contributor/environments/index.rst index 9d99d7a79..7192cae6a 100644 --- a/doc/source/contributor/environments.rst +++ b/doc/source/contributor/environments/index.rst @@ -6,6 +6,6 @@ The following Kayobe environments are provided with this configuration: .. toctree:: :maxdepth: 1 - environments/ci-aio - environments/ci-builder - environments/ci-multinode + ci-aio + ci-builder + ci-multinode diff --git a/doc/source/contributor/index.rst b/doc/source/contributor/index.rst index d80f0c4ce..1e1bdd2d6 100644 --- a/doc/source/contributor/index.rst +++ b/doc/source/contributor/index.rst @@ -7,4 +7,6 @@ This guide is for contributors of the StackHPC Kayobe configuration project. .. toctree:: :maxdepth: 1 - environments + documentation + release-notes + environments/index diff --git a/doc/source/contributor/release-notes.rst b/doc/source/contributor/release-notes.rst new file mode 100644 index 000000000..8deb85730 --- /dev/null +++ b/doc/source/contributor/release-notes.rst @@ -0,0 +1,38 @@ +============= +Release notes +============= + +StackHPC Kayobe configuration uses the following release notes sections: + +- ``features`` --- for new features or functionality; these should ideally + refer to the blueprint being implemented; +- ``fixes`` --- for fixes closing bugs; these must refer to the bug being + closed; +- ``upgrade`` --- for notes relevant when upgrading from previous version; + these should ideally be added only between major versions; required when + the proposed change affects behaviour in a non-backwards compatible way or + generally changes something impactful; +- ``deprecations`` --- to track deprecated features; relevant changes may + consist of only the commit message and the release note; +- ``prelude`` --- filled in by the PTL before each release or RC. + +Other release note types may be applied per common sense. +Each change should include a release note unless being a ``TrivialFix`` +change or affecting only docs or CI. Such changes should `not` include +a release note to avoid confusion. +Remember release notes are mostly for end users which, in case of Kolla, +are OpenStack administrators/operators. + +To add a release note, install the ``reno`` package in a Python virtual +environment, then run the following command: + +.. code-block:: console + + reno new + +Release notes for the current release are included in the :ref:`documentation`. +Note that a note won't be included in the generated documentation until it is +tracked by ``git``. + +All release notes can be inspected by browsing ``releasenotes/notes`` +directory. diff --git a/doc/source/index.rst b/doc/source/index.rst index a9906b35f..a7f770bc4 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -11,7 +11,16 @@ Welcome to StackHPC's Kayobe Config documentation! Overview ======== +This documentation covers the StackHPC Kayobe configuration. It is intended to +complement, rather than replace, the upstream :kayobe-doc:`Kayobe `, +:kolla-doc:`Kolla ` and :kolla-ansible-doc:`Kolla Ansible ` +documentation. +The configuration includes various things, such as: + +* Opinionated configuration +* Custom playbooks +* Continuous Integration (CI) workflows Contents ======== @@ -21,4 +30,5 @@ Contents usage configuration/index + release-notes contributor/index diff --git a/doc/source/release-notes.rst b/doc/source/release-notes.rst new file mode 100644 index 000000000..c2181c53a --- /dev/null +++ b/doc/source/release-notes.rst @@ -0,0 +1,6 @@ +========================= +Yoga Series Release Notes +========================= + +.. release-notes:: + :branch: stackhpc/yoga diff --git a/doc/source/usage.rst b/doc/source/usage.rst index 080ffa958..8152fbff2 100644 --- a/doc/source/usage.rst +++ b/doc/source/usage.rst @@ -12,9 +12,8 @@ based on the upstream kayobe-config, with some opinionated configuration changes applied. Since this repository makes changes to the base configuration, it works best -when used with Kayobe's `multiple environments -`__ -feature. +when used with Kayobe's :kayobe-doc:`multiple environments +` feature. This configuration should be consumed using the `StackHPC Kayobe fork `__, which includes @@ -24,35 +23,36 @@ New deployments --------------- If starting a new deployment, clone this repository as the starting point for -your configuration. +your configuration: -.. code-block:: console +.. parsed-literal:: - git clone https://github.com/stackhpc/stackhpc-kayobe-config -b stackhpc/yoga + git clone https://github.com/stackhpc/stackhpc-kayobe-config -b |current_release_git_branch_name| Existing deployments -------------------- If migrating an existing deployment to StackHPC Kayobe configuration, you will -need to merge the changes in this repository into your repository. +need to merge the changes in this repository into your repository: -.. code-block:: console +.. parsed-literal:: git remote add stackhpc https://github.com/stackhpc/stackhpc-kayobe-config git fetch stackhpc - git merge stackhpc/stackhpc/yoga + git merge stackhpc/|current_release_git_branch_name| Updating -------- This base configuration will be updated over time, to update repository versions, container image tags, and other configuration. Deployments may -consume these updates by merging in the changes with their local configuration. +consume these updates by merging in the changes with their local +configuration: -.. code-block:: console +.. parsed-literal:: git fetch stackhpc - git merge stackhpc/stackhpc/yoga + git merge stackhpc/|current_release_git_branch_name| The intention is to avoid merge conflicts where possible, but there may be cases where this is difficult. We are open to discussion on how best to diff --git a/etc/kayobe/ansible/rabbitmq-reset.yml b/etc/kayobe/ansible/rabbitmq-reset.yml index df6d0c4ca..d1a03eeb1 100644 --- a/etc/kayobe/ansible/rabbitmq-reset.yml +++ b/etc/kayobe/ansible/rabbitmq-reset.yml @@ -11,6 +11,21 @@ vars: - container_name: rabbitmq tasks: + - name: Checking timedatectl status + become: true + command: timedatectl status + register: timedatectl_status + changed_when: false + + - name: Fail if the clock is not synchronized + fail: + msg: >- + timedatectl sees the system clock as unsynchronized. + You may need to force synchronisation using `chronyc makestep`. + Otherwise, please wait for synchronization. + when: + - "'synchronized: yes' not in timedatectl_status.stdout" + - name: Inspect the {{ container_name }} container shell: cmd: "docker container inspect --format '{{ '{{' }} .State.Running {{ '}}' }}' {{ container_name }}" @@ -29,20 +44,20 @@ delay: 6 - name: Wait for the rabbitmq node to automatically start on container start - command: "docker exec -it {{ container_name }} /bin/bash -c 'rabbitmqctl wait /var/lib/rabbitmq/mnesia/rabbitmq.pid --timeout 60'" + command: "docker exec {{ container_name }} /bin/bash -c 'rabbitmqctl wait /var/lib/rabbitmq/mnesia/rabbitmq.pid --timeout 60'" when: inspection.stdout == 'false' - name: Stop app - command: "docker exec -it {{ container_name }} /bin/bash -c 'rabbitmqctl stop_app'" + command: "docker exec {{ container_name }} /bin/bash -c 'rabbitmqctl stop_app'" - name: Force reset app - command: "docker exec -it {{ container_name }} /bin/bash -c 'rabbitmqctl force_reset'" + command: "docker exec {{ container_name }} /bin/bash -c 'rabbitmqctl force_reset'" - name: Start app - command: "docker exec -it {{ container_name }} /bin/bash -c 'rabbitmqctl start_app'" + command: "docker exec {{ container_name }} /bin/bash -c 'rabbitmqctl start_app'" - name: Wait for all nodes to join the cluster - command: "docker exec -it {{ container_name }} /bin/bash -c 'rabbitmqctl await_online_nodes {{ groups['controllers'] | length }}'" + command: "docker exec {{ container_name }} /bin/bash -c 'rabbitmqctl await_online_nodes {{ groups['controllers'] | length }}'" - name: Restart OpenStack services hosts: controllers:compute diff --git a/releasenotes/config.yaml b/releasenotes/config.yaml index 125c1a52d..913f7c98a 100644 --- a/releasenotes/config.yaml +++ b/releasenotes/config.yaml @@ -1,4 +1,4 @@ --- # This needs to be updated to the latest release. default_branch: stackhpc/yoga -release_tag_re: stackhpc/\d+\.\d+\.\d+\.\d +release_tag_re: stackhpc/12\.\d+\.\d+\.\d diff --git a/releasenotes/notes/xena-batch-bc1da4e4d0f6257e.yaml b/releasenotes/notes/xena-batch-bc1da4e4d0f6257e.yaml new file mode 100644 index 000000000..1c9446dd5 --- /dev/null +++ b/releasenotes/notes/xena-batch-bc1da4e4d0f6257e.yaml @@ -0,0 +1,21 @@ +--- +features: + - | + Adds a custom playbook to run the `Anomoly Detection Visualiser (ADVise) + `_, ``advise-run.yml``. + - | + Adds a custom playbook to reset the RabbitMQ cluster and restart OpenStack + services that use it, ``rabbitmq-reset.yml``. + - | + Adds a custom playbook to configure swap, ``swap.yml``. + - | + Adds the `Kayobe Automation + `__ Git repository as a + submodule, and provides some basic configuration for it in an + ``.automation.conf`` directory. + - | + Adds support for deploying a Squid caching proxy as a custom container on + the seed. + - | + Enables Elasticsearch, Grafana, Kibana, Prometheus by default. Provides + standard dashboards for Grafana and alerting rules for Prometheus.