Wallaby release train mk1

README.rst

@@ -2,15 +2,14 @@
 StackHPC Kayobe Configuration
 =============================
 
-This repository provides a base Kayobe configuration for the Victoria release
+This repository provides a base Kayobe configuration for the Wallaby release
 of StackHPC OpenStack.
 
 StackHPC release train
 ======================
 
 StackHPC provides packages and container images for OpenStack via `Ark
-<https://ark.stackhpc.com>`__. For the Victoria release, only packages are
-currently provided.
+<https://ark.stackhpc.com>`__.
 
 Deployments should use a local `Pulp <https://pulpproject.org/>`__ repository
 server to synchronise content from Ark and serve it locally. Access to the
@@ -22,6 +21,7 @@ configuration. It currently provides the following:
 * Pulp repository definitions for CentOS Stream 8
 * Playbooks to synchronise a local Pulp service with Ark
 * Configuration to use the local Pulp repository mirrors on control plane hosts
+* Configuration to use the local Pulp container registry on control plane hosts
 
 This configuration defines two `Pulp distributions
 <https://docs.pulpproject.org/pulpcore/workflows/promotion.html>`__ for
@@ -41,10 +41,19 @@ The URL and credentials of the local Pulp server should be configured in
    pulp_username: admin
    pulp_password: <password>
 
-The client certificate and key issued by StackHPC should be stored in
-``certs/ark.stackhpc.com/client-cert.pem`` and
-``certs/ark.stackhpc.com/client-key.pem``, respectively, with the private key
-encrypted via Ansible Vault.
+The container image registry credentials issued by StackHPC should be
+configured in ``etc/kayobe/pulp.yml``, using Ansible Vault to encrypt the
+password:
+
+.. code-block:: yaml
+
+   stackhpc_release_pulp_username: <username>
+   stackhpc_release_pulp_password: <password>
+
+The client certificate and private key issued by StackHPC should be stored in
+``etc/kayobe/ansible/certs/ark.stackhpc.com/client-cert.pem`` and
+``etc/kayobe/ansible/certs/ark.stackhpc.com/client-key.pem``, respectively,
+with the private key encrypted via Ansible Vault.
 
 The distribution name for the environment should be configured as either
 ``development`` or ``production`` via ``stackhpc_repo_distribution`` in
@@ -53,9 +62,20 @@ The distribution name for the environment should be configured as either
 Usage
 =====
 
-Several custom playbooks are provided in ``etc/kayobe/ansible/``.  See the
-Kayobe `custom playbook documentation
-<https://docs.openstack.org/kayobe/victoria/custom-ansible-playbooks.html>`__
+The following custom playbooks are provided in ``etc/kayobe/ansible/``:
+
+* ``pulp-repo-sync.yml``: Synchronise package repositories in local Pulp with
+  Ark.
+* ``pulp-repo-publish.yml``: Publish synced package repositories under the
+  ``development`` distribution.
+* ``pulp-repo-promote.yml``: Promote the ``development`` distribution content
+  to the ``production`` distribution.
+* ``pulp-container-sync.yml``: Synchronise container repositories in local Pulp
+  with Ark.
+* ``pulp-container-publish.yml``: Publish synced container repositories.
+
+See the Kayobe `custom playbook documentation
+<https://docs.openstack.org/kayobe/wallaby/custom-ansible-playbooks.html>`__
 for information on how to run them.
 
 * ``pulp-repo-sync.yml``: Pull packages from Ark to the local Pulp. This will
@@ -78,7 +98,7 @@ for information on how to run them.
 Resources
 =========
 
-* Kayobe documentation: https://docs.openstack.org/kayobe/victoria/
+* Kayobe documentation: https://docs.openstack.org/kayobe/wallaby/
 * Kayobe source: https://opendev.org/openstack/kayobe
 * Kayobe bugs: https://storyboard.openstack.org/#!/project/openstack/kayobe-config
 * IRC: #openstack-kolla

etc/kayobe/ansible/pulp-container-publish.yml

@@ -0,0 +1,10 @@
+---
+- name: Publish Pulp container images
+  hosts: localhost
+  gather_facts: True
+  tasks:
+    - import_role:
+        name: stackhpc.pulp.pulp_distribution
+      # NOTE: use intermediate variable to avoid distributing RPMs.
+      vars:
+        pulp_distribution_container: "{{ stackhpc_pulp_distribution_container }}"

etc/kayobe/ansible/pulp-container-sync.yml

@@ -0,0 +1,10 @@
+---
+- name: Sync client Pulp container repositories
+  hosts: localhost
+  gather_facts: False
+  tasks:
+    - import_role:
+        name: stackhpc.pulp.pulp_repository
+      # NOTE: use intermediate variable to avoid syncing RPMs.
+      vars:
+        pulp_repository_container_repos: "{{ stackhpc_pulp_repository_container_repos }}"

etc/kayobe/docker.yml

@@ -22,7 +22,7 @@
 #docker_storage_volume_thinpool_meta_size:
 
 # URL of docker registry
-#docker_registry:
+docker_registry: "{{ stackhpc_docker_registry }}"
 
 # CA of docker registry
 #docker_registry_ca:

etc/kayobe/kolla.yml

@@ -66,26 +66,26 @@
 #kolla_base_distro:
 
 # Kolla container image type: binary or source. Default is 'binary'.
-#kolla_install_type:
+kolla_install_type: source
 
 # URL of docker registry to use for Kolla images. Default is not set, in which
 # case Dockerhub will be used.
 #kolla_docker_registry:
 
 # Docker namespace to use for Kolla images. Default is 'kolla'.
-#kolla_docker_namespace:
+kolla_docker_namespace: stackhpc
 
 # Username to use to access a docker registry. Default is not set, in which
 # case the registry will be used without authentication.
-#kolla_docker_registry_username:
+kolla_docker_registry_username: "{{ stackhpc_docker_registry_username }}"
 
 # Password to use to access a docker registry. Default is not set, in which
 # case the registry will be used without authentication.
-#kolla_docker_registry_password:
+kolla_docker_registry_password: "{{ stackhpc_docker_registry_password }}"
 
 # Kolla OpenStack release version. This should be a Docker image tag.
 # Default is {{ openstack_release }}.
-#kolla_openstack_release:
+kolla_openstack_release: wallaby-20211122T135752
 
 # Docker tag applied to built container images. Default is
 # {{ kolla_openstack_release }}.
@@ -104,8 +104,100 @@
 ###############################################################################
 # Kolla image build configuration.
 
+# List of repositories for CentOS Stream.
+stackhpc_centos_stream_repos:
+  - url: "{{ stackhpc_repo_centos_stream_baseos_url }}"
+    file: "CentOS-Stream-BaseOS.repo"
+  - url: "{{ stackhpc_repo_centos_stream_appstream_url }}"
+    file: "CentOS-Stream-AppStream.repo"
+  - url: "{{ stackhpc_repo_centos_stream_extras_url }}"
+    file: "CentOS-Stream-Extras.repo"
+
+# List of repositories for EPEL.
+stackhpc_epel_repos:
+  - url: "{{ stackhpc_repo_epel_url }}"
+    file: "epel.repo"
+  - url: "{{ stackhpc_repo_epel_modular_url }}"
+    file: "epel-modular.repo"
+
+# List of additional CentOS Stream repositories.
+stackhpc_centos_additional_repos:
+  - url: "{{ stackhpc_repo_centos_stream_advanced_virtualization_url }}"
+    file: "CentOS-Advanced-Virtualization.repo"
+  - url: "{{ stackhpc_repo_centos_stream_nfv_openvswitch_url }}"
+    file: "CentOS-NFV-OpenvSwitch.repo"
+  - url: "{{ stackhpc_repo_centos_stream_openstack_wallaby_url }}"
+    file: "CentOS-OpenStack-wallaby.repo"
+  - url: "{{ stackhpc_repo_centos_stream_powertools_url }}"
+    file: "CentOS-Stream-PowerTools.repo"
+  - url: "{{ stackhpc_repo_centos_opstools_url }}"
+    file: "CentOS-OpsTools.repo"
+  - url: "{{ stackhpc_repo_centos_storage_ceph_nautilus_url }}"
+    file: "CentOS-Ceph-Nautilus.repo"
+
+# List of third-party repositories.
+stackhpc_third_party_repos:
+  - url: "{{ stackhpc_repo_elasticsearch_logstash_kibana_7_x_url }}"
+    file: "elasticsearch.repo"
+  - url: "{{ stackhpc_repo_grafana_url }}"
+    file: "grafana.repo"
+  - url: "{{ stackhpc_repo_rabbitmq_erlang_url }}"
+    file: "rabbitmq_rabbitmq-erlang.repo"
+  - url: "{{ stackhpc_repo_rabbitmq_server_url }}"
+    file: "rabbitmq_rabbitmq-server.repo"
+  - url: "{{ stackhpc_repo_treasuredata_4_url }}"
+    file: "td.repo"
+
 # Dict mapping Jinja2 block names in kolla's Docker images to their contents.
-#kolla_build_blocks:
+kolla_build_blocks:
+  base_header: |
+    RUN \
+    {% for repo in stackhpc_centos_stream_repos %}
+        sed -i -e 's/^\(mirrorlist *=.*\)/#\1/g' \
+               -e 's/^[# ]*\(baseurl *=.*\)/#\1/g' \
+               -e '/#baseurl.*/a baseurl={{ repo.url }}' /etc/yum.repos.d/{{ repo.file }}{% if not loop.last %} && \
+    {% endif %}
+    {% endfor %}
+
+  base_centos_repo_overrides_post_yum: |
+    {# fixme #}
+        && \
+    {% for repo in stackhpc_centos_stream_repos + stackhpc_epel_repos + stackhpc_centos_additional_repos + stackhpc_third_party_repos %}
+        sed -i -e 's/^\(mirrorlist *=.*\)/#\1/g' \
+               -e 's/^[# ]*\(baseurl *=.*\)/#\1/g' \
+               -e '/#baseurl.*/a baseurl={{ repo.url }}' /etc/yum.repos.d/{{ repo.file }}{% if not loop.last %} &&{% endif %} \
+    {% endfor %}
+
+  ironic_inspector_footer: |
+    # Install our custom inspector plugins.
+    RUN pip3 install git+https://github.com/stackhpc/stackhpc-inspector-plugins.git@1.1.2 \
+        -c https://releases.openstack.org/constraints/upper/wallaby
+  magnum_api_footer: |
+    # Install StackHPC release of Magnum.
+    RUN pip3 install git+https://github.com/stackhpc/magnum.git@stackhpc/wallaby \
+        -c https://releases.openstack.org/constraints/upper/wallaby
+  magnum_conductor_footer: |
+    # Install StackHPC release of Magnum.
+    RUN pip3 install git+https://github.com/stackhpc/magnum.git@stackhpc/wallaby \
+        -c https://releases.openstack.org/constraints/upper/wallaby
+  grafana_plugins_install: |
+    RUN grafana-cli plugins install vonage-status-panel
+  prometheus_v2_server_repository_version: |
+    ARG prometheus_version='2.30.0'
+  prometheus_alertmanager_repository_version: |
+    ARG prometheus_alertmanager_version='0.23.0'
+  prometheus_cadvisor_repository_version: |
+    ARG prometheus_cadvisor_version='0.40.0'
+  node_exporter_repository_version: |
+    ARG node_exporter_version='1.2.2'
+  prometheus_memcached_exporter_repository_version: |
+    ARG memcached_exporter_version='0.9.0'
+  haproxy_exporter_repository_version: |
+    ARG haproxy_exporter_version='0.12.0'
+  mysqld_exporter_repository_version: |
+    ARG mysqld_exporter_version='0.13.0'
+  prometheus_blackbox_exporter_repository_version: |
+    ARG blackbox_exporter_version='0.19.0'
 
 # Dict mapping image customization variable names to their values.
 # Each variable takes the form: