From 8a1c39bae66288f72d9740bbad30b84b34af210a Mon Sep 17 00:00:00 2001 From: Mark Goddard Date: Mon, 28 Nov 2022 10:29:26 +0000 Subject: [PATCH] Move README content into documentation --- README.rst | 443 +----------------- doc/requirements.txt | 2 +- doc/source/configuration/index.rst | 11 + doc/source/configuration/release-train.rst | 155 ++++++ doc/source/contributor/environments.rst | 11 + .../contributor/environments/ci-aio.rst | 113 +++++ .../contributor/environments/ci-builder.rst | 115 +++++ .../contributor/environments/ci-multinode.rst | 55 +++ doc/source/contributor/index.rst | 10 + doc/source/index.rst | 13 +- doc/source/usage.rst | 59 +++ .../environments/ci-multinode/README.md | 48 -- tox.ini | 5 +- 13 files changed, 542 insertions(+), 498 deletions(-) create mode 100644 doc/source/configuration/index.rst create mode 100644 doc/source/configuration/release-train.rst create mode 100644 doc/source/contributor/environments.rst create mode 100644 doc/source/contributor/environments/ci-aio.rst create mode 100644 doc/source/contributor/environments/ci-builder.rst create mode 100644 doc/source/contributor/environments/ci-multinode.rst create mode 100644 doc/source/contributor/index.rst create mode 100644 doc/source/usage.rst delete mode 100644 etc/kayobe/environments/ci-multinode/README.md diff --git a/README.rst b/README.rst index dfd27bfa61..c441e9edad 100644 --- a/README.rst +++ b/README.rst @@ -5,449 +5,14 @@ StackHPC Kayobe Configuration This repository provides a base Kayobe configuration for the Xena release of StackHPC OpenStack. -StackHPC release train -====================== - -StackHPC provides packages and container images for OpenStack via `Ark -`__. - -Deployments should use a local `Pulp `__ repository -server to synchronise content from Ark and serve it locally. Access to the -repositories on Ark is controlled via X.509 certificates issued by StackHPC. - -This configuration is a base, and should be merged with any existing Kayobe -configuration. It currently provides the following: - -* Configuration to deploy a local Pulp service as a container on the seed -* Pulp repository definitions for CentOS Stream 8 -* Playbooks to synchronise a local Pulp service with Ark -* Configuration to use the local Pulp repository mirrors on control plane hosts -* Configuration to use the local Pulp container registry on control plane hosts - -This configuration defines two `Pulp distributions -`__ for -packages, ``development`` and ``production``. This allows packages to be -updated and tested in a development or staging environment before rolling them -out to production. - -How to consume this configuration -================================= - -This configuration is not a complete Kayobe configuration, rather it should be -treated as a base, in place of the `upstream kayobe-config -`__. Indeed, this repository is -based on the upstream kayobe-config, with some opinionated configuration -changes applied. - -Since this repository makes changes to the base configuration, it works best -when used with Kayobe's `multiple environments -`__ -feature. - -This configuration should be consumed using the `StackHPC Kayobe fork -`__, which includes -backported support for Ansible collections. - -New deployments ---------------- - -If starting a new deployment, clone this repository as the starting point for -your configuration. - -.. code-block:: console - - git clone https://github.com/stackhpc/stackhpc-kayobe-config -b stackhpc/xena - -Existing deployments --------------------- - -If migrating an existing deployment to StackHPC Kayobe configuration, you will -need to merge the changes in this repository into your repository. - -.. code-block:: console - - git remote add stackhpc https://github.com/stackhpc/stackhpc-kayobe-config - git fetch stackhpc - git merge stackhpc/stackhpc/xena - -Updating --------- - -This base configuration will be updated over time, to update repository -versions, container image tags, and other configuration. Deployments may -consume these updates by merging in the changes with their local configuration. +Documentation for this project is provided in the ``doc`` directory. To build +it, run: .. code-block:: console - git fetch stackhpc - git merge stackhpc/stackhpc/xena - -The intention is to avoid merge conflicts where possible, but there may be -cases where this is difficult. We are open to discussion on how best to -approach this on both sides. - -Configuration -============= - -Local Pulp server ------------------ - -The URL and credentials of the local Pulp server are configured in -``etc/kayobe/pulp.yml`` via ``pulp_url``, ``pulp_username`` and -``pulp_password``. In most cases, the default values should be sufficient. -An admin password must be generated and set as the value of a -``secrets_pulp_password`` variable, typically in an Ansible Vault encrypted -``etc/kayobe/secrets.yml`` file. This password will be automatically set on -Pulp startup. - -StackHPC Ark ------------- - -The container image registry credentials issued by StackHPC should be -configured in ``etc/kayobe/pulp.yml``, using Ansible Vault to encrypt the -password: - -.. code-block:: yaml - - stackhpc_release_pulp_username: - stackhpc_release_pulp_password: - -The client certificate and private key issued by StackHPC should be stored in -``etc/kayobe/ansible/certs/ark.stackhpc.com/client-cert.pem`` and -``etc/kayobe/ansible/certs/ark.stackhpc.com/client-key.pem``, respectively, -with the private key encrypted via Ansible Vault. - -The distribution name for the environment should be configured as either -``development`` or ``production`` via ``stackhpc_repo_distribution`` in -``etc/kayobe/stackhpc.yml``. - -Usage -===== - -The local Pulp service will be deployed as a `Seed custom container -`__ -on next ``kayobe seed service deploy`` or ``kayobe seed service upgrade``. - -The following custom playbooks are provided in ``etc/kayobe/ansible/``: - -See the Kayobe `custom playbook documentation -`__ -for information on how to run them. - -* ``pulp-repo-sync.yml``: Pull packages from Ark to the local Pulp. This will - create a new repository version (snapshot) for each repository in the local - Pulp server when new packages are available. The new packages will not be - available to cloud nodes until they have been published. -* ``pulp-repo-publish.yml``: Publish synchronised packages to the - ``development`` distribution in the local Pulp. This will make synchronised - packages available to cloud nodes using the ``development`` distribution - (typically a development or staging environment). The new packages will not - be available to cloud nodes using the ``production`` distribution until they - have been promoted. -* ``pulp-repo-promote-production.yml``: Promote packages in the ``development`` - distribution to the ``production`` distribution in the local Pulp. This will - make all packages currently available to cloud nodes using the - ``development`` distribution also available to cloud nodes using the - ``production`` distribution. Typically this would be done only once the new - packages have been validated in a development or staging environment. -* ``pulp-container-sync.yml``: Pull container images from Ark to the local - Pulp. This will create a new repository version (snapshot) for each - repository in the local Pulp server when new image tags are available. The - new image tags will not be available to cloud nodes until they have been - published. -* ``pulp-container-publish.yml``: Publish synchronised container images in the - local Pulp. This will make synchonised container images available to cloud - nodes. - -Working with pulp -================= - -The `pulp CLI -`__ tool can be used to administer your local -pulp installation. Please follow the upstream documentation for installation -instructions. - -pulp CLI tricks ---------------- - -Saving credentials -~~~~~~~~~~~~~~~~~~ - -This is useful to avoid the need to always supply your credentials when running commands -from the command line: - -.. code-block:: console - - (venv-pulp) [stack@seed ~]$ pulp config create --username admin --base-url http://:8080 --password - - -Troubleshooting --------------- - -HTTP Error 400: Bad Request {"name":["This field must be unique."]} -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -If you have previously tried to push an image to pulp e.g for local testing, you may -see this message when you later try to run ``pulp-container-sync.yml``: - -.. code-block:: console - - TASK [stackhpc.pulp.pulp_repository : Setup container repositories] ***************************** - failed: [localhost] (item=stackhpc/centos-source-prometheus-jiralert) => changed=false - ansible_loop_var: item - item: - name: stackhpc/centos-source-prometheus-jiralert - policy: on_demand - remote_password: password - remote_username: username - state: present - url: https://ark.stackhpc.com - msg: 'HTTP Error 400: Bad Request b''{"name":["This field must be unique."]}''' - -The issue is that pushing an image automatically creates a `container-push repository -`__ -which conflicts with the creation of a regular container repository of the same -name. You can resolve this conflict by deleting the distribution associated -with the push repository using the pulp CLI: - -.. code-block:: console - - (venv-pulp) [stack@seed ~]$ pulp --base-url http://:8080--username admin --password container distribution destroy --name stackhpc/centos-source-prometheus-jiralert - Started background task /pulp/api/v3/tasks/1f0a474a-b7c0-44b4-9ef4-ed633077f4d8/ - .Done. - -Environments -============ - -The following Kayobe environments are provided with this configuration: - -* ``ci-aio``: deploys an all-in-one converged control/compute cloud for testing -* ``ci-builder``: builds container images - -ci-aio ------- - -Prerequisites -^^^^^^^^^^^^^ - -* a CentOS Stream 8 or Ubuntu Focal 20.04 host -* access to the local Pulp server - -Setup -^^^^^ - -Access the host via SSH. - -Install package dependencies. - -On CentOS: - -.. code-block:: console - - sudo dnf install -y python3-virtualenv - -On Ubuntu: - -.. code-block:: console - - sudo apt update - sudo apt install -y python3-virtualenv - -Clone the Kayobe and Kayobe configuration repositories (this one): - -.. code-block:: console - - cd - mkdir -p src - pushd src - git clone https://github.com/stackhpc/kayobe.git -b stackhpc/xena - git clone https://github.com/stackhpc/stackhpc-kayobe-config -b stackhpc/xena kayobe-config - popd - -Create a virtual environment and install Kayobe: - -.. code-block:: console - - cd - mkdir -p venvs - pushd venvs - virtualenv kayobe - source kayobe/bin/activate - pip install -U pip - pip install ../src/kayobe - popd - -Add initial network configuration: - -.. code-block:: console - - sudo ip l add breth1 type bridge - sudo ip l set breth1 up - sudo ip a add 192.168.33.3/24 dev breth1 - sudo ip l add dummy1 type dummy - sudo ip l set dummy1 up - sudo ip l set dummy1 master breth1 - -Installation -^^^^^^^^^^^^ - -Acquire the Ansible Vault password for this repository, and store a copy at -``~/vault-pw``. - -The following commands install Kayobe and its dependencies, and prepare the -Ansible control host. - -.. code-block:: console - - export KAYOBE_VAULT_PASSWORD=$(cat ~/vault-pw) - pushd ~/venvs/kayobe - source bin/activate - popd - pushd ~/src/kayobe-config - source kayobe-env --environment ci-aio - kayobe control host bootstrap - -Deployment -^^^^^^^^^^ - -Next, configure the host OS & services. - -.. code-block:: console - - kayobe overcloud host configure - -Finally, deploy the overcloud services. - -.. code-block:: console - - kayobe overcloud service deploy - -The control plane should now be running. - -Testing -^^^^^^^ - -Run a smoke test: - -.. code-block:: console - - cd ~/kayobe - ./dev/overcloud-test-vm.sh - -ci-builder ----------- - -The ``ci-builder`` Kayobe environment is used to build Kolla container images. -Images are built using package repositories in the StackHPC development Pulp -service, and pushed there once built. - -Prerequisites -^^^^^^^^^^^^^ - -* a CentOS Stream 8 or Ubuntu Focal 20.04 host -* access to the local Pulp server - -Setup -^^^^^ - -Access the host via SSH. - -Install package dependencies. - -On CentOS: - -.. code-block:: console - - sudo dnf install -y python3-virtualenv - -On Ubuntu: - -.. code-block:: console - - sudo apt update - sudo apt install -y python3-virtualenv - -Clone the Kayobe and Kayobe configuration repositories (this one): - -.. code-block:: console - - cd - mkdir -p src - pushd src - git clone https://github.com/stackhpc/kayobe.git -b stackhpc/xena - git clone https://github.com/stackhpc/stackhpc-kayobe-config -b stackhpc/xena kayobe-config - popd - -Create a virtual environment and install Kayobe: - -.. code-block:: console - - cd - mkdir -p venvs - pushd venvs - virtualenv kayobe - source kayobe/bin/activate - pip install -U pip - pip install ../src/kayobe - popd - -Add initial network configuration: - -.. code-block:: console - - sudo ip l add breth1 type bridge - sudo ip l set breth1 up - sudo ip a add 192.168.33.3/24 dev breth1 - sudo ip l add dummy1 type dummy - sudo ip l set dummy1 up - sudo ip l set dummy1 master breth1 - -Installation -^^^^^^^^^^^^ - -Acquire the Ansible Vault password for this repository, and store a copy at -``~/vault-pw``. - -The following commands install Kayobe and its dependencies, and prepare the -Ansible control host. - -.. code-block:: console - - export KAYOBE_VAULT_PASSWORD=$(cat ~/vault-pw) - pushd ~/venvs/kayobe - source bin/activate - popd - pushd ~/src/kayobe-config - source kayobe-env --environment ci-builder - kayobe control host bootstrap - -Deployment -^^^^^^^^^^ - -Next, configure the host OS & services. - -.. code-block:: console - - kayobe seed host configure - -Building images -^^^^^^^^^^^^^^^ - -At this point you are ready to build and push some container images. - -.. code-block:: console - - kayobe seed container image build --push - kayobe overcloud container image build --push - -The container images are tagged as ``xena-``. This Kayobe -configuration includes a hook that writes the tag to ``~/kolla_tag``, since -it is not always simple to determine which tag was last applied to built -images. + tox -e docs -To use the new images, edit -``~/src/kayobe-config/etc/kayobe/kolla.yml`` to set the above -tag as the value of the ``kolla_openstack_release`` variable. +The HTML will be written to ``doc/build``. Resources ========= diff --git a/doc/requirements.txt b/doc/requirements.txt index 95b160c9c1..842c709a9b 100644 --- a/doc/requirements.txt +++ b/doc/requirements.txt @@ -3,5 +3,5 @@ # process, which may cause wedges in the gate later. reno>=3.1.0 # Apache-2.0 -sphinx>=2.0.0,!=2.1.0 # BSD +sphinx>=4.2.0 # BSD sphinxcontrib-svg2pdfconverter>=0.1.0 # BSD diff --git a/doc/source/configuration/index.rst b/doc/source/configuration/index.rst new file mode 100644 index 0000000000..1330457797 --- /dev/null +++ b/doc/source/configuration/index.rst @@ -0,0 +1,11 @@ +=================== +Configuration Guide +=================== + +This configuration guide describes how to use this configuration, and covers +the various features provided. + +.. toctree:: + :maxdepth: 1 + + release-train diff --git a/doc/source/configuration/release-train.rst b/doc/source/configuration/release-train.rst new file mode 100644 index 0000000000..8a25f0653d --- /dev/null +++ b/doc/source/configuration/release-train.rst @@ -0,0 +1,155 @@ +====================== +StackHPC Release Train +====================== + +StackHPC provides packages and container images for OpenStack via `Ark +`__. + +Deployments should use a local `Pulp `__ repository +server to synchronise content from Ark and serve it locally. Access to the +repositories on Ark is controlled via X.509 certificates issued by StackHPC. + +This configuration is a base, and should be merged with any existing Kayobe +configuration. It currently provides the following: + +* Configuration to deploy a local Pulp service as a container on the seed +* Pulp repository definitions for CentOS Stream 8 +* Playbooks to synchronise a local Pulp service with Ark +* Configuration to use the local Pulp repository mirrors on control plane hosts +* Configuration to use the local Pulp container registry on control plane hosts + +This configuration defines two `Pulp distributions +`__ for +packages, ``development`` and ``production``. This allows packages to be +updated and tested in a development or staging environment before rolling them +out to production. + +Configuration +============= + +Local Pulp server +----------------- + +The URL and credentials of the local Pulp server are configured in +``etc/kayobe/pulp.yml`` via ``pulp_url``, ``pulp_username`` and +``pulp_password``. In most cases, the default values should be sufficient. +An admin password must be generated and set as the value of a +``secrets_pulp_password`` variable, typically in an Ansible Vault encrypted +``etc/kayobe/secrets.yml`` file. This password will be automatically set on +Pulp startup. + +StackHPC Ark +------------ + +The container image registry credentials issued by StackHPC should be +configured in ``etc/kayobe/pulp.yml``, using Ansible Vault to encrypt the +password: + +.. code-block:: yaml + + stackhpc_release_pulp_username: + stackhpc_release_pulp_password: + +The client certificate and private key issued by StackHPC should be stored in +``etc/kayobe/ansible/certs/ark.stackhpc.com/client-cert.pem`` and +``etc/kayobe/ansible/certs/ark.stackhpc.com/client-key.pem``, respectively, +with the private key encrypted via Ansible Vault. + +The distribution name for the environment should be configured as either +``development`` or ``production`` via ``stackhpc_repo_distribution`` in +``etc/kayobe/stackhpc.yml``. + +Usage +===== + +The local Pulp service will be deployed as a `Seed custom container +`__ +on next ``kayobe seed service deploy`` or ``kayobe seed service upgrade``. + +The following custom playbooks are provided in ``etc/kayobe/ansible/``: + +See the Kayobe `custom playbook documentation +`__ +for information on how to run them. + +* ``pulp-repo-sync.yml``: Pull packages from Ark to the local Pulp. This will + create a new repository version (snapshot) for each repository in the local + Pulp server when new packages are available. The new packages will not be + available to cloud nodes until they have been published. +* ``pulp-repo-publish.yml``: Publish synchronised packages to the + ``development`` distribution in the local Pulp. This will make synchronised + packages available to cloud nodes using the ``development`` distribution + (typically a development or staging environment). The new packages will not + be available to cloud nodes using the ``production`` distribution until they + have been promoted. +* ``pulp-repo-promote-production.yml``: Promote packages in the ``development`` + distribution to the ``production`` distribution in the local Pulp. This will + make all packages currently available to cloud nodes using the + ``development`` distribution also available to cloud nodes using the + ``production`` distribution. Typically this would be done only once the new + packages have been validated in a development or staging environment. +* ``pulp-container-sync.yml``: Pull container images from Ark to the local + Pulp. This will create a new repository version (snapshot) for each + repository in the local Pulp server when new image tags are available. The + new image tags will not be available to cloud nodes until they have been + published. +* ``pulp-container-publish.yml``: Publish synchronised container images in the + local Pulp. This will make synchonised container images available to cloud + nodes. + +Working with pulp +================= + +The `pulp CLI +`__ tool can be used to administer your local +pulp installation. Please follow the upstream documentation for installation +instructions. + +pulp CLI tricks +--------------- + +Saving credentials +~~~~~~~~~~~~~~~~~~ + +This is useful to avoid the need to always supply your credentials when running commands +from the command line: + +.. code-block:: console + + (venv-pulp) [stack@seed ~]$ pulp config create --username admin --base-url http://:8080 --password + + +Troubleshooting +--------------- + +HTTP Error 400: Bad Request {"name":["This field must be unique."]} +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you have previously tried to push an image to pulp e.g for local testing, you may +see this message when you later try to run ``pulp-container-sync.yml``: + +.. code-block:: console + + TASK [stackhpc.pulp.pulp_repository : Setup container repositories] ***************************** + failed: [localhost] (item=stackhpc/centos-source-prometheus-jiralert) => changed=false + ansible_loop_var: item + item: + name: stackhpc/centos-source-prometheus-jiralert + policy: on_demand + remote_password: password + remote_username: username + state: present + url: https://ark.stackhpc.com + msg: 'HTTP Error 400: Bad Request b''{"name":["This field must be unique."]}''' + +The issue is that pushing an image automatically creates a `container-push repository +`__ +which conflicts with the creation of a regular container repository of the same +name. You can resolve this conflict by deleting the distribution associated +with the push repository using the pulp CLI: + +.. code-block:: console + + (venv-pulp) [stack@seed ~]$ pulp --base-url http://:8080--username admin --password container distribution destroy --name stackhpc/centos-source-prometheus-jiralert + Started background task /pulp/api/v3/tasks/1f0a474a-b7c0-44b4-9ef4-ed633077f4d8/ + .Done. diff --git a/doc/source/contributor/environments.rst b/doc/source/contributor/environments.rst new file mode 100644 index 0000000000..9d99d7a79d --- /dev/null +++ b/doc/source/contributor/environments.rst @@ -0,0 +1,11 @@ +Environments +============ + +The following Kayobe environments are provided with this configuration: + +.. toctree:: + :maxdepth: 1 + + environments/ci-aio + environments/ci-builder + environments/ci-multinode diff --git a/doc/source/contributor/environments/ci-aio.rst b/doc/source/contributor/environments/ci-aio.rst new file mode 100644 index 0000000000..b34bad4d6e --- /dev/null +++ b/doc/source/contributor/environments/ci-aio.rst @@ -0,0 +1,113 @@ +====== +ci-aio +====== + +This environment deploys an all-in-one converged control/compute cloud for +testing. + +Prerequisites +============= + +* a CentOS Stream 8 or Ubuntu Focal 20.04 host +* access to the local Pulp server + +Setup +===== + +Access the host via SSH. + +Install package dependencies. + +On CentOS: + +.. code-block:: console + + sudo dnf install -y python3-virtualenv + +On Ubuntu: + +.. code-block:: console + + sudo apt update + sudo apt install -y python3-virtualenv + +Clone the Kayobe and Kayobe configuration repositories (this one): + +.. code-block:: console + + cd + mkdir -p src + pushd src + git clone https://github.com/stackhpc/kayobe.git -b stackhpc/xena + git clone https://github.com/stackhpc/stackhpc-kayobe-config -b stackhpc/xena kayobe-config + popd + +Create a virtual environment and install Kayobe: + +.. code-block:: console + + cd + mkdir -p venvs + pushd venvs + virtualenv kayobe + source kayobe/bin/activate + pip install -U pip + pip install ../src/kayobe + popd + +Add initial network configuration: + +.. code-block:: console + + sudo ip l add breth1 type bridge + sudo ip l set breth1 up + sudo ip a add 192.168.33.3/24 dev breth1 + sudo ip l add dummy1 type dummy + sudo ip l set dummy1 up + sudo ip l set dummy1 master breth1 + +Installation +============ + +Acquire the Ansible Vault password for this repository, and store a copy at +``~/vault-pw``. + +The following commands install Kayobe and its dependencies, and prepare the +Ansible control host. + +.. code-block:: console + + export KAYOBE_VAULT_PASSWORD=$(cat ~/vault-pw) + pushd ~/venvs/kayobe + source bin/activate + popd + pushd ~/src/kayobe-config + source kayobe-env --environment ci-aio + kayobe control host bootstrap + +Deployment +========== + +Next, configure the host OS & services. + +.. code-block:: console + + kayobe overcloud host configure + +Finally, deploy the overcloud services. + +.. code-block:: console + + kayobe overcloud service deploy + +The control plane should now be running. + +Testing +======= + +Run a smoke test: + +.. code-block:: console + + cd ~/kayobe + ./dev/overcloud-test-vm.sh diff --git a/doc/source/contributor/environments/ci-builder.rst b/doc/source/contributor/environments/ci-builder.rst new file mode 100644 index 0000000000..f146b21073 --- /dev/null +++ b/doc/source/contributor/environments/ci-builder.rst @@ -0,0 +1,115 @@ +========== +ci-builder +========== + +The ``ci-builder`` Kayobe environment is used to build Kolla container images. +Images are built using package repositories in the StackHPC development Pulp +service, and pushed there once built. + +Prerequisites +============= + +* a CentOS Stream 8 or Ubuntu Focal 20.04 host +* access to the local Pulp server + +Setup +===== + +Access the host via SSH. + +Install package dependencies. + +On CentOS: + +.. code-block:: console + + sudo dnf install -y python3-virtualenv + +On Ubuntu: + +.. code-block:: console + + sudo apt update + sudo apt install -y python3-virtualenv + +Clone the Kayobe and Kayobe configuration repositories (this one): + +.. code-block:: console + + cd + mkdir -p src + pushd src + git clone https://github.com/stackhpc/kayobe.git -b stackhpc/xena + git clone https://github.com/stackhpc/stackhpc-kayobe-config -b stackhpc/xena kayobe-config + popd + +Create a virtual environment and install Kayobe: + +.. code-block:: console + + cd + mkdir -p venvs + pushd venvs + virtualenv kayobe + source kayobe/bin/activate + pip install -U pip + pip install ../src/kayobe + popd + +Add initial network configuration: + +.. code-block:: console + + sudo ip l add breth1 type bridge + sudo ip l set breth1 up + sudo ip a add 192.168.33.3/24 dev breth1 + sudo ip l add dummy1 type dummy + sudo ip l set dummy1 up + sudo ip l set dummy1 master breth1 + +Installation +============ + +Acquire the Ansible Vault password for this repository, and store a copy at +``~/vault-pw``. + +The following commands install Kayobe and its dependencies, and prepare the +Ansible control host. + +.. code-block:: console + + export KAYOBE_VAULT_PASSWORD=$(cat ~/vault-pw) + pushd ~/venvs/kayobe + source bin/activate + popd + pushd ~/src/kayobe-config + source kayobe-env --environment ci-builder + kayobe control host bootstrap + +Deployment +========== + +Next, configure the host OS & services. + +.. code-block:: console + + kayobe seed host configure + +Building images +=============== + +At this point you are ready to build and push some container images. + +.. code-block:: console + + kayobe seed container image build --push + kayobe overcloud container image build --push + +The container images are tagged as ``xena-``. This Kayobe +configuration includes a hook that writes the tag to ``~/kolla_tag``, since +it is not always simple to determine which tag was last applied to built +images. + +To use the new images, edit +``~/src/kayobe-config/etc/kayobe/kolla.yml`` to set the above +tag as the value of the ``kolla_openstack_release`` variable. diff --git a/doc/source/contributor/environments/ci-multinode.rst b/doc/source/contributor/environments/ci-multinode.rst new file mode 100644 index 0000000000..72aecc3e7f --- /dev/null +++ b/doc/source/contributor/environments/ci-multinode.rst @@ -0,0 +1,55 @@ +========================== +Multinode Test Environment +========================== + +Set up hosts +============ +1. Create four baremetal instances with a centos 8 stream LVM image, and a + Centos 8 stream vm +2. SSH into each baremetal and run ``sudo chown -R centos:.`` in the home + directory, then add the lines:: + + 10.0.0.34 pelican pelican.service.compute.sms-lab.cloud + 10.205.3.187 pulp-server pulp-server.internal.sms-cloud + + to ``/etc/hosts`` (if you're waiting on them starting up, you can progress + until ``kayobe overcloud host configure`` without this step) + +Basic Kayobe Setup +================== +1. SSH into the VM +2. ``sudo dnf install -y python3-virtualenv`` +3. ``mkdir src`` and ``cd src`` +4. Clone https://github.com/stackhpc/stackhpc-kayobe-config.git, then checkout + commit f31df6256f1b1fea99c84547d44f06c4cb74b161 +5. ``cd ..`` and ``mkdir venvs`` +6. ``virtualenv venvs/kayobe`` and source ``venvs/kayobe/bin/activate`` +7. ``pip install -U pip`` +8. ``pip install ./src/kayobe`` +9. Acquire the Ansible Vault password for this repository, and store a copy at + ``~/vault-pw`` +10. ``export KAYOBE_VAULT_PASSWORD=$(cat ~/vault-pw)`` + +Config changes +============== +1. In etc/kayobe/ansible/requirements.yml remove version from vxlan +2. In etc/kayobe/ansible/configure-vxlan.yml, change the group of + vxlan_interfaces so that the last octet is different e.g. 224.0.0.15 +3. Also under vxlan_interfaces, add vni:x where x is between 500 and 1000 +4. Also under vxlan_interfaces, check vxlan_dstport is not 4789 (this causes + conflicts, change to 4790) +5. In /etc/kayobe/environments/ci-multinode/tf-networks.yml, edit admin_ips so + that the compute and controller IPs line up with the + instances that were created earlier, remove the other IPs for seed and + cephOSD +6. In /etc/kayobe/environments/ci-multinode/network-allocation.yml, remove all + the entries and just assign ``aio_ips:`` an empty set ``[]`` +7. In etc/kayobe/environments/ci-multinode/inventory/hosts, remove the seed +8. run stackhpc-kayobe-config/etc/kayobe/ansible/growroot.yml (if this fails, + manually increase the partition size on each host) + +Final steps +=========== +1. ``source kayobe-env --environment ci-aio`` +2. Run ``kayobe overcloud host configure`` +3. Run ``kayobe overcloud service deploy`` diff --git a/doc/source/contributor/index.rst b/doc/source/contributor/index.rst new file mode 100644 index 0000000000..d80f0c4ce2 --- /dev/null +++ b/doc/source/contributor/index.rst @@ -0,0 +1,10 @@ +================= +Contributor Guide +================= + +This guide is for contributors of the StackHPC Kayobe configuration project. + +.. toctree:: + :maxdepth: 1 + + environments diff --git a/doc/source/index.rst b/doc/source/index.rst index 24f01975d9..a9906b35f7 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -16,12 +16,9 @@ Overview Contents ======== +.. toctree:: + :maxdepth: 2 - -Intro -===== - - - -Contributors Guide -================== + usage + configuration/index + contributor/index diff --git a/doc/source/usage.rst b/doc/source/usage.rst new file mode 100644 index 0000000000..f65b98af1d --- /dev/null +++ b/doc/source/usage.rst @@ -0,0 +1,59 @@ +===== +Usage +===== + +How to consume this configuration +================================= + +This configuration is not a complete Kayobe configuration, rather it should be +treated as a base, in place of the `upstream kayobe-config +`__. Indeed, this repository is +based on the upstream kayobe-config, with some opinionated configuration +changes applied. + +Since this repository makes changes to the base configuration, it works best +when used with Kayobe's `multiple environments +`__ +feature. + +This configuration should be consumed using the `StackHPC Kayobe fork +`__, which includes +backported support for Ansible collections. + +New deployments +--------------- + +If starting a new deployment, clone this repository as the starting point for +your configuration. + +.. code-block:: console + + git clone https://github.com/stackhpc/stackhpc-kayobe-config -b stackhpc/xena + +Existing deployments +-------------------- + +If migrating an existing deployment to StackHPC Kayobe configuration, you will +need to merge the changes in this repository into your repository. + +.. code-block:: console + + git remote add stackhpc https://github.com/stackhpc/stackhpc-kayobe-config + git fetch stackhpc + git merge stackhpc/stackhpc/xena + +Updating +-------- + +This base configuration will be updated over time, to update repository +versions, container image tags, and other configuration. Deployments may +consume these updates by merging in the changes with their local configuration. + +.. code-block:: console + + git fetch stackhpc + git merge stackhpc/stackhpc/xena + +The intention is to avoid merge conflicts where possible, but there may be +cases where this is difficult. We are open to discussion on how best to +approach this on both sides. diff --git a/etc/kayobe/environments/ci-multinode/README.md b/etc/kayobe/environments/ci-multinode/README.md deleted file mode 100644 index c2511150ca..0000000000 --- a/etc/kayobe/environments/ci-multinode/README.md +++ /dev/null @@ -1,48 +0,0 @@ -# Multinode Test Environment - -## Set up hosts -1. Create four baremetal instances with a centos 8 stream LVM image, and a -Centos 8 stream vm -2. SSH into each baremetal and run `sudo chown -R centos:.` in the home directory, - then add the lines - >`10.0.0.34 pelican pelican.service.compute.sms-lab.cloud` - >`10.205.3.187 pulp-server pulp-server.internal.sms-cloud` - - to /etc/hosts (if you're waiting on them starting up, you can progress until - `kayobe overcloud host configure` without this step) - -## Basic Kayobe Setup -1. SSH into the VM -1. `sudo dnf install -y python3-virtualenv` -1. `mkdir src` and `cd src` -1. Clone https://github.com/stackhpc/stackhpc-kayobe-config.git, then checkout - commit f31df6256f1b1fea99c84547d44f06c4cb74b161 -1. `cd ..` and `mkdir venvs` -1. `virtualenv venvs/kayobe` and source `venvs/kayobe/bin/activate` -1. `pip install -U pip` -1. `pip install ./src/kayobe` -1. Acquire the Ansible Vault password for this repository, and store a copy at -``~/vault-pw`` -1. `export KAYOBE_VAULT_PASSWORD=$(cat ~/vault-pw)` - -## Config changes -1. In etc/kayobe/ansible/requirements.yml remove version from vxlan -4. In etc/kayobe/ansible/configure-vxlan.yml, change the group of -vxlan_interfaces so that the last octet is different e.g. 224.0.0.15 -5. Also under vxlan_interfaces, add vni:x where x is between 500 and 1000 -5. Also under vxlan_interfaces, check vxlan_dstport is not 4789 (this causes -conflicts, change to 4790) -6. In /etc/kayobe/environments/ci-multinode/tf-networks.yml, - edit admin_ips so that the compute and controller IPs line up with the - instances that were created earlier, remove the other IPs for seed and - cephOSD -7. In /etc/kayobe/environments/ci-multinode/network-allocation.yml, remove all -the entries and just assign `aio_ips:` an empty set `[]` -8. In etc/kayobe/environments/ci-multinode/inventory/hosts, remove the seed -9. run stackhpc-kayobe-config/etc/kayobe/ansible/growroot.yml (if this fails, -manually increase the partition size on each host) - -## Final steps -1. `source kayobe-env --environment ci-aio` -10. Run `kayobe overcloud host configure` -11. Run `kayobe overcloud service deploy` diff --git a/tox.ini b/tox.ini index 5e1f5a7cbb..b4dd6e620e 100644 --- a/tox.ini +++ b/tox.ini @@ -27,9 +27,10 @@ commands = [testenv:docs] allowlist_externals = rm skip_install = true +# NOTE: Need yoga UC for sphinx 4.2.0 for python 3.10 support deps = - -c{env:TOX_CONSTRAINTS_FILE:https://releases.openstack.org/constraints/upper/xena} + -c{env:TOX_CONSTRAINTS_FILE:https://releases.openstack.org/constraints/upper/yoga} -r{toxinidir}/doc/requirements.txt commands = rm -rf doc/build/html - sphinx-build -W --keep-going -b html doc/source doc/build/html \ No newline at end of file + sphinx-build -W --keep-going -b html doc/source doc/build/html