pulp overcloud image hosting #292
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,121 @@ | ||
| --- | ||
| name: Promote overcloud host image | ||
| on: | ||
| workflow_dispatch: | ||
| inputs: | ||
| os_image: | ||
| description: Image to promote | ||
| type: choice | ||
| required: true | ||
| default: 'CentOS Stream 8' | ||
| options: | ||
| - 'CentOS Stream 8' | ||
| - 'Rocky Linux 8' | ||
| - 'Ubuntu Focal 20.04' | ||
| image_tag: | ||
| description: Tag to promote | ||
| type: string | ||
| required: true | ||
| env: | ||
| ANSIBLE_FORCE_COLOR: True | ||
| jobs: | ||
| overcloud-host-image-promote: | ||
| name: Promote overcloud host image | ||
| if: github.repository == 'stackhpc/stackhpc-kayobe-config' | ||
| runs-on: [self-hosted, stackhpc-kayobe-config-kolla-builder] | ||
| steps: | ||
| - uses: actions/checkout@v3 | ||
| with: | ||
| path: src/kayobe-config | ||
|
|
||
| - name: Clone StackHPC Kayobe repository | ||
| uses: actions/checkout@v3 | ||
| with: | ||
| repository: stackhpc/kayobe | ||
| ref: refs/heads/stackhpc/xena | ||
| path: src/kayobe | ||
|
|
||
| # FIXME: Failed in kolla-ansible : Ensure the latest version of pip is installed | ||
| - name: Install dependencies | ||
| run: | | ||
| sudo dnf -y install python3-virtualenv | ||
| - name: Setup networking | ||
| run: | | ||
| if ! ip l show breth1 >/dev/null 2>&1; then | ||
| sudo ip l add breth1 type bridge | ||
| fi | ||
| sudo ip l set breth1 up | ||
| if ! ip a show breth1 | grep 192.168.33.3/24; then | ||
| sudo ip a add 192.168.33.3/24 dev breth1 | ||
| fi | ||
| if ! ip l show dummy1 >/dev/null 2>&1; then | ||
| sudo ip l add dummy1 type dummy | ||
| fi | ||
| sudo ip l set dummy1 up | ||
| sudo ip l set dummy1 master breth1 | ||
| - name: Install Kayobe | ||
| run: | | ||
| mkdir -p venvs && | ||
| pushd venvs && | ||
| python3 -m venv kayobe && | ||
| source kayobe/bin/activate && | ||
| pip install -U pip && | ||
| pip install ../src/kayobe | ||
| - name: Bootstrap the control host | ||
| run: | | ||
| source venvs/kayobe/bin/activate && | ||
| source src/kayobe-config/kayobe-env --environment ci-builder && | ||
| kayobe control host bootstrap | ||
| - name: Configure the seed host | ||
|
There was a problem hiding this comment. nit: I would guess that this is unnecessary. |
||
| run: | | ||
| source venvs/kayobe/bin/activate && | ||
| source src/kayobe-config/kayobe-env --environment ci-builder && | ||
| kayobe seed host configure | ||
| env: | ||
| KAYOBE_VAULT_PASSWORD: ${{ secrets.KAYOBE_VAULT_PASSWORD }} | ||
|
|
||
| - name: Promote CentOS Stream 8 overcloud host image artifact | ||
| run: | | ||
| source venvs/kayobe/bin/activate && | ||
| source src/kayobe-config/kayobe-env --environment ci-builder && | ||
| kayobe playbook run \ | ||
| src/kayobe-config/etc/kayobe/ansible/pulp-host-image-promote.yml \ | ||
| -e image_path='/opt/kayobe/images/overcloud-centos-stream-8' \ | ||
| -e os_distribution='centos' \ | ||
| -e os_release='8-stream' | ||
| env: | ||
| OVERCLOUD_HOST_IMAGE_TAG: ${{ inputs.image_tag }} | ||
| KAYOBE_VAULT_PASSWORD: ${{ secrets.KAYOBE_VAULT_PASSWORD }} | ||
| if: os_image == 'CentOS Stream 8' | ||
|
|
||
| - name: Promote Rocky Linux 8 overcloud host image artifact | ||
| run: | | ||
| source venvs/kayobe/bin/activate && | ||
| source src/kayobe-config/kayobe-env --environment ci-builder && | ||
| kayobe playbook run \ | ||
| src/kayobe-config/etc/kayobe/ansible/pulp-host-image-promote.yml \ | ||
| -e image_path='/opt/kayobe/images/overcloud-rocky-linux-8' \ | ||
| -e os_distribution='rocky' \ | ||
| -e os_release='8' | ||
| env: | ||
| OVERCLOUD_HOST_IMAGE_TAG: ${{ inputs.image_tag }} | ||
| KAYOBE_VAULT_PASSWORD: ${{ secrets.KAYOBE_VAULT_PASSWORD }} | ||
| if: os_image == 'Rocky Linux 8' | ||
|
|
||
| - name: Promote Ubuntu Focal 20.04 overcloud host image artifact | ||
| run: | | ||
| source venvs/kayobe/bin/activate && | ||
| source src/kayobe-config/kayobe-env --environment ci-builder && | ||
| kayobe playbook run \ | ||
| src/kayobe-config/etc/kayobe/ansible/pulp-host-image-promote.yml \ | ||
| -e image_path='/opt/kayobe/images/overcloud-ubuntu-focal' \ | ||
| -e os_distribution='ubuntu' \ | ||
| -e os_release='focal' | ||
| env: | ||
| OVERCLOUD_HOST_IMAGE_TAG: ${{ inputs.image_tag }} | ||
| KAYOBE_VAULT_PASSWORD: ${{ secrets.KAYOBE_VAULT_PASSWORD }} | ||
| if: os_image == 'Ubuntu Focal 20.04' | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| --- | ||
markgoddard marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| # Tag is the version of the image to promote, stored in an env variable named OVERCLOUD_HOST_IMAGE_TAG | ||
| - name: Promote an image to production | ||
| hosts: localhost | ||
| vars: | ||
| remote_pulp_url: "{{ stackhpc_release_pulp_url }}" | ||
| remote_pulp_username: "{{ stackhpc_image_repository_username }}" | ||
| remote_pulp_password: "{{ stackhpc_image_repository_password }}" | ||
| repository_name: "kayobe-images-{{ openstack_release }}-{{ os_distribution }}-{{ os_release }}" | ||
| base_path: "kayobe-images/{{ openstack_release }}/{{ os_distribution }}/{{ os_release }}" | ||
| promotion_tag: "{{ lookup('env', 'OVERCLOUD_HOST_IMAGE_TAG') }}" | ||
| tasks: | ||
|
|
||
| - name: Check whether the image exists | ||
| pulp.squeezer.file_distribution: | ||
| pulp_url: "{{ remote_pulp_url }}" | ||
| username: "{{ remote_pulp_username }}" | ||
| password: "{{ remote_pulp_password }}" | ||
| name: "{{ repository_name }}_{{ promotion_tag }}" | ||
| base_path: "{{ base_path }}/{{ promotion_tag }}" | ||
| register: distribution_details | ||
|
|
||
| - name: Fail if the image does not exist | ||
| fail: | ||
| msg: "Image {{ promotion_tag }} does not exist" | ||
| when: distribution_details.distribution is none | ||
|
|
||
| - name: Ensure production content guard is set | ||
| pulp.squeezer.file_distribution: | ||
| pulp_url: "{{ remote_pulp_url }}" | ||
| username: "{{ remote_pulp_username }}" | ||
| password: "{{ remote_pulp_password }}" | ||
| name: "{{ repository_name }}_{{ promotion_tag }}" | ||
| base_path: "{{ base_path }}/{{ promotion_tag }}" | ||
| content_guard: release | ||
| state: present | ||
|
|
||
| - name: Print version tag and os | ||
| debug: | ||
| msg: "Promoted tag: {{ promotion_tag }}" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,109 @@ | ||
| --- | ||
| - name: Upload and create a distribution for an image | ||
| hosts: localhost | ||
| vars: | ||
| remote_pulp_url: "{{ stackhpc_release_pulp_url }}" | ||
| remote_pulp_username: "{{ stackhpc_image_repository_username }}" | ||
| remote_pulp_password: "{{ stackhpc_image_repository_password }}" | ||
| repository_name: "kayobe-images-{{ openstack_release }}-{{ os_distribution }}-{{ os_release }}" | ||
| base_path: "kayobe-images/{{ openstack_release }}/{{ os_distribution }}/{{ os_release }}" | ||
| tasks: | ||
| - name: Get filename | ||
| find: | ||
| paths: "{{ image_path }}" | ||
| patterns: '*.qcow2' | ||
| register: found_files | ||
|
|
||
| - name: Upload an artifact | ||
| pulp.squeezer.artifact: | ||
| pulp_url: "{{ remote_pulp_url }}" | ||
| username: "{{ remote_pulp_username }}" | ||
| password: "{{ remote_pulp_password }}" | ||
| file: "{{ found_files.files[0].path }}" | ||
| state: present | ||
|
|
||
| - name: Get sha256 hash | ||
| ansible.builtin.stat: | ||
| path: "{{ found_files.files[0].path }}" | ||
| checksum_algorithm: sha256 | ||
| register: file_stats | ||
|
|
||
| - name: Create file content from artifact | ||
| pulp.squeezer.file_content: | ||
| pulp_url: "{{ remote_pulp_url }}" | ||
| username: "{{ remote_pulp_username }}" | ||
| password: "{{ remote_pulp_password }}" | ||
| sha256: "{{ file_stats.stat.checksum }}" | ||
| relative_path: "{{ found_files.files[0].path | basename }}" | ||
| state: present | ||
|
|
||
| - name: Ensure file repo exists | ||
| pulp.squeezer.file_repository: | ||
| pulp_url: "{{ remote_pulp_url }}" | ||
| username: "{{ remote_pulp_username }}" | ||
| password: "{{ remote_pulp_password }}" | ||
| name: "{{ repository_name }}" | ||
| state: present | ||
|
|
||
| - name: Add content to file repo | ||
| pulp.squeezer.file_repository_content: | ||
| pulp_url: "{{ remote_pulp_url }}" | ||
| username: "{{ remote_pulp_username }}" | ||
| password: "{{ remote_pulp_password }}" | ||
| repository: "{{ repository_name }}" | ||
| present_content: | ||
| - relative_path: "{{ found_files.files[0].path | basename }}" | ||
| sha256: "{{ file_stats.stat.checksum }}" | ||
|
|
||
| - name: Create a new publication to point to this version | ||
| pulp.squeezer.file_publication: | ||
| pulp_url: "{{ remote_pulp_url }}" | ||
| username: "{{ remote_pulp_username }}" | ||
| password: "{{ remote_pulp_password }}" | ||
| repository: "{{ repository_name }}" | ||
| state: present | ||
| register: publication_details | ||
|
|
||
| - name: Update distribution for latest version | ||
| pulp.squeezer.file_distribution: | ||
| pulp_url: "{{ remote_pulp_url }}" | ||
| username: "{{ remote_pulp_username }}" | ||
| password: "{{ remote_pulp_password }}" | ||
| name: "{{ repository_name }}_latest" | ||
| base_path: "{{ base_path }}/latest" | ||
| publication: "{{ publication_details.publication.pulp_href }}" | ||
| content_guard: development | ||
| state: present | ||
| register: latest_distribution_details | ||
|
|
||
| - name: Create distribution for given version | ||
| pulp.squeezer.file_distribution: | ||
| pulp_url: "{{ remote_pulp_url }}" | ||
| username: "{{ remote_pulp_username }}" | ||
| password: "{{ remote_pulp_password }}" | ||
| name: "{{ repository_name }}_{{ ansible_facts.date_time.iso8601_basic_short }}" | ||
| base_path: "{{ base_path }}/{{ ansible_facts.date_time.iso8601_basic_short }}" | ||
| publication: "{{ publication_details.publication.pulp_href }}" | ||
| content_guard: development | ||
| state: present | ||
| when: latest_distribution_details.changed | ||
|
|
||
| - name: Update new images file with versioned path | ||
| lineinfile: | ||
| path: /tmp/updated_images.txt | ||
| line: "{{ remote_pulp_url }}/pulp/content/{{ base_path }}/\ | ||
| {{ ansible_facts.date_time.iso8601_basic_short }}/{{ found_files.files[0].path | basename }}" | ||
| create: true | ||
| when: latest_distribution_details.changed | ||
|
|
||
| - name: Update new images file with latest path | ||
| lineinfile: | ||
| path: /tmp/updated_images.txt | ||
| line: "{{ remote_pulp_url }}/pulp/content/{{ base_path }}/\ | ||
| latest/{{ found_files.files[0].path | basename }}" | ||
| when: latest_distribution_details.changed | ||
|
|
||
| - name: Print version tag | ||
| debug: | ||
| msg: "New tag: {{ ansible_facts.date_time.iso8601_basic_short }}" | ||
| when: latest_distribution_details.changed |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.