ci-automation/release.sh: Run plume to release cloud images #475

pothos · 2022-09-15T10:57:28Z

The mantle plume tool has two steps, pre-release is the mere upload and release is the publication. In the past this was used to run the tests inbetween but we don't do this anymore.
Run plume pre-release and release in a single job. Since plume can't push to GCS in our case, we upload the files to bincache. Also do the cloudformation update which was previously done in flatcar-build-scripts but could only be run after the sync to Origin. It requires the "aws" tool in the mantle container until we implement this in plume directly.

Fixes flatcar/Flatcar#838

How to use

Add the aws tool to the mantle container. Add the new credentials to jenkins.

Maybe we can cut a dummy Alpha release and test the pipeline with it? The git tag for the Alpha would have to use this branch.
→ Doing it for a trial alpha-3367.0.0

Testing done

In http://jenkins.infra.kinvolk.io:8080/job/container/job/release/37/console

Closes: flatcar/Flatcar#739

The cloudformation update is done with the aws tool as of now, and we need it in the build pipeline flatcar/scripts#475 until we move this logic into plume.

ci-automation/release.sh

pothos · 2022-09-19T12:13:39Z

Tempted to also add the Azure community gallery upload but maybe let's keep that for later.

ci-automation/release.sh

The mantle plume tool has two steps, pre-release is the mere upload and release is the publication. In the past this was used to run the tests inbetween but we don't do this anymore. Run plume pre-release and release in a single job. Since plume can't push to GCS in our case, we upload the files to bincache. Also do the cloudformation update which was previously done in flatcar-build-scripts but could only be run after the sync to Origin. It requires the "aws" tool in the mantle container until we implement this in plume directly.

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>

the JSON object is passed from the Groovy script to the release script, we just need to extract the correct AWS Marketplace product ID based on the "<channel>-<arch>". Exception for the stable-amd64 where we also need to get the stable-pro product ID. Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>

When GCS auth is expected, plume would upload the AMI list to GCS.

This failed when used from ( secret_to_file ... VAR ; cat $VAR ) because ( ) starts a new subshell PID and secret_to_file's returned /proc/PID/fd/X path was then using the wrong PID.

We need to run plume only once for each arch, move it out of the loop. Also, address some smaller things that shellcheck complains about.

ci-automation/release.sh: Run plume to release cloud images

Temporary commit to use flatcar/mantle#475 for testing which ships the btrfs backwards compatibility test. Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>

pothos added a commit to flatcar/mantle that referenced this pull request Sep 15, 2022

Dockerfile: Add aws CLI tool

9879b53

The cloudformation update is done with the aws tool as of now, and we need it in the build pipeline flatcar/scripts#475 until we move this logic into plume.

This was referenced Sep 15, 2022

Dockerfile: Add aws CLI tool flatcar/mantle#376

Merged

sdk_container: publish the SDK on a Docker registry #439

Closed

pothos force-pushed the kai/plume-release branch 4 times, most recently from 3cd2063 to d808276 Compare September 19, 2022 08:12

tormath1 reviewed Sep 19, 2022

View reviewed changes