[patch] Pin CLI image versions in gitops Job templates

[tomklapiscak]

Description

https://jsw.ibm.com/browse/MASCORE-5516

Define $_cli_image_tag as a constant in each Job file and use as the tag for the quay.io/ibmmas/cli image.

This needs to be taken into account when computing job names in certain cases (where the image field is immutable for non-hook Jobs) - so it is included when computing the adler32sum that we append to job names.

Because computing job names is getting quite complex, this PR also pulls out $_job_name_prefix, $_job_config_values and $_job_version constants and uses them to compute $_job_hash and $_job_name in a standardised manner.

• $_job_hash is an adler32sum based on the combined value of $_job_config_values, $_cli_image_tag and $_job_version.
• $_job_name is $_job_name_prefix-$_job_name_hash.

Having these constants and calculations defined in a standardised way makes scripting the process of updating them (e.g. updating all $_cli_image_tag values across the templates) and validating them (e.g. in a git pre-commit hook and a Githhub Action to check for conformance to the convention) feasible.

Note: to shorten job names, the $_job_version is now bundled into the adler32sum appended to the end of the job name rather than being embedded verbatim into the job name. In some other cases, specific differentiators (e.g. the custom-sa key, db name in 700-702-postsync-db2-manage.yaml) have also been removed from the job name and bundled into the adler32sum instead. This protects against the possibility of exceeding the 63 char job name limit in specific instances where long names have been chosen for these discriminators.

PR also includes:

• set-cli-image-tag.sh script to automate update across all templates
• verify-job-definitions.sh script to validate that all cli image tags are parametrised correctly, and (where necessary) the constants for guarding against updates to immutable Job fields are defined and used correctly. It also runs checks that should guard against generating job names that are longer than 63 chars in length. This script is used for enforcement as a pre-commit hook and in the lint Github Action.

Note: Adopting a gitops release with the changes in this PR will cause all Jobs to rerun in all environments.

Testing

Migrating to these changes in an existing env with no parameterised cli image tags (noble6/mascore3763):

Updating the image tag for all Jobs in an existing env (noble6/mascore3763):

Resulting commit: 17aebf7

Deploying to a fresh environment (fvtsaas/premium)

All cluster/instance apps synced and healthy (all jobs ran successfully)

Updating the CLI image tag across all templates

All jobs reran and completed on cluster and instance apps:

Orphaned Jobs left behind as expected (auto_delete: false), some examples:

pre-commit hook

> git -c user.useConfigOnly=true commit --quiet --allow-empty-message --file -
verify-jobs..............................................................Failed
- hook id: verify-jobs
- exit code: 1

Checking 2 files
---------
instance-applications/120-ibm-db2u-database/templates/07-postsync-setup-db2_Job.yaml:
    Missing {{- $_cli_image_tag := "..." }}

instance-applications/510-550-ibm-mas-suite-app-config/templates/700-702-postsync-db2-manage.yaml:
    Invalid CLI image tag found: "latest" (should be "{{ $_cli_image_tag }}")
    Missing {{- $_job_hash := "..." }}


Complete
  2 file(s) scanned
     0 valid
     2 invalid
     0 skipped

Invalid files were found, please consult logs above for details

Github Action

[rbinns]

Reviewed with Tom