update verify pipeline and omnibus build/test to use containers

[evanahlberg]

Signed-off-by: Evan Ahlberg evanahlberg@gmail.com

Description

Update verify and omnibus release pipelines to use containers

The following will happen when we merge this PR:

• verify pipeline will be replaced with the new pipeline that uses containers
• verify/adhoc pipeline will be created
• verify/adhoc-canary pipeline will be created
• verify/release pipeline will be created

Future PRs will run the new verify pipeline

While we test the new pipelines and incrementally move over to them, the omnibus/release pipeline will still run on merges to main for now.

We needed the new pipelines to be created so we can investigate esoteric builds, uploading to artifactory, and test the verify/adhoc pipelines.

Related Issue

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Chore (non-breaking change that does not add functionality or fix an issue)

Checklist:

• I have read the CONTRIBUTING document.
• I have run the pre-merge tests locally and they pass.
• I have updated the documentation accordingly.
• I have added tests to cover my changes.
• All new and existing tests passed.
• All commits have been signed-off for the Developer Certificate of Origin.

[johnmccrae]

After reviewing the video and walking the code I can really see how this can be better.

[tpowell-progress]

Feels like switching to bash HEREDOCs will make this more maintainable and readable.

Still reviewing the rest of the PR

[gcs-devel]

@tpowell-progress We did implement the pipeline using heredoc on a branch but we found a few downsides in using it.

1. You need to indent using tabs (not spaces) which is problematic when people (like me) use spaces instead of tabs in their text editor.
2. If you pipe the heredoc to sed to handle the indentation, then the heredoc isn't "what you see is what you get" and you have to figure out how many spaces to indent and pass that to sed
3. It starts to get weird when if statements and case statements are mixed in.

I agree it is more readable but these are the reasons why we stuck with echo for now. In the future it would be nice to refactor this into a ruby script using ruby heredoc or erb. We actually tried that too but ruby isn't available on the agent outside the container at the moment.

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

[neha-p6]

Do we have any plan for Mac and AIX systems?