ci: use ghcr, signatures, sboms, slsa, reusable workflows, ossf

[xopham]

Fixes #624

Description

Major changes:

• new build system
  • implement GHCR to store images as packages in pipeline
  • use buildkit for image build -> SBOMs, Provenance
  • sign packages using cosign
• modular CI (reusable workflows and actions)
• refactor CI

Improvements:

• integrated with GitHub Security
• modular CI
• faster builds
• signed builds
• nightly builds
• provenance and SBOMs added
• broader code scanning coverage

TBD:

• summary update
• rm trivy-released-image-scan into sca job
• join docs into main workflows
• update "required tests"
• matrix strategy for dockerhub-check

Open Question:

• OK to add registry to helm chart? seems more explicit and less reliant on a random default
• OK to rename "nightly-scans" to "nightly"?
• can we drop tag engineering?
• should we make everything reusable with selector for which jobs to drop?
• now only relying on Github security not pipeline fail
• IMPORANT: mitigate risk of overwriting tags!

Missing (create issues for remaining):

• release workflow + test
• somehow getRoot fails on nightly
• update public getRoot image
• rework all actions
• consider caching for build and integration tests
• speedup by reduction of integration tests
• remove update upload
• consider usage of https://github.com/marketplace/actions/docker-metadata-action
• use GH action specific parsing: https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions
• add verify tag functionality for cosign validation to connaisseur
• sign scans
• add semgrep
• add checkov
• automated local testing with remote builds
• confirm dependabot update actions by commitSHA (checkout actions are purposely outdated by two minor versions)
• finetune checkov
• resolve OSSF Scorecard findings
• publish OSSF Scorecard findings
• merge upgrade int test into bash script
• why has cosign output changed?
• harmonize usage of yq (see ci: use ghcr, signatures, sboms, slsa, reusable workflows, ossf #622 (comment))
• consider dropping Uninstall when not necessary (see ci: use ghcr, signatures, sboms, slsa, reusable workflows, ossf #622 (comment))
• switch back to strict github security evaluation
• don't run integration tests on push to develop
• external MRs are not allowed to initiate push to packages? good? bad?

Checklist

• PR is rebased to/aimed at branch develop
• PR follows Contributing Guide
• Added tests (if necessary)
• Extended README/Documentation (if necessary)
• Adjusted versions of image and Helm chart in values.yaml and Chart.yaml (if necessary)

[codecov-commenter]

Codecov Report

Base: 97.13% // Head: 97.13% // No change to project coverage 👍

Coverage data is based on head (391222d) compared to base (eea60b7).
Patch has no changes to coverable lines.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files

@@           Coverage Diff            @@
##           develop     #622   +/-   ##
========================================
  Coverage    97.13%   97.13%           
========================================
  Files           23       23           
  Lines         1290     1290           
========================================
  Hits          1253     1253           
  Misses          37       37           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.