WIP(mesh)[SEC-1079/Do-Not-Merge]: Mesh SLSA build provenance and image signatures for verification #7231

saisatishkarra · 2024-04-14T21:13:51Z

Description

JIRA: https://konghq.atlassian.net/browse/SEC-1079
Target version:
- Image Signing and SBOM: >=2.7.4
- Image Provenance and Binary Provenance: >=2.8.0
Draft /WIP: Pending official Kong Mesh release tag for specified <target_version> to verify provenance (NOT generated on preview tags / push to release branches).
Docs Type: Customer-facing docs

Summary

PR is focussed to allow customers to take full advantage of the recent SLSA related changes to the Kong Mesh build that implement:

cosign-based image signing (Image Signatures)
slsa-container generator based image provenance verification (Image build provenance)
slsa-generic generator based binary artifacts verification (Binary build provenance)

Sources

JIRA:

https://konghq.atlassian.net/browse/SEC-1018
https://konghq.atlassian.net/browse/SEC-1016

PRs:

Image Provenance for Kong Mesh - https://github.com/Kong/kong-mesh/pull/5623
Binary Provenance for Kong Mesh - https://github.com/Kong/kong-mesh/pull/5582
SCA / SBOM - https://github.com/Kong/kong-mesh/pull/5525
Image Signing - https://github.com/Kong/kong-mesh/pull/5547

Testing instructions

Preview link:

Checklist

Review label added
Conditional version tags added, if applicable.

For example, if this change is for an upcoming 3.6 release, enclose your content in {% if_version gte:3.6.x %} <content> {% endif_version %} tags (or if_plugin_version tags for plugins).

Use any of the following keys:

gte:<version> - greater than or equal to a specific version
lte:<version> - less than or equal to a specific version
eq:<version> - exactly equal to a specific version

You can do the same for older versions.

github-actions · 2024-04-14T21:14:03Z

⚠️ Please add at least one of the following review labels to this PR:

review:copyedit: Request for writer review.
review:general: Review for general accuracy and presentation.
Does the doc work? Does it output correctly?
review:tech: Request for technical review for a docs platform change.
review:sme: Request for review from an SME (engineer, PM, etc.).

Note: Only Kong employees can add labels due to a GitHub limitation.
If you're an OSS contributor, thank you! The maintainers will label this PR for you.

netlify · 2024-04-14T21:14:09Z

✅ Deploy Preview for kongdocs ready!

Name	Link
🔨 Latest commit	`8881add`
🔍 Latest deploy log	https://app.netlify.com/sites/kongdocs/deploys/6675b51309af390008fc1496
😎 Deploy Preview	https://deploy-preview-7231--kongdocs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
Lighthouse	9 paths audited Performance: 93 (no change from production) Accessibility: 93 (no change from production) Best Practices: 98 (🟢 up 8 from production) SEO: 91 (🟢 up 1 from production) PWA: - View the detailed breakdown and full score reports

To edit notification comments on pull requests, go to your Netlify site configuration.

app/_data/docs_nav_mesh_2.7.x.yml

app/_src/mesh/features/index.md

app/_data/docs_nav_mesh_2.7.x.yml

saisatishkarra · 2024-04-17T23:24:37Z

app/_data/docs_nav_mesh_2.8.x.yml

+            - text: Verify Build Provenance for Signed Kong Mesh Images
+              url: /features/provenance-verification-images
+            - text: Verify Build Provenance for Signed Kong Mesh Binaries
+              url: /features/provenance-verification-binaries


Image provenance and binary provenance are available from 2.8.x

app/_data/tables/features/mesh.yml

app/_src/mesh/features/provenance-verification-binaries.md

app/_src/mesh/features/provenance-verification-images.md

app/_src/mesh/sbom.md

app/_src/mesh/features/provenance-verification-images.md

saisatishkarra · 2024-06-10T21:42:58Z

@lahabana I would the need the following from mesh team before i can proceed with comments i posted to update <placeholder> values:

Backport PR to release-2.7 branch. Refer for Context
Blocked until official Kong Mesh release tag for specified <target_version> to populate assets in cloudsmith and dockerhub kong/notary
@lahabana @vsofronievk Review this PR for any feedback: https://deploy-preview-7231--kongdocs.netlify.app/

@saisatishkarra to update placeholder values after the above steps are completed. This PR will then be ready for merge.

app/_src/mesh/sbom.md

…fication

…ssets

…et links Signed-off-by: saisatishkarra <saisatish.karra@konghq.com>

Signed-off-by: saisatishkarra <saisatish.karra@konghq.com>

app/_src/mesh/sbom.md

saisatishkarra · 2024-06-20T19:47:10Z

@lahabana This PR is ready for review and needs only one clarification on this comment

lahabana

I think 2 mains things are required:

tabs for cosign vs slsa-verifier would be much more readable.
Just provide things that can be copy pasted directly that's really what someone landing on this page would want.

app/_src/mesh/features/provenance-verification-binaries.md

lahabana · 2024-06-21T07:55:51Z

app/_src/mesh/features/provenance-verification-binaries.md

+   --print-provenance \
+   --provenance-path 'kong-mesh.intoto.jsonl' \
+   --source-uri 'github.com/Kong/kong-mesh' \
+   kong-mesh-2.7.4-*-*.tar.gz


Suggested change

kong-mesh-2.7.4-*-*.tar.gz

kong-mesh-{{ page.latest_version }}-*-*.tar.gz

I am using the {{page.kong_latest.version}} everywhere instead {{ page.latest_version }} which doesn't seem to work. Not sure why though.

lahabana · 2024-06-21T07:57:20Z

app/_src/mesh/features/provenance-verification-binaries.md

+PASSED: Verified SLSA provenance
+```
+
+### Complete example


I don't understand the actual diff for the minimal and the complete example. It just seems to have the tag in which case why just make it simple and have the complete example directly?

the minimalexample is the bare minimum required parameters to verify a provenance

the complete example uses is the additional options to establish more trust ex: tag from which it was generated etc.

I can remove the "minimal" example and have the "complete" as the only option to verify

lahabana · 2024-06-21T08:00:06Z

app/_src/mesh/features/provenance-verification-binaries.md

+
+1. Ensure `slsa-verifier` is installed.
+
+2. [Download security assets](https://cloudsmith.io/~kong/repos/kong-mesh-binaries-release/packages/?q=name%3Asecurity-assets*+version%3A{{page.kong_latest.version}}) for the required version of {{site.mesh_product_name}} binaries


Why link to cloudsmith and not directly to the thing to download: https://packages.konghq.com/public/kong-mesh-binaries-release/raw/names/security-assets/versions/2.7.4/security-assets.tar.gz ?

Seems like the in-toto is not included but that's fine.

I don't see the in-toto in the security-assets is this normal?

No, the release tags should have the intoto.jsonl within the security-assets

Let me look into this!!

@lahabana The tag 2.7.4 was using the slsa provenance generator v1.10.0 instead of v2.0.0.

An improvement / breaking change to use v4 to allow downloading the artifacts during the workflow run for later jobs instead of waiting until the end of workflow run.

The step download-artifact@v4 used in the distributions to upload to cloudsmith failed to find / filter (check sreenshot for workflow warnings on GH Summary page) kong-mesh.intoto.jsonl from workflow assets since it was uploaded using an older API of actions/upload-artifacts with the issue mentioned above

The file should be published to cloudmsmith automatically in release-2.8 as it uses the version to make artifacts available to be accessible later in the workflow. NO change is needed.

@lahabana LMK if a backport of this might is needed for 2.7.x or will later be ported automatically on the next minor version bump. Hence I am moving the doc content for for Image Provenance and Binary Provenance to be available from: '>=2.8.0'

lahabana · 2024-06-21T08:09:07Z

app/_src/mesh/features/provenance-verification-images.md

+| `<repo>` | GitHub repository | `kong-mesh` |
+| `<workflow name>` | GitHub workflow name | `build-test-distribute` |
+| `<workflow trigger>` | Github workflow trigger name | `push` |
+| `<version>` | Artifact version to download | `2.7.4` |


Use page.version

lahabana · 2024-06-21T08:09:49Z

app/_src/mesh/features/provenance-verification-images.md

+
+### Prerequisites
+
+For both examples, you need to:


Would it be simpler to use tabs. 1 for cosign, 1 for slsa-verifier I think it'd be just less confusing for users.

lahabana · 2024-06-21T08:11:41Z

app/_src/mesh/features/provenance-verification-images.md

+
+```sh
+cosign verify-attestation \
+   <image>:<tag>@sha256:<manifest_digest> \


You need to explain to folks how to get the sha from a tag I think (docker inspect kong/kuma-cp:2.7.4 -f '{{ .RepoDigests }}')

Refer #7231 (comment). LMK if this is enough

lahabana · 2024-06-21T08:18:01Z

app/_src/mesh/features/signed-images.md

+{:.important .no-icon}
+> Github owner is case-sensitive (`Kong/kong-mesh` vs `kong/kong-mesh`).
+
+### Minimal example


Same here I don't see the point of complete vs minimal. Let's just go complete always it's simpler for users.
Also I would just give something users can copy/paste so no need to have a templated example first and then the full example

app/_src/mesh/sbom.md

Co-authored-by: Charly Molter <charly.molter@konghq.com>

saisatishkarra · 2024-06-21T16:25:20Z

app/_src/mesh/features/provenance-verification-binaries.md

+   --print-provenance \
+   --provenance-path 'kong-mesh.intoto.jsonl' \
+   --source-uri 'github.com/Kong/kong-mesh' \
+   kong-mesh-2.7.4-*-*.tar.gz


I am using the {{page.kong_latest.version}} everywhere instead {{ page.latest_version }} which doesn't seem to work. Not sure why though.

saisatishkarra · 2024-06-21T16:37:50Z

app/_src/mesh/features/provenance-verification-binaries.md

+PASSED: Verified SLSA provenance
+```
+
+### Complete example


the minimalexample is the bare minimum required parameters to verify a provenance

the complete example uses is the additional options to establish more trust ex: tag from which it was generated etc.

I can remove the "minimal" example and have the "complete" as the only option to verify

saisatishkarra · 2024-06-21T16:47:11Z

app/_src/mesh/features/provenance-verification-images.md

+
+3. Collect the necessary image details.
+
+4. Parse the `<manifest_digest>` for the image using `regctl`.


@lahabana This explains how to parse the <manifest_digest> https://github.com/Kong/docs.konghq.com/pull/7231/files#diff-d7e8fed39f1d31af5236253c55c02e9d8e51b24aaa7a1b3741f1b1c0be215c02R41

app/_src/mesh/sbom.md

saisatishkarra · 2024-06-21T17:18:01Z

@lahabana Due to the below issues related to provenance in 2.7.x.

missing provenance file intoto.jsonl for binary artifacts - (Pending fix)
improper notary repository for image provenance = PRThe fixes for the above only targetrelease-2.8` branch and master since 2.7.4 is already generated.

i am updating the target versions for the controls in the docs to below:

Image Signing and SBOM: >=2.7.4
Image Provenance and Binary Provenance: '>=2.8.0'

LMK if you plan on otherwise.

saisatishkarra · 2024-06-21T17:59:18Z

app/_data/docs_nav_mesh_2.7.x.yml

@@ -158,6 +163,8 @@ inherit:
          url: /features/access-audit
        - text: MeshGlobalRateLimit (beta)
          url: /features/meshglobalratelimit
+        - text: Verify Signatures for Signed Kong Mesh Images
+          url: /features/signed-images


Removed build and image provenance from 2.7.x

saisatishkarra · 2024-06-21T18:00:57Z

app/_src/mesh/features/provenance-verification-binaries.md

@@ -0,0 +1,68 @@
+---
+title: Verify Build Provenance for Kong Mesh Binaries


Waiting to test until 2.8.0 is released and docs versions are updated

saisatishkarra · 2024-06-21T18:01:20Z

app/_src/mesh/features/provenance-verification-images.md

@@ -0,0 +1,81 @@
+---
+title: Verify Build Provenance for Signed Kong Mesh Images


Waiting to test until 2.8.0 is released and docs versions are updated

saisatishkarra · 2024-06-21T18:02:24Z

app/_src/mesh/features/provenance-verification-images.md

+
+```sh
+cosign verify-attestation \
+   'kong/kuma-cp:2.8.0@<TODO_IMAGE_DIGEST>' \


wait for release and replace with 2.8.0 digest

saisatishkarra · 2024-06-21T18:02:29Z

app/_src/mesh/features/provenance-verification-images.md

+
+```sh
+slsa-verifier verify-image \
+   'kong/kuma-cp:2.8.0@<TODO_IMAGE_DIGEST>' \


wait and replace with 2.8.0 digest

saisatishkarra added blocked:kong Issues/PRs waiting on Kong PRs to merge before it can be solved do not merge Issues/ PRs whose changes should not be merged at this time labels Apr 14, 2024

saisatishkarra changed the title ~~feat(slsa/provenance)[SEC-1079]: mesh image and provenance verification~~ wip(slsa/provenance)[SEC-1079]: mesh image and provenance verification Apr 14, 2024

saisatishkarra force-pushed the feat/slsa-mesh branch from a85c321 to 57bf12f Compare April 15, 2024 19:14

saisatishkarra requested review from lahabana and vsofronievk April 15, 2024 19:17

saisatishkarra changed the title ~~wip(slsa/provenance)[SEC-1079]: mesh image and provenance verification~~ wip(slsa/provenance)[SEC-1079]: mesh image sigs and provenance verification Apr 15, 2024

saisatishkarra changed the title ~~wip(slsa/provenance)[SEC-1079]: mesh image sigs and provenance verification~~ wip(slsa/provenance)[SEC-1079]: Mesh build provenance and image signatures for verification Apr 17, 2024

lahabana reviewed Apr 17, 2024

View reviewed changes

app/_data/docs_nav_mesh_2.7.x.yml Outdated Show resolved Hide resolved

lahabana reviewed Apr 17, 2024

View reviewed changes

app/_src/mesh/features/index.md Outdated Show resolved Hide resolved

saisatishkarra force-pushed the feat/slsa-mesh branch 3 times, most recently from 2ef5ebf to cf4e0c5 Compare April 17, 2024 23:22

saisatishkarra requested a review from lahabana April 17, 2024 23:40

saisatishkarra force-pushed the feat/slsa-mesh branch 4 times, most recently from 9bb8c9c to d96e357 Compare June 10, 2024 21:29

saisatishkarra commented Jun 10, 2024

View reviewed changes

app/_src/mesh/features/provenance-verification-images.md Outdated Show resolved Hide resolved

lahabana reviewed Jun 11, 2024

View reviewed changes

app/_src/mesh/sbom.md Outdated Show resolved Hide resolved

saisatishkarra commented Jun 11, 2024

View reviewed changes

app/_src/mesh/sbom.md Outdated Show resolved Hide resolved

saisatishkarra commented Jun 11, 2024

View reviewed changes

app/_src/mesh/sbom.md Outdated Show resolved Hide resolved

saisatishkarra force-pushed the feat/slsa-mesh branch 3 times, most recently from 20a3910 to 54592a5 Compare June 11, 2024 17:08

saisatishkarra requested a review from lahabana June 11, 2024 17:38

saisatishkarra force-pushed the feat/slsa-mesh branch from bdb6715 to 391785b Compare June 12, 2024 04:44

saisatishkarra force-pushed the feat/slsa-mesh branch 2 times, most recently from e4b3b32 to 05810ba Compare June 20, 2024 19:08

saisatishkarra added 8 commits June 20, 2024 14:09

feat(slsa/provenance)[SEC-1079]: mesh image and provenance verification

9559931

feat(slsa/provenance)[SEC-1079]: mesh binary artifact provenance veri…

371b01d

…fication

feat(slsa/provenance)[SEC-1079]: add to mesh v2.8

a02ad7e

feat(slsa/provenance)[SEC-1079]: Add cloudsmith links for mesh slsa a…

695ceb2

…ssets

feat(slsa/provenance)[SEC-1079]: use latest version for mesh sbom ass…

d1283df

…et links Signed-off-by: saisatishkarra <saisatish.karra@konghq.com>

fix variable for mesh sbom link

4e0ad78

fix slsa verifier signature repository for image provenance

c9c6f2e

Signed-off-by: saisatishkarra <saisatish.karra@konghq.com>

docs(mesh): use official tags and add version checks for mesh slsa

f9d167e

Signed-off-by: saisatishkarra <saisatish.karra@konghq.com>

saisatishkarra force-pushed the feat/slsa-mesh branch from 05810ba to f9d167e Compare June 20, 2024 19:10

saisatishkarra commented Jun 20, 2024

View reviewed changes

app/_src/mesh/sbom.md Outdated Show resolved Hide resolved

saisatishkarra marked this pull request as ready for review June 20, 2024 19:47

saisatishkarra requested a review from a team as a code owner June 20, 2024 19:47

saisatishkarra changed the title ~~wip(slsa/provenance)[SEC-1079]: Mesh build provenance and image signatures for verification~~ ci(slsa/provenance)[SEC-1079]: Mesh build provenance and image signatures for verification Jun 20, 2024

saisatishkarra changed the title ~~ci(slsa/provenance)[SEC-1079]: Mesh build provenance and image signatures for verification~~ docs(mesh)[SEC-1079]: Mesh SLSA build provenance and image signatures for verification Jun 20, 2024

lahabana reviewed Jun 21, 2024

View reviewed changes

saisatishkarra and others added 2 commits June 21, 2024 11:16

Update app/_src/mesh/features/provenance-verification-binaries.md

fbdc7f6

Co-authored-by: Charly Molter <charly.molter@konghq.com>

docs(slsa): fix slsa links and use dynamic variables for versions

4b1d12f

saisatishkarra commented Jun 21, 2024

View reviewed changes

saisatishkarra added 2 commits June 21, 2024 12:09

docs(mesh): remove minimal examples and example templates

f6c3ab8

docs(mesh): Add image and binary provenance for versions >=2.8.x

8881add

saisatishkarra requested a review from lahabana June 21, 2024 17:58

saisatishkarra commented Jun 21, 2024

View reviewed changes

saisatishkarra changed the title ~~docs(mesh)[SEC-1079]: Mesh SLSA build provenance and image signatures for verification~~ WIP(mesh)[SEC-1079/Do-Not-Merge]: Mesh SLSA build provenance and image signatures for verification Jun 21, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP(mesh)[SEC-1079/Do-Not-Merge]: Mesh SLSA build provenance and image signatures for verification #7231

WIP(mesh)[SEC-1079/Do-Not-Merge]: Mesh SLSA build provenance and image signatures for verification #7231

saisatishkarra commented Apr 14, 2024 •

edited

github-actions bot commented Apr 14, 2024

netlify bot commented Apr 14, 2024 •

edited

saisatishkarra Apr 17, 2024 •

edited

saisatishkarra commented Jun 10, 2024 •

edited

saisatishkarra commented Jun 20, 2024

lahabana left a comment

lahabana Jun 21, 2024

saisatishkarra Jun 21, 2024

lahabana Jun 21, 2024

saisatishkarra Jun 21, 2024

saisatishkarra Jun 21, 2024

lahabana Jun 21, 2024

lahabana Jun 21, 2024

lahabana Jun 21, 2024

saisatishkarra Jun 21, 2024

saisatishkarra Jun 21, 2024

saisatishkarra Jun 21, 2024 •

edited

saisatishkarra Jun 21, 2024 •

edited

lahabana Jun 21, 2024

lahabana Jun 21, 2024

lahabana Jun 21, 2024

saisatishkarra Jun 21, 2024

lahabana Jun 21, 2024

saisatishkarra Jun 21, 2024

saisatishkarra Jun 21, 2024

saisatishkarra Jun 21, 2024 •

edited

saisatishkarra commented Jun 21, 2024 •

edited

saisatishkarra Jun 21, 2024

saisatishkarra Jun 21, 2024

saisatishkarra Jun 21, 2024

saisatishkarra Jun 21, 2024

saisatishkarra Jun 21, 2024

	kong-mesh-2.7.4--.tar.gz
	kong-mesh-{{ page.latest_version }}--.tar.gz


		1. Ensure `slsa-verifier` is installed.

		2. [Download security assets](https://cloudsmith.io/~kong/repos/kong-mesh-binaries-release/packages/?q=name%3Asecurity-assets*+version%3A{{page.kong_latest.version}}) for the required version of {{site.mesh_product_name}} binaries


		3. Collect the necessary image details.

		4. Parse the `<manifest_digest>` for the image using `regctl`.

		@@ -0,0 +1,68 @@
		---
		title: Verify Build Provenance for Kong Mesh Binaries

		@@ -0,0 +1,81 @@
		---
		title: Verify Build Provenance for Signed Kong Mesh Images

WIP(mesh)[SEC-1079/Do-Not-Merge]: Mesh SLSA build provenance and image signatures for verification #7231

Are you sure you want to change the base?

WIP(mesh)[SEC-1079/Do-Not-Merge]: Mesh SLSA build provenance and image signatures for verification #7231

Conversation

saisatishkarra commented Apr 14, 2024 • edited

Description

Summary

Sources

Testing instructions

Checklist

github-actions bot commented Apr 14, 2024

netlify bot commented Apr 14, 2024 • edited

✅ Deploy Preview for kongdocs ready!

saisatishkarra Apr 17, 2024 • edited

Choose a reason for hiding this comment

saisatishkarra commented Jun 10, 2024 • edited

saisatishkarra commented Jun 20, 2024

lahabana left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

saisatishkarra Jun 21, 2024 • edited

Choose a reason for hiding this comment

saisatishkarra Jun 21, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

saisatishkarra Jun 21, 2024 • edited

Choose a reason for hiding this comment

saisatishkarra commented Jun 21, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

saisatishkarra commented Apr 14, 2024 •

edited

netlify bot commented Apr 14, 2024 •

edited

saisatishkarra Apr 17, 2024 •

edited

saisatishkarra commented Jun 10, 2024 •

edited

saisatishkarra Jun 21, 2024 •

edited

saisatishkarra Jun 21, 2024 •

edited

saisatishkarra Jun 21, 2024 •

edited

saisatishkarra commented Jun 21, 2024 •

edited