Use ClusterImagePolicy with Keyless + e2e tests for CIP with kind #1650

vaikas · 2022-03-23T21:34:02Z

Summary

Plumb through the Keyless validation from ClusterImagePolicy

Ticket Link

Fixes

Release Note

 * Start enforcing ClusterImagePolicy. Supports both Key, Keyless Authorities.

Signed-off-by: Ville Aikas <vaikas@chainguard.dev>

we need to make sure that all policies have at least one validating authority. Signed-off-by: Ville Aikas <vaikas@chainguard.dev>

understand how to test fully with both rekor/fulcio the good case. Signed-off-by: Ville Aikas <vaikas@chainguard.dev>

codecov-commenter · 2022-03-24T22:00:16Z

Codecov Report

Merging #1650 (58ff276) into main (b102404) will increase coverage by 0.87%.
The diff coverage is 78.57%.

@@            Coverage Diff             @@
##             main    #1650      +/-   ##
==========================================
+ Coverage   27.89%   28.77%   +0.87%     
==========================================
  Files         139      139              
  Lines        7972     8139     +167     
==========================================
+ Hits         2224     2342     +118     
- Misses       5512     5544      +32     
- Partials      236      253      +17

Impacted Files	Coverage Δ
pkg/cosign/kubernetes/webhook/validator.go	`80.49% <76.19%> (-1.22%)`	⬇️
pkg/apis/config/image_policies.go	`69.38% <100.00%> (ø)`
pkg/cosign/kubernetes/webhook/validation.go	`80.21% <100.00%> (+1.64%)`	⬆️
pkg/cosign/tuf/client.go	`62.34% <0.00%> (-0.95%)`	⬇️
pkg/cosign/common.go	`0.00% <0.00%> (ø)`
cmd/cosign/cli/clean.go	`0.00% <0.00%> (ø)`
cmd/cosign/cli/options/clean.go	`0.00% <0.00%> (ø)`
pkg/cosign/verify.go	`16.32% <0.00%> (+4.65%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b102404...58ff276. Read the comment docs.

Signed-off-by: Ville Aikas <vaikas@chainguard.dev>

.github/workflows/kind-cluster-image-policy.yaml

mattmoor · 2022-03-24T22:27:11Z

.github/workflows/kind-cluster-image-policy.yaml

+    # These set up the env variables so that
+    - name: Set the endpoints on the cluster and grab secrets
+      run: |
+        REKOR_URL=`kubectl -n rekor-system get --no-headers ksvc rekor | cut -d ' ' -f 4`
+        echo "REKOR_URL=$REKOR_URL" >> $GITHUB_ENV
+        curl -s $REKOR_URL/api/v1/log/publicKey > ./rekor-public.pem
+
+        FULCIO_URL=`kubectl -n fulcio-system get --no-headers ksvc fulcio | cut -d ' ' -f 4`
+        echo "FULCIO_URL=$FULCIO_URL" >> $GITHUB_ENV
+        CTLOG_URL=`kubectl -n ctlog-system get --no-headers ksvc ctlog | cut -d ' ' -f 4`
+        echo "CTLOG_URL=$CTLOG_URL" >> $GITHUB_ENV
+
+        ISSUER_URL=`kubectl get --no-headers ksvc gettoken | cut -d ' ' -f 4`
+        echo "ISSUER_URL=$ISSUER_URL" >> $GITHUB_ENV
+        OIDC_TOKEN=`curl -s $ISSUER_URL`
+        echo "OIDC_TOKEN=$OIDC_TOKEN" >> $GITHUB_ENV
+
+        kubectl -n ctlog-system get secrets ctlog-public-key -o=jsonpath='{.data.public}' | base64 -d > ./ctlog-public.pem
+        echo "SIGSTORE_CT_LOG_PUBLIC_KEY_FILE=./ctlog-public.pem" >> $GITHUB_ENV
+
+        kubectl -n fulcio-system get secrets fulcio-secret -ojsonpath='{.data.cert}' | base64 -d > ./fulcio-root.pem
+        echo "SIGSTORE_ROOT_FILE=./fulcio-root.pem" >> $GITHUB_ENV
+
+    - name: Deploy ClusterImagePolicy
+      run: |
+        kubectl apply -f ./test/testdata/cosigned/e2e/cip.yaml
+
+    - name: build cosign
+      run: |
+        make cosign
+
+    - name: Sign demoimage with cosign
+      run: |
+        ./cosign sign --rekor-url ${{ env.REKOR_URL }} --fulcio-url ${{ env.FULCIO_URL }} --force --allow-insecure-registry ${{ env.demoimage }} --identity-token ${{ env.OIDC_TOKEN }}
+
+    - name: Verify with cosign
+      run: |
+        SIGSTORE_TRUST_REKOR_API_PUBLIC_KEY=1 COSIGN_EXPERIMENTAL=1 ./cosign verify --rekor-url ${{ env.REKOR_URL }} --allow-insecure-registry ${{ env.demoimage }}
+
+    - name: Deploy jobs and verify signed works, unsigned fails
+      run: |
+        kubectl create namespace demo
+        kubectl label namespace demo cosigned.sigstore.dev/include=true
+
+        echo '::group:: test job success'
+        # We signed this above, this should work
+        if ! kubectl create -n demo job demo --image=${{ env.demoimage }} ; then
+          echo Failed to create Job in namespace without label!
+          exit 1
+        else
+          echo Succcessfully created Job with signed image
+        fi
+        echo '::endgroup:: test job success'
+
+        echo '::group:: test job rejection'
+        # We did not sign this, should fail
+        if kubectl create -n demo job demo2 --image=${{ env.demoimage2 }} ; then
+          echo Failed to block unsigned Job creation!
+          exit 1
+        else
+          echo Successfully blocked Job creation with unsigned image
+        fi
+        echo '::endgroup::'


Move this into scripts like these?

cosign/.github/workflows/kind-e2e-cosigned.yaml

Lines 185 to 196 in bdef009

- name: Run Insecure Registry Tests

run: |

go install github.com/google/go-containerregistry/cmd/crane

./test/e2e_test_insecure_registry.sh

- name: Run Image Policy Tests

run: |

./test/e2e_test_policy_crd.sh

- name: Run Cosigned Tests

run: |

./test/e2e_test_cosigned.sh

Then you can avoid some of the GITHUB_ENV bits for passing env vars across steps, and folks can more easily run this locally.

Yeah! I was planning on doing that as a follow on. Part of the original reasoning for the test/testadata/cosigned was to dump things into there so that we can then easier pick and choose what we run and automate that. I wasn't quite sure how to structure that :)

There's some stuff that I'd like to pass from the cluster (like the oidc_token, or a way to fetch it, or fulcio URL), etc. but I reckon when I call from actions to shell, env persists?

pkg/cosign/kubernetes/webhook/validator.go

mattmoor · 2022-03-24T22:56:44Z

test/testdata/cosigned/e2e/cip.yaml

+  - glob: registry.local:5000/knative/demo*
+  authorities:
+  - keyless:
+      url: http://fulcio.fulcio-system.svc/api/v1/rootCert


Can we drop /api/v1/rootCert everywhere now?

(per my comment above)

Yes, it actually doesn't work with this anymore, I had fixed it already 👍

pkg/cosign/kubernetes/webhook/validator.go

mattmoor

All of my comments are nits, so feel free to iterate.

pkg/cosign/kubernetes/webhook/validator.go

vaikas · 2022-03-25T00:24:19Z

Thanks for the reviews / feedbacks. Since you were both in the approval mood, I'll merge this now and then I'll address / refactor in a follow up PRs. I think it's good to get started testing the pieces and now we also have e2e tests to start building more complex policy validations.

Signed-off-by: Ville Aikas <vaikas@chainguard.dev>

* First batch of followups to #1650 Signed-off-by: Ville Aikas <vaikas@chainguard.dev> * Fix few typos. Signed-off-by: Ville Aikas <vaikas@chainguard.dev> * remove some unnecessary cruft since the actions handle them. Signed-off-by: Ville Aikas <vaikas@chainguard.dev> * kubectl apply vs. create Signed-off-by: Ville Aikas <vaikas@chainguard.dev> * try harder Signed-off-by: Ville Aikas <vaikas@chainguard.dev> * registry before installations. Signed-off-by: Ville Aikas <vaikas@chainguard.dev> * Remove trying with 1.23 which might have different job retry behaviour. Signed-off-by: Ville Aikas <vaikas@chainguard.dev>

Signed-off-by: Ville Aikas <vaikas@chainguard.dev>

* Refactor based on discussions in #1650 Signed-off-by: Ville Aikas <vaikas@chainguard.dev> * Simplify return signature. Signed-off-by: Ville Aikas <vaikas@chainguard.dev> * pr feedback. Signed-off-by: Ville Aikas <vaikas@chainguard.dev> * remove now unnecessary var. Signed-off-by: Ville Aikas <vaikas@chainguard.dev>

…gstore#1650) * Just setting up the cluster tests. Signed-off-by: Ville Aikas <vaikas@chainguard.dev> * Sign/verify a simple container. Signed-off-by: Ville Aikas <vaikas@chainguard.dev> * Plumb Keyless with Fulcio / Rekor through. Signed-off-by: Ville Aikas <vaikas@chainguard.dev> * Start wiring tests together. Not complete. Signed-off-by: Ville Aikas <vaikas@chainguard.dev> * Ok, now the test should work, last run should've failed, it did. woot! Signed-off-by: Ville Aikas <vaikas@chainguard.dev> * Return policies instead of all the authorities in one fell swoop since we need to make sure that all policies have at least one validating authority. Signed-off-by: Ville Aikas <vaikas@chainguard.dev> * Cleanup, add keyless unit tests. For now just failure cases, need to understand how to test fully with both rekor/fulcio the good case. Signed-off-by: Ville Aikas <vaikas@chainguard.dev> * Removed the API path portion of the test CIP as no longer necessary. Signed-off-by: Ville Aikas <vaikas@chainguard.dev>

* First batch of followups to sigstore#1650 Signed-off-by: Ville Aikas <vaikas@chainguard.dev> * Fix few typos. Signed-off-by: Ville Aikas <vaikas@chainguard.dev> * remove some unnecessary cruft since the actions handle them. Signed-off-by: Ville Aikas <vaikas@chainguard.dev> * kubectl apply vs. create Signed-off-by: Ville Aikas <vaikas@chainguard.dev> * try harder Signed-off-by: Ville Aikas <vaikas@chainguard.dev> * registry before installations. Signed-off-by: Ville Aikas <vaikas@chainguard.dev> * Remove trying with 1.23 which might have different job retry behaviour. Signed-off-by: Ville Aikas <vaikas@chainguard.dev>

* Refactor based on discussions in sigstore#1650 Signed-off-by: Ville Aikas <vaikas@chainguard.dev> * Simplify return signature. Signed-off-by: Ville Aikas <vaikas@chainguard.dev> * pr feedback. Signed-off-by: Ville Aikas <vaikas@chainguard.dev> * remove now unnecessary var. Signed-off-by: Ville Aikas <vaikas@chainguard.dev>

vaikas added 3 commits March 22, 2022 10:09

Just setting up the cluster tests.

289f99a

Signed-off-by: Ville Aikas <vaikas@chainguard.dev>

Sign/verify a simple container.

98d6cc3

Signed-off-by: Ville Aikas <vaikas@chainguard.dev>

Plumb Keyless with Fulcio / Rekor through.

53781a0

Signed-off-by: Ville Aikas <vaikas@chainguard.dev>

vaikas force-pushed the scaffolding branch 5 times, most recently from 8cbd24c to 3c3903f Compare March 23, 2022 23:22

Start wiring tests together. Not complete.

c944189

Signed-off-by: Ville Aikas <vaikas@chainguard.dev>

vaikas force-pushed the scaffolding branch from 3c3903f to c944189 Compare March 23, 2022 23:28

vaikas added 3 commits March 23, 2022 16:43

Ok, now the test should work, last run should've failed, it did. woot!

8397a04

Signed-off-by: Ville Aikas <vaikas@chainguard.dev>

Return policies instead of all the authorities in one fell swoop since

0312782

we need to make sure that all policies have at least one validating authority. Signed-off-by: Ville Aikas <vaikas@chainguard.dev>

Cleanup, add keyless unit tests. For now just failure cases, need to

38ecf23

understand how to test fully with both rekor/fulcio the good case. Signed-off-by: Ville Aikas <vaikas@chainguard.dev>

Removed the API path portion of the test CIP as no longer necessary.

58ff276

Signed-off-by: Ville Aikas <vaikas@chainguard.dev>

mattmoor reviewed Mar 24, 2022

View reviewed changes

mattmoor approved these changes Mar 24, 2022

View reviewed changes

hectorj2f approved these changes Mar 24, 2022

View reviewed changes

hectorj2f reviewed Mar 24, 2022

View reviewed changes

pkg/cosign/kubernetes/webhook/validator.go Show resolved Hide resolved

hectorj2f reviewed Mar 24, 2022

View reviewed changes

pkg/cosign/kubernetes/webhook/validator.go Show resolved Hide resolved

hectorj2f reviewed Mar 24, 2022

View reviewed changes

pkg/cosign/kubernetes/webhook/validator.go Show resolved Hide resolved

vaikas merged commit 340b6c6 into sigstore:main Mar 25, 2022

vaikas deleted the scaffolding branch March 25, 2022 00:24

github-actions bot added this to the v1.7.0 milestone Mar 25, 2022

vaikas added a commit to vaikas/cosign that referenced this pull request Mar 25, 2022

First batch of followups to sigstore#1650

fd9ac38

Signed-off-by: Ville Aikas <vaikas@chainguard.dev>

vaikas mentioned this pull request Mar 25, 2022

First batch of followups to #1650 #1664

Merged

vaikas added a commit to vaikas/cosign that referenced this pull request Mar 28, 2022

Refactor based on discussions in sigstore#1650

045410b

Signed-off-by: Ville Aikas <vaikas@chainguard.dev>

vaikas mentioned this pull request Mar 28, 2022

Refactor based on discussions in #1650 #1674

Merged

vaikas restored the scaffolding branch September 24, 2022 22:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use ClusterImagePolicy with Keyless + e2e tests for CIP with kind #1650

Use ClusterImagePolicy with Keyless + e2e tests for CIP with kind #1650

vaikas commented Mar 23, 2022 •

edited

Loading

codecov-commenter commented Mar 24, 2022 •

edited

Loading

mattmoor Mar 24, 2022

mattmoor Mar 24, 2022

vaikas Mar 24, 2022

mattmoor Mar 24, 2022

mattmoor Mar 24, 2022

vaikas Mar 25, 2022

mattmoor left a comment

vaikas commented Mar 25, 2022

	- name: Run Insecure Registry Tests
	run: \|
	go install github.com/google/go-containerregistry/cmd/crane
	./test/e2e_test_insecure_registry.sh

	- name: Run Image Policy Tests
	run: \|
	./test/e2e_test_policy_crd.sh

	- name: Run Cosigned Tests
	run: \|
	./test/e2e_test_cosigned.sh

Use ClusterImagePolicy with Keyless + e2e tests for CIP with kind #1650

Use ClusterImagePolicy with Keyless + e2e tests for CIP with kind #1650

Conversation

vaikas commented Mar 23, 2022 • edited Loading

Summary

Ticket Link

Release Note

codecov-commenter commented Mar 24, 2022 • edited Loading

Codecov Report

mattmoor Mar 24, 2022

Choose a reason for hiding this comment

mattmoor Mar 24, 2022

Choose a reason for hiding this comment

vaikas Mar 24, 2022

Choose a reason for hiding this comment

mattmoor Mar 24, 2022

Choose a reason for hiding this comment

mattmoor Mar 24, 2022

Choose a reason for hiding this comment

vaikas Mar 25, 2022

Choose a reason for hiding this comment

mattmoor left a comment

Choose a reason for hiding this comment

vaikas commented Mar 25, 2022

vaikas commented Mar 23, 2022 •

edited

Loading

codecov-commenter commented Mar 24, 2022 •

edited

Loading