Skip to content

Commit

Permalink
test: Update images README and add OWNERS
Browse files Browse the repository at this point in the history
Document the review criteria as part of accepting rebase.
  • Loading branch information
smarterclayton committed Dec 15, 2020
1 parent 6c22a3c commit 9236dc8
Show file tree
Hide file tree
Showing 2 changed files with 48 additions and 1 deletion.
9 changes: 9 additions & 0 deletions test/extended/util/image/OWNERS
@@ -0,0 +1,9 @@
# See the OWNERS docs at https://go.k8s.io/owners
reviewers:
- smarterclayton
- soltysh
- sttts
approvers:
- smarterclayton
- soltysh
- sttts
40 changes: 39 additions & 1 deletion test/extended/util/image/README.md
Expand Up @@ -50,4 +50,42 @@ When a new version of Kubernetes is introduced new images will likely need to be
4. Retest the PR, which should pass or identify new failures
5. If an upstream image is removed that OpenShift tests depend on, those tests should be refactored to use the appropriate equivalent.

Step 3 only has to be run once per new image version introduced in a test.
Step 3 only has to be run once per new image version introduced in a test.


## When reviewing

We control images so that we are confident that if a user ran the tests binary in a controlled and protected offline environment that we are not introducing excessive risk for the user by running the tests (which run privileged). That means:

* Using images that are reproducible - can be updated if a security vulnerability is found
* Using images that are published to a secured location - a malicious third party shouldn't be able to trivially take over the location the image is published to to inject an invalid tag
* Using images that are versioned - `latest` or rolling tags where the API of the image can be broken MUST NOT be allowed, because then a future mirror might regress old tests in old versions

Kubernetes has a working process that we consider acceptable for upstream images documented at https://github.com/kubernetes/kubernetes/blob/master/test/images/README.md - images maintained by other communities likely do not satisfy these criteria and must be reviewed with these criteria in mind.

OpenShift test images must be built via CI and published to quay in a versioned fashion (no regressions).

New images should be added when:

1. An upstream component refactors to use a different image
1. Ask whether the upstream image is a better image (i.e. is it better managed, more generic, well built, kept up to date by some process)
2. A new test is added and needs an image AND none of the existing images are sufficient AND none of the existing images can be extended to solve it
1. I.e. agnhost is a generic tool for simulating clients inside a pod, and so it is better to use that function OR extend it than adding a separate test simulation
2. The shell image is the ultimate catch all - ANY bash code that isn't wierd should use that. If the bash code needs a novel new command we should add it to the `tools` image (which shell image points to) if it matches the criteria for tools (small Linux utilities that are useful for debugging an openshift cluster / node that are likely to be useful in a wide range of areas)
3. Don't introduce new versions of an existing image unless there is no choice - i.e. if you need `redis` and are not testing a specific version of redis, just use the existing image

### Mirroring images for approved changes before the PR is merged

In order to merge the PR, the tests have to pass, which means the new image has to be mirrored prior to merge.

When mirroring from a PR (granting access), you should check out the PR in question and build locally. You should probably rebase the local PR to ensure you don't stomp changes in master (checking out a PR doesn't exactly match what is tested).

Then run

openshift-tests images --upstream --to-repository quay.io/openshift/community-e2e-images

to verify that all things check out. If everything looks good, run

openshift-tests images --upstream --to-repository quay.io/openshift/community-e2e-images | oc image mirror -f - --filter-by-os=.*

You must be logged in (to docker, using `oc registry login --registry=quay.io` or `skopeo login` or `docker login`) to a quay account that has write permission to `quay.io/openshift/community-e2e-images` which every OWNER should have.

0 comments on commit 9236dc8

Please sign in to comment.