Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement AssignImage mutator #2429

Merged
merged 34 commits into from
Jan 24, 2023

Conversation

davis-haba
Copy link
Contributor

@davis-haba davis-haba commented Dec 6, 2022

Signed-off-by: davis-haba davishaba@google.com

Implements the new AssignImage mutator, which is designed for setting docker image strings. The mutator allows setting the domain, path, and tag separately. #2381

An image string is parsed into its components using mostly the same logic used by docker, the difference being we do not impose any defaults (e.g. we do not assume any empty domain should be interpreted as "docker.io", or that an empty tag is "latest").

To ensure an AssignImage cannot define a non convergent mutator, we perform validation on the 3 component fields. Each component is validated separately using similar validation used by docker. This prevents users from putting tokens we use to split the image string into components where they do not belong. For example, without validation, setting the tag to /abc/def on the string nginx:latest would change it to nginx/abc/def on the first round, but since no tag is parsed out of this string, the next round would produce nginx/abc/def/abc/def and so on.

In addition to validating each component individually, we also require that the path is not set to a "domain-like" value if domain is not set. Since docker domains can have / and ., it would otherwise be possible to set a path like abc.io/repo/app to modify an image string like nginx:latest, which would cause abc.io to be parsed as part of the domain on the second round of mutation, resulting in non convergence.

@codecov-commenter
Copy link

codecov-commenter commented Dec 7, 2022

Codecov Report

Base: 53.92% // Head: 53.51% // Decreases project coverage by -0.42% ⚠️

Coverage data is based on head (d2caf9f) compared to base (73611cc).
Patch coverage: 45.80% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2429      +/-   ##
==========================================
- Coverage   53.92%   53.51%   -0.42%     
==========================================
  Files         116      120       +4     
  Lines       10287    10635     +348     
==========================================
+ Hits         5547     5691     +144     
- Misses       4314     4512     +198     
- Partials      426      432       +6     
Flag Coverage Δ
unittests 53.51% <45.80%> (-0.42%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...pis/mutations/unversioned/zz_generated.deepcopy.go 1.50% <0.00%> (-0.42%) ⬇️
...mutation/mutators/assignmeta/assignmeta_mutator.go 29.80% <0.00%> (ø)
pkg/mutation/mutators/conversion.go 50.00% <0.00%> (-16.67%) ⬇️
pkg/mutation/schema/node.go 91.06% <ø> (ø)
pkg/mutation/types/mutator.go 33.33% <ø> (ø)
pkg/webhook/policy.go 38.73% <0.00%> (-1.41%) ⬇️
pkg/mutation/mutators/core/mutator.go 36.66% <36.66%> (ø)
...tation/mutators/assignimage/assignimage_mutator.go 38.26% <38.26%> (ø)
.../mutation/mutators/modifyset/modify_set_mutator.go 50.33% <42.85%> (+0.33%) ⬆️
pkg/readiness/ready_tracker.go 68.84% <44.73%> (-0.99%) ⬇️
... and 7 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

pkg/mutation/mutators/assignimage/assignimage_mutator.go Outdated Show resolved Hide resolved
pkg/mutation/mutators/assignimage/imageparser.go Outdated Show resolved Hide resolved
pkg/mutation/mutators/assignimage/imageparser.go Outdated Show resolved Hide resolved
pkg/mutation/mutators/assignimage/imageparser.go Outdated Show resolved Hide resolved
test/bats/tests/mutations/assign_image.yaml Outdated Show resolved Hide resolved
test/bats/tests/mutations/nginx_pod.yaml Outdated Show resolved Hide resolved
@tahirraza
Copy link

Have not gone thru all changes yet.
Just calling out we are covering scenarios where we can

  • replace with any arbitrary image repository
  • add image repository (with default one) - in cases if only image was defined without repo path

Copy link
Contributor

@acpana acpana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this LGTM! Just some small questions. There's also a chance to use require for testing if you want to; that's just a non blocking suggestion!

Copy link
Contributor

@maxsmythe maxsmythe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting very close. Just the error testing and hardening the validation of the mutator parameters a bit more.

Signed-off-by: davis-haba <davishaba@google.com>
Signed-off-by: davis-haba <davishaba@google.com>
Signed-off-by: davis-haba <davishaba@google.com>
Signed-off-by: davis-haba <davishaba@google.com>
Signed-off-by: davis-haba <davishaba@google.com>
Signed-off-by: davis-haba <davishaba@google.com>
Signed-off-by: davis-haba <davishaba@google.com>
Signed-off-by: davis-haba <davishaba@google.com>
Signed-off-by: davis-haba <davishaba@google.com>
Signed-off-by: davis-haba <davishaba@google.com>
Signed-off-by: davis-haba <davishaba@google.com>
Signed-off-by: davis-haba <davishaba@google.com>
Signed-off-by: davis-haba <davishaba@google.com>
Signed-off-by: davis-haba <davishaba@google.com>
@tahirraza
Copy link

tahirraza commented Jan 5, 2023

@maxsmythe and @davis-haba any updates on this change?
I'm thinking I can achieve something like this based on the changes proposed?

apiVersion: mutations.gatekeeper.sh/v1alpha1
kind: AssignImage
metadata:
  name: defaultimageregistry
spec:
  applyTo:
    - groups: [ "" ]
      kinds: [ "Pod" ]
      versions: [ "v1" ]
  location: "spec.containers[name:*].image"
  parameters:
    assignDomain: "1234567.dkr.ecr.us-east-1.amazonaws.com"
  match:
    source: "All"
    scope: Cluster # (Cluster, Namespaced)
    kinds:
      - apiGroups: [ "*" ]
        kinds: [ "Pod" ]
    excludedNamespaces: ["excl-ns-1"]

And if I do this, will it work for cases where domain is defined and not defined? I'm happy to test if you can provide guidance.

@maxsmythe
Copy link
Contributor

@tahirraza the PR is still being worked on, but has been dormant for a bit due to the holidays.

The PR should work exactly as you describe! I think it should be functional as-is (most of the back-and-forth is about safety and ensuring convergence IIRC). If you have the cycles and want to run a test build of the code, let us know how it goes!

@tahirraza
Copy link

Okay. I'll take a jab at it.

Copy link
Contributor

@maxsmythe maxsmythe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of testing and wording nits, but basically LGTM (will LGTM after nit fix)

pkg/mutation/mutators/assignimage/errors.go Outdated Show resolved Hide resolved
pkg/mutation/mutators/assignimage/errors.go Outdated Show resolved Hide resolved
pkg/mutation/mutators/assignimage/errors.go Outdated Show resolved Hide resolved
…nt error.

Signed-off-by: davis-haba <davishaba@google.com>
Copy link
Contributor

@maxsmythe maxsmythe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after nit fixes!

Signed-off-by: davis-haba <davishaba@google.com>
Signed-off-by: davis-haba <davishaba@google.com>
@davis-haba
Copy link
Contributor Author

@ritazh @sozercan PTAL

Copy link
Member

@sozercan sozercan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! LGTM

davis-haba and others added 2 commits January 20, 2023 18:56
Co-authored-by: Rita Zhang <rita.z.zhang@gmail.com>
Signed-off-by: Davis Haba <52938648+davis-haba@users.noreply.github.com>
Co-authored-by: Rita Zhang <rita.z.zhang@gmail.com>
Signed-off-by: Davis Haba <52938648+davis-haba@users.noreply.github.com>
Signed-off-by: davis-haba <davishaba@google.com>
Copy link
Member

@ritazh ritazh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@davis-haba davis-haba merged commit 7824f68 into open-policy-agent:master Jan 24, 2023
@davis-haba davis-haba deleted the docker-image-mutator branch January 24, 2023 22:14
davis-haba added a commit to davis-haba/gatekeeper that referenced this pull request Mar 7, 2023
* Implement AssignImage mutator
Signed-off-by: davis-haba <davishaba@google.com>

* fix tests
Signed-off-by: davis-haba <davishaba@google.com>

* fix controller gen setup
Signed-off-by: davis-haba <davishaba@google.com>

* fix helm manifest generation
Signed-off-by: davis-haba <davishaba@google.com>

* WIP assignimage byPod status
Signed-off-by: davis-haba <davishaba@google.com>

* mutator pod status working for assignimage
Signed-off-by: davis-haba <davishaba@google.com>

* e2e test assignimage mutator deleted
Signed-off-by: davis-haba <davishaba@google.com>

* old kubectl run
Signed-off-by: davis-haba <davishaba@google.com>

* address comments. domain must have '.' unless localhost
Signed-off-by: davis-haba <davishaba@google.com>

* appease linter
Signed-off-by: davis-haba <davishaba@google.com>

* fix gator tests
Signed-off-by: davis-haba <davishaba@google.com>

* add test domain ending in colon still converges
Signed-off-by: davis-haba <davishaba@google.com>

* docs for assignimage
Signed-off-by: davis-haba <davishaba@google.com>

* remove newline
Signed-off-by: davis-haba <davishaba@google.com>

* address comments
Signed-off-by: davis-haba <davishaba@google.com>

* appease linter
Signed-off-by: davis-haba <davishaba@google.com>

* cleanup dead code branch
Signed-off-by: davis-haba <davishaba@google.com>

* validateDomain to use splitDomain
Signed-off-by: davis-haba <davishaba@google.com>

* future-proof validateImageParts. Add custom error types.
Signed-off-by: davis-haba <davishaba@google.com>

* fix readiness tracker test
Signed-off-by: davis-haba <davishaba@google.com>

* make manifests
Signed-off-by: davis-haba <davishaba@google.com>

* validate that splitting a valid tag never returns a path
Signed-off-by: davis-haba <davishaba@google.com>

* degenerate cases for unit tests. do not expose regex on image component error.
Signed-off-by: davis-haba <davishaba@google.com>

* test missing image field. update error copy.
Signed-off-by: davis-haba <davishaba@google.com>

* tag error copy
Signed-off-by: davis-haba <davishaba@google.com>

* Update pkg/expansion/system_test.go

Co-authored-by: Rita Zhang <rita.z.zhang@gmail.com>
Signed-off-by: Davis Haba <52938648+davis-haba@users.noreply.github.com>

* Update pkg/mutation/mutators/assignimage/assignimage_mutator.go

Co-authored-by: Rita Zhang <rita.z.zhang@gmail.com>
Signed-off-by: Davis Haba <52938648+davis-haba@users.noreply.github.com>

* errors.As instead of type casting in unit tests
Signed-off-by: davis-haba <davishaba@google.com>

* fix error type checking
Signed-off-by: davis-haba <davishaba@google.com>

Signed-off-by: Davis Haba <52938648+davis-haba@users.noreply.github.com>
Co-authored-by: Sertaç Özercan <852750+sozercan@users.noreply.github.com>
Co-authored-by: Rita Zhang <rita.z.zhang@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants