Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sketch out an alternate way of injecting clients/informers #2210

Merged
merged 3 commits into from
Aug 18, 2021

Conversation

mattmoor
Copy link
Member

One of the long-standing motivations for the injection-based controller architecture has been to enable us to synthesize different backing implementations, which can be made available to controller constructors by substituting what foo.Get(ctx) accesses.

In this change I have implemented codegen that sets up a new injection.Dynamic with an alternate implementation of clients and listers all backed by k8s.io/client-go/dynamic.Interface. This is currently not usable via knative.dev/pkg/injection/sharedmain, but can be used directly by replacing injection.Default.SetupInformers with:

	// Tell it which dynamic client to use (not automatically set up!)
	injCtx := context.WithValue(ctx, dynamicclient.Key{}, /* the dynamic client to use */)
	// Set up the injected context based on the above dynamic client.
	injCtx = injection.Dynamic.SetupDynamic(injCtx)

The first commit has the changes to injection/ and codegen/ to support this (including the manually crafted namespacedkube implementation for secrets). The second commit has the results of code-generation, and a handful of *_expansion.go files needed to fully satisfy kubernetes.Interface.

As far as testing this... with what's here, we can generate a build-able kubernetes.Interface, I have also been playing with testing this in a downstream tekton repository, and the main gap running e2e testing is some of the event-handling logic, which lives in client_expansion.go and is currently NYI (I left all *_expansion.go skeletal to try to keep the scope slightly narrower).

/kind feature
/assign @n3wscott

@google-cla google-cla bot added the cla: yes Indicates the PR's author has signed the CLA. label Aug 16, 2021
@knative-prow-robot knative-prow-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. approved Indicates a PR has been approved by an approver from all required OWNERS files. area/test-and-release labels Aug 16, 2021
@codecov
Copy link

codecov bot commented Aug 16, 2021

Codecov Report

Merging #2210 (7240a38) into main (bf176d5) will decrease coverage by 2.08%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2210      +/-   ##
==========================================
- Coverage   65.99%   63.91%   -2.09%     
==========================================
  Files         220      220              
  Lines        9232     9529     +297     
==========================================
- Hits         6093     6090       -3     
- Misses       2869     3168     +299     
- Partials      270      271       +1     
Impacted Files Coverage Δ
injection/clients.go 70.00% <0.00%> (-30.00%) ⬇️
injection/informers.go 68.75% <0.00%> (-26.91%) ⬇️
test/gcs/mock/mock.go 92.47% <0.00%> (-1.08%) ⬇️
webhook/configmaps/configmaps.go 86.31% <0.00%> (-0.15%) ⬇️
webhook/psbinding/psbinding.go 84.50% <0.00%> (-0.11%) ⬇️
...k/resourcesemantics/validation/validation_admit.go 93.25% <0.00%> (-0.08%) ⬇️
test/ha/ha.go 17.39% <0.00%> (ø)
tracing/zipkin.go 68.42% <0.00%> (ø)
metrics/exporter.go 84.09% <0.00%> (ø)
tracker/interface.go 100.00% <0.00%> (ø)
... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 50410e0...7240a38. Read the comment docs.

@mattmoor
Copy link
Member Author

The eventing failure is real. For packages like duckv1 without any types market with // +genclient we should skip the DuckV1() method. I'll push an update shortly with a fix.

@mattmoor
Copy link
Member Author

I believe the other two issues should be resolved by:

Where these repos were using older K8s clients.

@mattmoor
Copy link
Member Author

They were not, but it turns out their funniness is a result of the dance we're doing around kubebuilder layouts downstream. I believe these should now be fixed by:

🤞

@mattmoor
Copy link
Member Author

Woot, finally clean (failure is the coverage report, which always seems to fail)!

Copy link
Contributor

@n3wscott n3wscott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the direction, a couple nits.

/approve

codegen/cmd/injection-gen/generators/filtered_informer.go Outdated Show resolved Hide resolved
codegen/cmd/injection-gen/generators/filtered_informer.go Outdated Show resolved Hide resolved
codegen/cmd/injection-gen/generators/informer.go Outdated Show resolved Hide resolved
@@ -124,4 +175,49 @@ func Get(ctx {{.contextContext|raw}}) {{.informersTypedInformer|raw}} {
}
return untyped.({{.informersTypedInformer|raw}})
}

type wrapper struct {
client {{.clientSetInterface|raw}}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
client {{.clientSetInterface|raw}}
client {{ .clientSetInterface|raw }}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit, half the file does that way and half does not add spaces, so idk, I don't care too much.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, the squished way was making my eyes bleed. Perhaps we should follow up with a formatting pass?

@knative-prow-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mattmoor, n3wscott

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

…ion.

This also includes a number of manually created files of the form `*_expansion.go`, which are needed to satisfy some of the Kubernetes manual type "expansions".  At present, these are all simply `panic("NYI")`, but may become more than that in the future.
Copy link
Member Author

@mattmoor mattmoor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I incorporated the feedback as a separate commit (in the middle) and there were no changes when codegen ran. 👍

codegen/cmd/injection-gen/generators/filtered_informer.go Outdated Show resolved Hide resolved
codegen/cmd/injection-gen/generators/filtered_informer.go Outdated Show resolved Hide resolved
codegen/cmd/injection-gen/generators/informer.go Outdated Show resolved Hide resolved
@@ -124,4 +175,49 @@ func Get(ctx {{.contextContext|raw}}) {{.informersTypedInformer|raw}} {
}
return untyped.({{.informersTypedInformer|raw}})
}

type wrapper struct {
client {{.clientSetInterface|raw}}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, the squished way was making my eyes bleed. Perhaps we should follow up with a formatting pass?

@n3wscott
Copy link
Contributor

I think this is real interesting and could be used to solve the heavy mix of duck type and strongly typed clients in reconcilers, and our fun and hard to find cache mixing bugs.

/lgtm

@knative-prow-robot knative-prow-robot added the lgtm Indicates that a PR is ready to be merged. label Aug 18, 2021
@knative-prow-robot knative-prow-robot merged commit 7b5ecbc into knative:main Aug 18, 2021
@mattmoor mattmoor deleted the dynamic-injection branch August 18, 2021 15:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/test-and-release cla: yes Indicates the PR's author has signed the CLA. kind/feature lgtm Indicates that a PR is ready to be merged. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants