Update runtime-controller #3362

benjaminjb · 2022-08-23T18:02:02Z

Checklist:

Have you added an explanation of what your changes do and why you'd like them to be included?
Have you updated or added documentation for the change, as applicable?
Have you tested your changes on all related environments with successful results, as applicable?
- Have you added automated tests?

Type of Changes:

What is the current behavior (link to any open issues here)?
runtime-controller is old

What is the new behavior (if this is a feature change)?

Breaking change (fix or feature that would cause existing functionality to change)

update runtime-controller
adjust logger
adjust envtest
adjust tests

Other Information:
Issue [sc-12818]

internal/controller/postgrescluster/controller.go

tjmoore4

I think this looks good overall, but there are a few things I want to make sure I understand before approving.

config/crd/bases/postgres-operator.crunchydata.com_postgresclusters.yaml

internal/controller/postgrescluster/controller.go

tjmoore4 · 2022-08-26T16:26:17Z

internal/logging/logr.go

-}
+// SetLogSink replaces the global logr.Logger with sink. Before this is called,
+// the global logr.Logger is a no-op.
+func SetLogSink(sink logr.LogSink) { global = logr.New(sink) }


🤔 Just so I'm clear, was our previous method of replacing the global logger deprecated?

internal/kubeapi/patch.go

tjmoore4 · 2022-08-26T17:45:12Z

internal/logging/logr.go

+	fnInfo  func(int, string, ...interface{})
+}
+
+var _ logr.LogSink = (*sink)(nil)


⛏️ I don't know that I'm completely tracking how all of these methods will ultimately be used. I think a few comments might be helpful.

I'm not sure I am either, but

the LogSink interface includes all the public funcs below (Enabled, Init, et al. : https://github.com/go-logr/logr/blob/a49c9717d660988d99b6f931e67c49745caacbcd/logr.go#L416

using function pointers in our sink struct makes it easy to swap out the actual funcs (good for testing)

I'm not actually sure that WithName is being used anywhere

actual implementation of fnError and fnInfo is handled by the Logrus func

I'm not actually sure that WithName is being used anywhere

We don't call it but other things do, e.g. https://github.com/kubernetes-sigs/controller-runtime/blob/v0.12.3/pkg/manager/manager.go#L355

internal/logging/logrus.go

tjmoore4 · 2022-08-26T17:54:42Z

internal/logging/logrus.go

+	}
+}
+
+func logrusFields(entry *logrus.Entry, kv ...interface{}) *logrus.Entry {


⛏️ A couple comments might be helpful here as well.

internal/controller/postgrescluster/pgbackrest_test.go

cbandy · 2022-09-12T21:21:19Z

internal/controller/postgrescluster/apply_test.go


 	t.Run("StatefulSetPodTemplate", func(t *testing.T) {
+		if serverVersion.LessThan(version.MustParseGeneric("1.22.01")) {
+			t.Skip("Statefulset.Status added field in 1.22.01, cannot test `patch` in earlier versions")


❓ This version number is strange. I see that AvailableReplicas was added in 1.22.0, made required in 1.23.0, then made optional again in 1.24.0. It was supposed to be back-patched, but it looks like that never happened... 🤔 Are we sending a 0 and the API complains? I thought sending extra fields was generally okay. ("Sending unknown fields" is one of the things I want to document for ourselves regarding backward compatibility.)

Yes, that version number is strange, and I don't love skipping tests in two supported version (k8s 20 and 21), but: looks like in api 0.24.2, the availableReplicas field is optional, but not omitemtpy, so we are sending a 0; the error we get here is from line 174-5 below (the before is the sts intent):

assert.NilError(t, cc.Patch(ctx, before, client.Apply, client.ForceOwnership, reconciler.Owner))

And the error we get is

assertion failed: error is not nil: failed to create typed patch object: .status.availableReplicas: field not declared in schema

the error we get here is from line 174-5 below (the before is the sts intent):

assert.NilError(t, cc.Patch(ctx, before, client.Apply, client.ForceOwnership, reconciler.Owner))

I see. So the ... plain/upstream apply can't setup the test.

🤔 The TODO you've added a few lines down is fair as well. We don't need this code since we've stopped supporting 1.18. I'll try something out.

internal/controller/postgrescluster/controller.go

cbandy · 2022-09-12T21:49:31Z

internal/pgadmin/reconcile.go

 		},
 		ReadinessProbe: &corev1.Probe{
-			Handler: corev1.Handler{
+			ProbeHandler: corev1.ProbeHandler{


📝 Well that was an interesting read. Go structs were renamed because they were over-shared between different parts of the API.

internal/upgradecheck/header_test.go

cbandy · 2022-09-13T13:31:41Z

internal/logging/logr.go

+	fnInfo  func(int, string, ...interface{})
+}
+
+var _ logr.LogSink = (*sink)(nil)


👍🏻 Changes to logging LGTM.

📝 🌱 I added the Logrus implementation of logr.Logger in this package when we were first incorporating controller-runtime alongside the v4 controllers. I did it so the new logs would look similar to the old ones. Those controllers are gone now along with their direct calls into the Logrus package. At some point, we should consider whether or not we want to keep maintaining our own logger.

How did we decide to use Zap elsewhere?

(It is interesting that we have this sort of set up as if we were going to have log levels, but essentially we just have a bool for debug on/off in main.go.)

(Also, this logging is 100% from your branch, so I'm glad it looks good :) )

How did we decide to use Zap elsewhere?

I think we used it because controller-runtime provides some glue/enhancements for it, and/or it's part of the Kubebuilder project template.

internal/controller/postgrescluster/pgbackrest_test.go

We have not supported Kubernetes 1.18 for some time now. OpenShift 4.6 is based on Kubernetes 1.19.

internal/controller/postgrescluster/apply.go

benjaminjb · 2022-09-26T20:23:11Z

internal/controller/postgrescluster/apply_test.go

+		case serverVersion.LessThan(version.MustParseGeneric("1.22")):
+			assert.ErrorContains(t,
+				cc.Patch(ctx, upstream, client.Apply, client.ForceOwnership, reconciler.Owner),
+				"field not declared in schema",
+				"expected https://issue.k8s.io/109210")

 		default:
-			assert.DeepEqual(t,
-				after.Spec.Template.Spec.SecurityContext,
-				intent.Spec.Template.Spec.SecurityContext)
+			assert.NilError(t,
+				cc.Patch(ctx, upstream, client.Apply, client.ForceOwnership, reconciler.Owner))
 		}

-		// Our apply method corrects it.
-		again := intent.DeepCopy()
+		// Our apply method generates the correct apply-patch.
+		again := constructor("status-local")
 		assert.NilError(t, reconciler.apply(ctx, again))


Just to walk through this,
(a) we try to client.Patch, which fails below 1.22, and succeeds above that version
(b) we try to use our apply, which with a statefulset now passes it through to reconciler.patch which passes it to client.Patch

But we're not setting/checking the SecurityContext in either case (because in both cases it will be there)?

Just to walk through this,
(a) we try to client.Patch, which fails below 1.22, and succeeds above that version
(b) we try to use our apply, which with a statefulset now passes it through to reconciler.patch which passes it to client.Patch

Right. client.Patch marshals the StatefulSet struct which lacks an omitempty and fails on certain server versions. Our method does more than one marshal to produce an apply-patch without that field.

But we're not setting/checking the SecurityContext in either case (because in both cases it will be there)?

The diff makes it look like one test changed, but it's really a new test. This isn't about the security context anymore.

upstream, local, again... If there are comments or names that can be improved to make this clear, go for it!

* update runtime-controller * adjust logger * adjust envtest * adjust tests Issue [sc-12818]

benjaminjb commented Aug 23, 2022

View reviewed changes

internal/controller/postgrescluster/controller.go Show resolved Hide resolved

tjmoore4 self-requested a review August 25, 2022 21:46

tjmoore4 reviewed Aug 26, 2022

View reviewed changes

benjaminjb commented Aug 26, 2022

View reviewed changes

internal/controller/postgrescluster/pgbackrest_test.go Show resolved Hide resolved

cbandy reviewed Sep 13, 2022

View reviewed changes

andrewlecuyer mentioned this pull request Sep 14, 2022

Fix compatibility with Kubernetes 1.25 #3370

Merged

benjaminjb force-pushed the update-runtime-controller branch from 24ba359 to 610bf4c Compare September 22, 2022 16:01

Remove unused SSA workarounds for Kubernetes 1.18

2572676

We have not supported Kubernetes 1.18 for some time now. OpenShift 4.6 is based on Kubernetes 1.19.

cbandy force-pushed the update-runtime-controller branch from 610bf4c to 661b8cc Compare September 26, 2022 17:04

benjaminjb commented Sep 26, 2022

View reviewed changes

internal/controller/postgrescluster/apply.go Show resolved Hide resolved

benjaminjb commented Sep 26, 2022

View reviewed changes

benjaminjb added 3 commits September 26, 2022 16:15

Update runtime-controller

d7d1221

* update runtime-controller * adjust logger * adjust envtest * adjust tests Issue [sc-12818]

update crd

5e3e6ce

remove potentially unnecessary cleanup

73e19e1

benjaminjb force-pushed the update-runtime-controller branch from e65ac03 to 73e19e1 Compare September 26, 2022 21:15

cbandy approved these changes Sep 26, 2022

View reviewed changes

benjaminjb merged commit 39a8e1f into CrunchyData:master Sep 27, 2022

Update runtime-controller #3362

Update runtime-controller #3362

Uh oh!

Conversation

benjaminjb commented Aug 23, 2022

Uh oh!

Uh oh!

tjmoore4 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants