Skip to content

Operator fails with nil pointer dereference when CR validation is not passed #1696

@dmvolod

Description

@dmvolod

Please, answer some short questions which should help us to understand your problem / question better?

  • Which image of the operator are you using? e.g. registry.opensource.zalan.do/acid/postgres-operator:v1.7.1
  • Where do you run it - cloud or metal? Kubernetes or OpenShift? [Bare Metal Kind]
  • Are you running Postgres Operator in production? [no]
  • Type of issue? [Bug report]

When validation of the Postgresql CR is not passed, i.e. very long cluster name or cluster name must match {TEAM}-{NAME} format. or another validation issue, the operator fails with the following logs.

time="2021-11-19T08:44:50Z" level=info msg="DELETE event has been queued" cluster-name=default/acid-minimal-cluster-sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss pkg=controller worker=0
time="2021-11-19T08:44:50Z" level=error msg="unknown cluster: \"default/acid-minimal-cluster-sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss\"" cluster-name=default/acid-minimal-cluster-sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss pkg=controller worker=0
time="2021-11-19T09:07:39Z" level=info msg="no clusters running" pkg=controller
time="2021-11-19T09:22:36Z" level=debug msg="skipping \"ADD\" event for the invalid cluster: name cannot be longer than 58 characters" cluster-name=default/acid-minimal-cluster-sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss pkg=controller
time="2021-11-19T09:22:36Z" level=error msg="could not get reference for Postgresql CR default/acid-minimal-cluster-sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss: no kind is registered for the type v1.Postgresql in scheme \"k8s.io/client-go/kubernetes/scheme/register.go:72\"" pkg=controller
E1119 09:22:36.202290       1 runtime.go:78] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)
goroutine 38 [running]:
k8s.io/apimachinery/pkg/util/runtime.logPanic(0x1d1cba0, 0x30c2200)
	/workspace/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:74 +0x95
k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
	/workspace/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:48 +0x86
panic(0x1d1cba0, 0x30c2200)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
k8s.io/client-go/tools/record.(*recorderImpl).makeEvent(0xc000253ec0, 0x0, 0x0, 0x1fdf76d, 0x7, 0x1fde774, 0x6, 0xc00035e810, 0x28, 0xc0006fc410)
	/workspace/vendor/k8s.io/client-go/tools/record/event.go:365 +0x51
k8s.io/client-go/tools/record.(*recorderImpl).generateEvent(0xc000253ec0, 0x2340e18, 0x0, 0x0, 0x1fdf76d, 0x7, 0x1fde774, 0x6, 0xc00035e810, 0x28)
	/workspace/vendor/k8s.io/client-go/tools/record/event.go:338 +0x359
k8s.io/client-go/tools/record.(*recorderImpl).Event(...)
	/workspace/vendor/k8s.io/client-go/tools/record/event.go:352
k8s.io/client-go/tools/record.(*recorderImpl).Eventf(0xc000253ec0, 0x2340e18, 0x0, 0x1fdf76d, 0x7, 0x1fde774, 0x6, 0x1fdca02, 0x2, 0xc000946290, ...)
	/workspace/vendor/k8s.io/client-go/tools/record/event.go:356 +0xd6
github.com/zalando/postgres-operator/pkg/controller.(*Controller).queueClusterEvent(0xc00056fc00, 0x0, 0xc000067000, 0x1fdcdf2, 0x3)
	/workspace/pkg/controller/postgresql.go:446 +0x707
github.com/zalando/postgres-operator/pkg/controller.(*Controller).postgresqlAdd(0xc00056fc00, 0x1f9dc60, 0xc000067000)
	/workspace/pkg/controller/postgresql.go:506 +0x85
k8s.io/client-go/tools/cache.ResourceEventHandlerFuncs.OnAdd(...)
	/workspace/vendor/k8s.io/client-go/tools/cache/controller.go:231
k8s.io/client-go/tools/cache.(*processorListener).run.func1()
	/workspace/vendor/k8s.io/client-go/tools/cache/shared_informer.go:777 +0xc2
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc00020c760)
	/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:155 +0x5f
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc00063df60, 0x2331d40, 0xc000618000, 0x1ccb601, 0xc00077e000)
	/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:156 +0x9b
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc00020c760, 0x3b9aca00, 0x0, 0x1, 0xc00077e000)
	/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133 +0x98
k8s.io/apimachinery/pkg/util/wait.Until(...)
	/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:90
k8s.io/client-go/tools/cache.(*processorListener).run(0xc000214100)
	/workspace/vendor/k8s.io/client-go/tools/cache/shared_informer.go:771 +0x95
k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1(0xc00024e060, 0xc00077c000)
	/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:73 +0x51
created by k8s.io/apimachinery/pkg/util/wait.(*Group).Start
	/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:71 +0x65
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x19c3511]

goroutine 38 [running]:
k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
	/workspace/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:55 +0x109
panic(0x1d1cba0, 0x30c2200)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
k8s.io/client-go/tools/record.(*recorderImpl).makeEvent(0xc000253ec0, 0x0, 0x0, 0x1fdf76d, 0x7, 0x1fde774, 0x6, 0xc00035e810, 0x28, 0xc0006fc410)
	/workspace/vendor/k8s.io/client-go/tools/record/event.go:365 +0x51
k8s.io/client-go/tools/record.(*recorderImpl).generateEvent(0xc000253ec0, 0x2340e18, 0x0, 0x0, 0x1fdf76d, 0x7, 0x1fde774, 0x6, 0xc00035e810, 0x28)
	/workspace/vendor/k8s.io/client-go/tools/record/event.go:338 +0x359
k8s.io/client-go/tools/record.(*recorderImpl).Event(...)
	/workspace/vendor/k8s.io/client-go/tools/record/event.go:352
k8s.io/client-go/tools/record.(*recorderImpl).Eventf(0xc000253ec0, 0x2340e18, 0x0, 0x1fdf76d, 0x7, 0x1fde774, 0x6, 0x1fdca02, 0x2, 0xc000946290, ...)
	/workspace/vendor/k8s.io/client-go/tools/record/event.go:356 +0xd6
github.com/zalando/postgres-operator/pkg/controller.(*Controller).queueClusterEvent(0xc00056fc00, 0x0, 0xc000067000, 0x1fdcdf2, 0x3)
	/workspace/pkg/controller/postgresql.go:446 +0x707
github.com/zalando/postgres-operator/pkg/controller.(*Controller).postgresqlAdd(0xc00056fc00, 0x1f9dc60, 0xc000067000)
	/workspace/pkg/controller/postgresql.go:506 +0x85
k8s.io/client-go/tools/cache.ResourceEventHandlerFuncs.OnAdd(...)
	/workspace/vendor/k8s.io/client-go/tools/cache/controller.go:231
k8s.io/client-go/tools/cache.(*processorListener).run.func1()
	/workspace/vendor/k8s.io/client-go/tools/cache/shared_informer.go:777 +0xc2
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc00020c760)
	/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:155 +0x5f
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc00063df60, 0x2331d40, 0xc000618000, 0x1ccb601, 0xc00077e000)
	/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:156 +0x9b
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc00020c760, 0x3b9aca00, 0x0, 0x1, 0xc00077e000)
	/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133 +0x98
k8s.io/apimachinery/pkg/util/wait.Until(...)
	/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:90
k8s.io/client-go/tools/cache.(*processorListener).run(0xc000214100)
	/workspace/vendor/k8s.io/client-go/tools/cache/shared_informer.go:771 +0x95
k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1(0xc00024e060, 0xc00077c000)
	/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:73 +0x51
created by k8s.io/apimachinery/pkg/util/wait.(*Group).Start
	/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:71 +0x65

The reason of this problem is that new event recorder

recorder := eventBroadcaster.NewRecorder(scheme.Scheme, v1.EventSource{Component: myComponentName})
initialize with the default set of schemas, but acid.zalan.do/v1 need to be registered also for this case.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions