Use logging consistently

[andreasgerstmayr]

{"level":"info","ts":"2023-05-30T16:06:33.314811295Z","msg":"Starting EventSource","controller":"tempostack","controllerGroup":"tempo.grafana.com","controllerKind":"TempoStack","source":"kind source: *v1.Route"}
{"level":"info","ts":"2023-05-30T16:06:33.314827535Z","msg":"Starting EventSource","controller":"tempostack","controllerGroup":"tempo.grafana.com","controllerKind":"TempoStack","source":"kind source: *v1.Secret"}
{"level":"info","ts":"2023-05-30T16:06:33.314885107Z","msg":"Starting Controller","controller":"tempostack","controllerGroup":"tempo.grafana.com","controllerKind":"TempoStack"}
{"level":"info","ts":"2023-05-30T16:06:34.124201867Z","msg":"Starting workers","controller":"tempostack","controllerGroup":"tempo.grafana.com","controllerKind":"TempoStack","worker count":1}
W0530 16:08:21.838587 1 reflector.go:424] pkg/mod/k8s.io/client-go@v0.25.0/tools/cache/reflector.go:169: failed to list *v1.PrometheusRule: prometheusrules.monitoring.coreos.com is forbidden: User "system:serviceaccount:openshift-operators:tempo-operator-controller-manager" cannot list resource "prometheusrules" in API group "monitoring.coreos.com" at the cluster scope
E0530 16:08:21.838661 1 reflector.go:140] pkg/mod/k8s.io/client-go@v0.25.0/tools/cache/reflector.go:169: Failed to watch *v1.PrometheusRule: failed to list *v1.PrometheusRule: prometheusrules.monitoring.coreos.com is forbidden: User "system:serviceaccount:openshift-operators:tempo-operator-controller-manager" cannot list resource "prometheusrules" in API group "monitoring.coreos.com" at the cluster scope
W0530 16:08:22.989190 1 reflector.go:424] pkg/mod/k8s.io/client-go@v0.25.0/tools/cache/reflector.go:169: failed to list *v1.PrometheusRule: prometheusrules.monitoring.coreos.com is forbidden: User "system:serviceaccount:openshift-operators:tempo-operator-controller-manager" cannot list resource "prometheusrules" in API group "monitoring.coreos.com" at the cluster scope
E0530 16:08:22.989244 1 reflector.go:140] pkg/mod/k8s.io/client-go@v0.25.0/tools/cache/reflector.go:169: Failed to watch *v1.PrometheusRule: failed to list *v1.PrometheusRule: prometheusrules.monitoring.coreos.com is forbidden: User "system:serviceaccount:openshift-operators:tempo-operator-controller-manager" cannot list resource "prometheusrules" in API group "monitoring.coreos.com" at the cluster scope

The last error messages don't have the proper formatting.

Tasks

Beta Give feedback

1. use log handler in operator-sdk
2. specify logger names for every log message
3. adjust log levels ("resource has been unchanged")

[iblancasa]

Those logging messages are generated by the client-go library. It seems we're not the only ones having some problems with those error/warning messages: kubernetes/client-go#1247

[andreasgerstmayr]

I think this happens if we return an error in the Reconcile() function, but I didn't debug it much yet.

reflector.go calls HandleError (https://github.com/kubernetes/client-go/blob/bbdc95deee6fdda42bba28a631130978a67163bd/tools/cache/reflector.go#L148), which is defined here: https://github.com/kubernetes/apimachinery/blob/v0.27.2/pkg/util/runtime/runtime.go#L102 - maybe there is a way to add a custom ErrorHandler which uses our logger instead of the default one.