Log Level Not Setting

[boomam]

MetalLB Version

0.13.12

Deployment method

Charts

Main CNI

Flannel

Kubernetes Version

v1.28.3+k3s2

Cluster Distribution

k3s

Describe the bug

Logging level of MetalLb componnets is not setting to 'warning' as defined in the values.yaml used during deployment.
When i observe the various pods, i can see the logging level is still set as 'info'.
For example, on the controller pod -

W1113 12:40:47.394740       1 warnings.go:70] metallb.io v1beta1 AddressPool is deprecated, consider using IPAddressPool
W1113 12:47:29.392394       1 warnings.go:70] metallb.io v1beta1 AddressPool is deprecated, consider using IPAddressPool
{"level":"info","ts":"2023-11-13T12:53:49Z","logger":"cert-rotation","msg":"no cert refresh needed"}
{"level":"info","ts":"2023-11-13T12:53:49Z","logger":"cert-rotation","msg":"Ensuring CA cert","name":"metallb-webhook-configuration","gvk":"admissionregistration.k8s.io/v1, Kind=ValidatingWebhookConfiguration","name":"metallb-webhook-configuration","gvk":"admissionregistration.k8s.io/v1, Kind=ValidatingWebhookConfiguration"}
{"level":"info","ts":"2023-11-13T12:53:50Z","logger":"cert-rotation","msg":"Ensuring CA cert","name":"addresspools.metallb.io","gvk":"apiextensions.k8s.io/v1, Kind=CustomResourceDefinition","name":"addresspools.metallb.io","gvk":"apiextensions.k8s.io/v1, Kind=CustomResourceDefinition"}
{"level":"info","ts":"2023-11-13T12:53:50Z","logger":"cert-rotation","msg":"Ensuring CA cert","name":"bgppeers.metallb.io","gvk":"apiextensions.k8s.io/v1, Kind=CustomResourceDefinition","name":"bgppeers.metallb.io","gvk":"apiextensions.k8s.io/v1, Kind=CustomResourceDefinition"}
W1113 12:56:59.394627       1 warnings.go:70] metallb.io v1beta1 AddressPool is deprecated, consider using IPAddressPool
W1113 13:02:57.399426       1 warnings.go:70] metallb.io v1beta1 AddressPool is deprecated, consider using IPAddressPool
W1113 13:10:45.406481       1 warnings.go:70] metallb.io v1beta1 AddressPool is deprecated, consider using IPAddressPool
W1113 13:17:18.411224       1 warnings.go:70] metallb.io v1beta1 AddressPool is deprecated, consider using IPAddressPool
W1113 13:25:28.412663       1 warnings.go:70] metallb.io v1beta1 AddressPool is deprecated, consider using IPAddressPool
W1113 13:32:05.419205       1 warnings.go:70] metallb.io v1beta1 AddressPool is deprecated, consider using IPAddressPool
{"level":"info","ts":"2023-11-13T13:36:12Z","logger":"cert-rotation","msg":"no cert refresh needed"}
{"level":"info","ts":"2023-11-13T13:36:12Z","logger":"cert-rotation","msg":"Ensuring CA cert","name":"metallb-webhook-configuration","gvk":"admissionregistration.k8s.io/v1, Kind=ValidatingWebhookConfiguration","name":"metallb-webhook-configuration","gvk":"admissionregistration.k8s.io/v1, Kind=ValidatingWebhookConfiguration"}
{"level":"info","ts":"2023-11-13T13:36:12Z","logger":"cert-rotation","msg":"Ensuring CA cert","name":"addresspools.metallb.io","gvk":"apiextensions.k8s.io/v1, Kind=CustomResourceDefinition","name":"addresspools.metallb.io","gvk":"apiextensions.k8s.io/v1, Kind=CustomResourceDefinition"}
{"level":"info","ts":"2023-11-13T13:36:12Z","logger":"cert-rotation","msg":"Ensuring CA cert","name":"bgppeers.metallb.io","gvk":"apiextensions.k8s.io/v1, Kind=CustomResourceDefinition","name":"bgppeers.metallb.io","gvk":"apiextensions.k8s.io/v1, Kind=CustomResourceDefinition"}
W1113 13:37:13.421321       1 warnings.go:70] metallb.io v1beta1 AddressPool is deprecated, consider using IPAddressPool
{"level":"info","ts":"2023-11-13T13:43:00Z","logger":"cert-rotation","msg":"no cert refresh needed"}
{"level":"info","ts":"2023-11-13T13:43:00Z","logger":"cert-rotation","msg":"Ensuring CA cert","name":"metallb-webhook-configuration","gvk":"admissionregistration.k8s.io/v1, Kind=ValidatingWebhookConfiguration","name":"metallb-webhook-configuration","gvk":"admissionregistration.k8s.io/v1, Kind=ValidatingWebhookConfiguration"}
{"level":"info","ts":"2023-11-13T13:43:00Z","logger":"cert-rotation","msg":"Ensuring CA cert","name":"addresspools.metallb.io","gvk":"apiextensions.k8s.io/v1, Kind=CustomResourceDefinition","name":"addresspools.metallb.io","gvk":"apiextensions.k8s.io/v1, Kind=CustomResourceDefinition"}
{"level":"info","ts":"2023-11-13T13:43:00Z","logger":"cert-rotation","msg":"Ensuring CA cert","name":"bgppeers.metallb.io","gvk":"apiextensions.k8s.io/v1, Kind=CustomResourceDefinition","name":"bgppeers.metallb.io","gvk":"apiextensions.k8s.io/v1, Kind=CustomResourceDefinition"}
W1113 13:44:09.423456       1 warnings.go:70] metallb.io v1beta1 AddressPool is deprecated, consider using IPAddressPool

& As i was asked on slack for "the loglevel parameter of the speakers" -
Looking at the speakers & the controller, i cannot see an environment variable for the log level, however it is specified within 'arguments' as --port=7472 --log-level=warn on both.

To Reproduce

I am not sure if this will reproduce for everyone, but the steps for myself -

1. Remove old metallb namespace and CRDs (this was a migration to the helm chart install method, overall).
2. Create new namespace.
3. Create values.yaml
4. Deploy with helm install metallb metallb/metallb -f .\values.yaml --namespace metallb-system --create-namespace

Expected Behavior

Log level across pods to be set as 'warning'.

Additional Context

values.yaml used for deployment -

controller:
  logLevel: warn
speaker:
  logLevel: warn

For the checkboxes on bug submission, i have ticked them all as they are required fields, however some of these are not possible to do based on the nature of the issue - can't set 'debug' when the logging level wont set.

I've read and agree with the following

• I've checked all open and closed issues and my request is not there.
• I've checked all open and closed pull requests and my request is not there.

I've read and agree with the following

• I've checked all open and closed issues and my issue is not there.
• This bug is reproducible when deploying MetalLB from the main branch
• I have read the troubleshooting guide and I am still not able to make it work
• I checked the logs and MetalLB is not discarding the configuration as not valid
• I enabled the debug logs, collected the information required from the cluster using the collect script and will attach them to the issue
• I will provide the definition of my service and the related endpoint slices and attach them to this issue

[clee]

Still seeing the same buggy behavior here with v0.14.3. I set -log-level=warn for both controller and speaker and yet I'm still getting info-level messages from both.

Weirdly, setting the log-level to debug does work, as I then see debug messages.

[fedepaol]

hey, sorry for the delay!
Those logs are coming from cert-controller, a piece that we use to sign the certificates (a-la cert-manager, but embedded in metallb). We'll work on a way to forward the log level to that part too.

Thanks