Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Controller restarts haproxy every 5 minutes with message "restart required : Global config updated: " #652

Closed
odoucet opened this issue May 9, 2024 · 4 comments
Assignees

Comments

@odoucet
Copy link
Contributor

odoucet commented May 9, 2024

When troubleshooting an issue with haproxy logging, I discovered that haproxy process was restarting every five minutes.
It appears that the controller whas restarting haproxy with the following message :
INFO instance/configuration.go:17 [transactionID=xxxxxxxxx] restart required : Global config updated:

The afferent code is here : https://github.com/haproxytech/kubernetes-ingress/blob/master/pkg/controller/global.go#L94

        env.SetGlobal(newGlobal, &newLg, c.haproxy.Env)
	diff := newGlobal.Diff(*global)
	if len(diff) != 0 {
		logger.Error(c.haproxy.GlobalPushConfiguration(*newGlobal))
		instance.Restart("Global config updated: %s", strings.Join(updated, "\n"))
	}

Two issues here :

  • variable updated is not used, so always empty => info message is incomplete.
  • there is definitely an issue because global config is not updated every five minutes on my cluster :)

The code was modified by commit c01d3b3 "MAJOR: crd: add v1 crd and deprecate alpha2" that to me, removed the output that would help here diagnose what the issue is by using a new variable "diff" instead of "updated".

HOW TO REPRODUCE
I get this issue on two different ingressController installed, both having a custom "cr-global" entry.

The cr looks like this :

apiVersion: core.haproxy.org/v1alpha2
kind: Global
metadata:
  name: global-internal
spec:
  config:
    tune_options:
      bufsize: 32768
    ssl_default_bind_ciphers: "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305"
    ssl_default_bind_ciphersuites: "TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256"
    ssl_default_bind_options: "no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets"

I'll make a pull request in a few minutes to at least fix the message, that will help troubleshoot this issue.

@hdurand0710
Copy link
Contributor

Hello @odoucet ,

I was not able to reproduce the issue.
I notice that you use apiVersion: core.haproxy.org/v1alpha2 that is now deprecated.
Could you please try to use apiVersion: ingress.v1.haproxy.org/v1 for your CR and let us know if this is still happening ?
Thanks

@odoucet
Copy link
Contributor Author

odoucet commented May 28, 2024

I've updated apiVersion and it seeems controller does not restart anymore \o/

info message is still incomplete though, please review PR #653 :

INFO    instance/configuration.go:17 [transactionID=8dcb2eb8-6b80-4983-bff7-2c041067b8dc] restart required : Global config updated:
INFO    instance/configuration.go:17 [transactionID=8dcb2eb8-6b80-4983-bff7-2c041067b8dc] restart required : Global log targets updated:

@hdurand0710
Copy link
Contributor

@odoucet ,
Thanks for your PR, it should be merged soon.

@hdurand0710
Copy link
Contributor

@odoucet
the PR has been merged. Can we close this issue ?
Thanks
Helene

@odoucet odoucet closed this as completed May 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants