-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: Prevented memory leak generated by not correctly cleaned HTTP resources #5293
Conversation
…onnections Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>
Thank you for your contribution! 🙏 We will review your PR as soon as possible. While you are waiting, make sure to:
Learn more about: |
Signed-off-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es>
/run-e2e |
Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>
Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>
/run-e2e |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, great job!
…sources (kedacore#5293) Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es> Signed-off-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es> Signed-off-by: anton.lysina <alysina@gmail.com>
… cleaned HTTP resources (kedacore#5293) Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es> Signed-off-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es>
… cleaned HTTP resources (kedacore#5293) Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es> Signed-off-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es>
… cleaned HTTP resources (kedacore#5293) Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es> Signed-off-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es> This wasn't a completely clean pick, I had to adjust the swift scaler client variable to be a pointer so it was nillable, but everything else was okay.
OCPBUGS-30145: fix: Prevented memory leak generated by not correctly cleaned HTTP resources (kedacore#5293)
"scaler": "prometheusScaler", "error": "prometheus query api returned error. status: 403 response: {"message":"Credential should be scoped to a valid region. "}"} any suggestions on this error |
This issue was closed 4 months ago and it's totally unrelated with your topic. I'd suggest opening a new issue for your case |
After profiling KEDA operator looking for this memory leak, I found a weird behavior.
When I deploy the OP given SO, during the first minutes, prometheus trigger returns 404 and the memory grows, but at some point, the memory usage becomes stable and then prometheus trigger start to produce timeouts.
This impacts in 2 way, significativelly increasing the allocation of HTTP resources
![image](https://private-user-images.githubusercontent.com/36899226/290934004-d5c5b669-0213-446a-8492-de506668707c.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjM3MDM5NDksIm5iZiI6MTcyMzcwMzY0OSwicGF0aCI6Ii8zNjg5OTIyNi8yOTA5MzQwMDQtZDVjNWI2NjktMDIxMy00NDZhLTg0OTItZGU1MDY2Njg3MDdjLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTUlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODE1VDA2MzQwOVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWM1NmIyZmQ5YTgzMzAxYWQzZWY0MmJlZTliY2ZiNjFmM2Q4Nzg1OWQ0ZjgwOWJlMDIzNDAwNjBmMjk1NTBlZWEmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.k9TrLe6HGaFW_SHR3K60SnWGYuOhqKApDOGJJ_6BvZY)
And also increasing the size of the in use memory due to stored rootCAs of the clients (we are cloning the current cached certs but we aren't modifying the cloned item, so doesn't make sense, at least atm)
In order to solve these issues, this PR does:
(scaler).Close(context.Context)
to close them deterministically and not delegating this action to the garbage collector. This helps to reduce the allocated resources for stablishing connectionsChecklist
Fixes #5248