Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OCPBUGS-30145: fix: Prevented memory leak generated by not correctly cleaned HTTP resources (#5293) #26

Merged
merged 1 commit into from
Mar 27, 2024

Conversation

jkyros
Copy link

@jkyros jkyros commented Mar 27, 2024

We had a memory leak situation where scalers with improper fields in the metadata (e.g. namespaces that did not exist) were causing memory to leak because http clients weren't getting cleaned up.

It just so happens that Joel noticed this was fixed upstream already in 2.13 (thanks Joel!), so this cherry-picks it back from 2.13 to 2.12.

With this fix, memory usage seems to plateau at 72Meg (for me, with 1 scaledObject) and no longer leaks.

NOTE: It wasn't a completely clean pick, I had to mess with the openstack swift scaler, they changed clients between 2.12 and 2.13 so the code didn't line up, but I don't think I broke anything.

Fixes # OCPBUGS-30145

@openshift-ci-robot openshift-ci-robot added jira/severity-moderate Referenced Jira bug's severity is moderate for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Mar 27, 2024
@openshift-ci-robot
Copy link

@jkyros: This pull request references Jira Issue OCPBUGS-30145, which is invalid:

  • expected the bug to target the "4.16.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

We had a memory leak situation where scalers with improper fields in the metadata (e.g. namespaces that did not exist) were causing memory to leak because http clients weren't getting cleaned up.

It just so happens that Joel noticed this was fixed upstream already in 2.13 (thanks Joel!), so this cherry-picks it back from 2.13 to 2.12.

With this fix, memory usage seems to plateau at 72Meg (for me, with 1 scaledObject) and no longer leaks.

Fixes # OCPBUGS-30145

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot requested a review from joelsmith March 27, 2024 00:33
@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 27, 2024
@jkyros
Copy link
Author

jkyros commented Mar 27, 2024

/jira refresh

@openshift-ci-robot openshift-ci-robot added the jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. label Mar 27, 2024
@openshift-ci-robot
Copy link

@jkyros: This pull request references Jira Issue OCPBUGS-30145, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.16.0) matches configured target version for branch (4.16.0)
  • bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

No GitHub users were found matching the public email listed for the QA contact in Jira (schoudha@redhat.com), skipping review request.

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot removed the jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. label Mar 27, 2024
@openshift-ci-robot
Copy link

@jkyros: This pull request references Jira Issue OCPBUGS-30145, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.16.0) matches configured target version for branch (4.16.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

No GitHub users were found matching the public email listed for the QA contact in Jira (schoudha@redhat.com), skipping review request.

In response to this:

We had a memory leak situation where scalers with improper fields in the metadata (e.g. namespaces that did not exist) were causing memory to leak because http clients weren't getting cleaned up.

It just so happens that Joel noticed this was fixed upstream already in 2.13 (thanks Joel!), so this cherry-picks it back from 2.13 to 2.12.

With this fix, memory usage seems to plateau at 72Meg (for me, with 1 scaledObject) and no longer leaks.

NOTE: It wasn't a completely clean pick, I had to mess with the openstack swift scaler, they changed clients between 2.12 and 2.13 so the code didn't line up, but I don't think I broke anything.

Fixes # OCPBUGS-30145

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

… cleaned HTTP resources (kedacore#5293)

Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>
Signed-off-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es>

This wasn't a completely clean pick, I had to adjust the swift scaler
client variable to be a pointer so it was nillable, but everything else
was okay.
@joelsmith
Copy link

joelsmith commented Mar 27, 2024

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Mar 27, 2024
Copy link

openshift-ci bot commented Mar 27, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jkyros, joelsmith

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@joelsmith
Copy link

joelsmith commented Mar 27, 2024

/override ci/prow/keda-e2e-aws-ovn
Ignoring known-flaky test

Copy link

openshift-ci bot commented Mar 27, 2024

@joelsmith: Overrode contexts on behalf of joelsmith: ci/prow/keda-e2e-aws-ovn

In response to this:

/override ci/prow/keda-e2e-aws-ovn

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Copy link

openshift-ci bot commented Mar 27, 2024

@jkyros: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@openshift-merge-bot openshift-merge-bot bot merged commit dc4c94b into openshift:main Mar 27, 2024
8 checks passed
@openshift-ci-robot
Copy link

@jkyros: Jira Issue OCPBUGS-30145: All pull requests linked via external trackers have merged:

Jira Issue OCPBUGS-30145 has been moved to the MODIFIED state.

In response to this:

We had a memory leak situation where scalers with improper fields in the metadata (e.g. namespaces that did not exist) were causing memory to leak because http clients weren't getting cleaned up.

It just so happens that Joel noticed this was fixed upstream already in 2.13 (thanks Joel!), so this cherry-picks it back from 2.13 to 2.12.

With this fix, memory usage seems to plateau at 72Meg (for me, with 1 scaledObject) and no longer leaks.

NOTE: It wasn't a completely clean pick, I had to mess with the openstack swift scaler, they changed clients between 2.12 and 2.13 so the code didn't line up, but I don't think I broke anything.

Fixes # OCPBUGS-30145

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/severity-moderate Referenced Jira bug's severity is moderate for the branch this PR is targeting. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants