Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add work queue bound on LicenseException retries in RPMConnectionServ… #1606

Merged
merged 2 commits into from Nov 28, 2023

Conversation

kanderson250
Copy link
Contributor

Fixes #1428

This PR addresses OutOfMemory exceptions that were reported during rotation to an invalid license key. The cause of the memory leak was found to be excessive creation of RPMConnectionTasks. To mitigate the issue, the number of RPMConnectionTasks created during processing of a LicenseException is now bounded.

  • The ScheduledExecutorService is replaced with a ScheduledThreadPoolExecutor, in order to get access to the size of its work queue.
  • The LicenseException handler in the doConnect method of RPMService limits reconnect attempts when the work queue acquires a certain number of tasks.
  • The number of tasks is bounded at 5000.

There is no way for the agent to recover from a bad license key, so although these mitigation steps will prevent a memory leak, a JVM restart is still required to get the agent back up.

@codecov-commenter
Copy link

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (c94127d) 70.61% compared to head (152ca75) 70.63%.
Report is 114 commits behind head on main.

Files Patch % Lines
...m/newrelic/agent/rpm/RPMConnectionServiceImpl.java 50.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #1606      +/-   ##
============================================
+ Coverage     70.61%   70.63%   +0.02%     
- Complexity     9761     9913     +152     
============================================
  Files           814      826      +12     
  Lines         39313    39804     +491     
  Branches       5970     6011      +41     
============================================
+ Hits          27759    28115     +356     
- Misses         8866     8975     +109     
- Partials       2688     2714      +26     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@kanderson250 kanderson250 merged commit 86ffe28 into main Nov 28, 2023
102 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

OOM on API Key change
3 participants