Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Boost leaking memory when a large number of Data Sources disabled #3477

Closed
TheWitness opened this issue Apr 19, 2020 · 2 comments
Closed

Boost leaking memory when a large number of Data Sources disabled #3477

TheWitness opened this issue Apr 19, 2020 · 2 comments
Labels
blocker Blocking release boost Boost related issue bug Undesired behaviour confirmed Bug is confirm by dev team performance Performance related affects big sites resolved A fixed issue
Milestone

Comments

@TheWitness
Copy link
Member

Describe the bug

During a Re-Index, if a Data Source is disabled, Cacti's API is not purging it's records from the Boost table resulting in memory leakage and failed boost runs.

To Reproduce

Steps to reproduce the behavior:

  1. Create 10's of thousands of Data Sources from Data Queries

  2. Have the re-index not fail, bur return only a fraction of the records.

  3. Ensure your boost table has > 6M records

  4. Attempt to do a boost run and watch the memory value. Boost can consume several GB of memory.

Expected behavior

Boost should not leak memory, Cacti should remove these Data Sources from the Boost tables as well during the disable process.

@TheWitness TheWitness added bug Undesired behaviour unverified Some days we don't have a clue blocker Blocking release boost Boost related issue and removed unverified Some days we don't have a clue labels Apr 19, 2020
@TheWitness TheWitness added this to the 1.2.12 milestone Apr 19, 2020
TheWitness added a commit that referenced this issue Apr 19, 2020
Boost leaking memory when a large number of Data Sources disabled
@TheWitness TheWitness added performance Performance related affects big sites resolved A fixed issue labels Apr 19, 2020
@ddb4github
Copy link
Contributor

Just one comment:
function boost_rrdtool_function_update have only two return_value '' or OK

  • In most case, function boost_rrdtool_function_update return OK
  • If rrdfile not exist, and create fail, function boost_rrdtool_function_update return ''
  • boost_process_output output WARN message if return_value is ''
  • boost_process_poller_output always ignore return_value whatever '' and OK

So I prefer to ignore '' in boost_process_output, and do not output WARN for each disabled DataSource:

2020/04/21 05:17:45 - BOOST WARNING: RRD Update Warning '' for Local Data ID '97063'

@TheWitness
Copy link
Member Author

I agree.

@netniV netniV added the confirmed Bug is confirm by dev team label Apr 26, 2020
@github-actions github-actions bot locked and limited conversation to collaborators Jul 26, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
blocker Blocking release boost Boost related issue bug Undesired behaviour confirmed Bug is confirm by dev team performance Performance related affects big sites resolved A fixed issue
Projects
None yet
Development

No branches or pull requests

3 participants