Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reset the obsolete flag on service thread #15892

Merged
merged 5 commits into from Sep 6, 2023

Conversation

MrZammler
Copy link
Contributor

Summary

This PR makes 2 small changes:

The first is if the service thread does not clean a chart (because of timing factors) to re-set the host wide RRDHOST_FLAG_PENDING_OBSOLETE_CHARTS flag to re-try on the next iteration.

The second change is to execute rrdset_free in all memory modes.

The goal is to see whether memory usage can be reduced in case of many ephemeral charts.

Test Plan
Additional Information
For users: How does this change affect me?

@MrZammler
Copy link
Contributor Author

This change appears to somewhat alleviate the case of #15066

@MrZammler
Copy link
Contributor Author

Noticed on an agent running this patch the following during shutdown:

2023-08-31 12:22:26: netdata INFO  : DAEMON_COMMAND : (1696@database/sqlite/sqli:metadata_sync_s): METADATA: Sending a shutdown command
2023-08-31 12:22:26: netdata INFO  : DAEMON_COMMAND : (1701@database/sqlite/sqli:metadata_sync_s): METADATA: Waiting for shutdown ACK
2023-08-31 12:22:26: netdata INFO  : METASYNC : (1675@database/sqlite/sqli:metadata_event_): METADATA: Shutting down event loop
2023-08-31 12:22:26: netdata INFO  : DAEMON_COMMAND : (1704@database/sqlite/sqli:metadata_sync_s): METADATA: Shutdown complete
2023-08-31 12:22:26: netdata INFO  : DAEMON_COMMAND : (0440@daemon/main.c       :netdata_cleanup): NETDATA SHUTDOWN: in       0 ms, stop metasync threads - next: wait for dbengine collectors to finish
2023-08-31 12:22:26: netdata ERROR : DAEMON_COMMAND : (0450@daemon/main.c       :netdata_cleanup): waiting for 2142 collectors to finish (sleeping for 100000 microseconds every time this happens)
2023-08-31 12:22:26: netdata ERROR : DAEMON_COMMAND : (0450@daemon/main.c       :netdata_cleanup): waiting for 2142 collectors to finish (similar messages repeated 3 times in the last 1 secs) (sleeping for 100000 microseconds every time this happens)
2023-08-31 12:22:27: netdata ERROR : DAEMON_COMMAND : (0450@daemon/main.c       :netdata_cleanup): waiting for 2142 collectors to finish (similar messages repeated 10 times in the last 1 secs) (sleeping for 100000 microseconds every time this happens)
....

8 minutes later the same message repeats. Not sure if it's because of this patch or not, will have a look.

@github-actions github-actions bot added area/web area/health area/collectors Everything related to data collection area/database area/ml Machine Learning Related Issues collectors/plugins.d labels Sep 4, 2023
@MrZammler MrZammler marked this pull request as ready for review September 5, 2023 09:34
Copy link
Contributor

@thiagoftsm thiagoftsm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR worked as expected when I obsolete a thread in eBPF.plugin, LGTM!

@MrZammler MrZammler merged commit f059f4d into netdata:master Sep 6, 2023
137 checks passed
@MrZammler MrZammler deleted the service-reset-obsolete-flag branch September 6, 2023 08:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants