Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

systemd-Journal by file #16038

Merged
merged 92 commits into from Oct 2, 2023
Merged

systemd-Journal by file #16038

merged 92 commits into from Oct 2, 2023

Conversation

ktsaou
Copy link
Member

@ktsaou ktsaou commented Sep 24, 2023

Operating systemd-journal file-by-file is about 2x faster compared to the default.

On my tests, the current master version of systemd-journal needs 29.2 seconds, while the new one needs 13.5 seconds, for a given time-frame query.

The changes on this PR:

  • maintain a registry of journal files
  • add support for volatile journal files, even when netdata runs in a container (/run/log/journal)
  • display the actual realtime timestamp associated with a log entry. The current master presents the timestamp the log was received by journald, while this PR extracts the actual timestamp the log was generated.
  • overloads fstat64() libc system call, to speed up libsystemd (Journalctl: 13x faster by caching fstat() and optimizing timestamps retrieval systemd/systemd#29261)
  • support for multiple log sources (including system and user logs, journald namespaces, logs from remote hosts), allowing the user to select the sources of interest.
  • fixed several issues related to forward/backward data-only queries
  • support for slicing the logs using journal matches

image

@github-actions github-actions bot added area/collectors Everything related to data collection collectors/systemd-journal labels Sep 24, 2023
@github-actions github-actions bot added the area/build Build system (autotools and cmake). label Sep 25, 2023
@Ancairon
Copy link
Member

Ancairon commented Oct 2, 2023

Morning! Just a note, I see a README file in the additions, (I see you used the old way of adding metadata to it, doesn't work anymore, no worries) I can add it to Learn, under Data Collection as its own entry (outside of integrations folders, on the same level as System metrics and the likes. For it to appear as an integration we would need to have it inside integrations.js (I feel this is not the goal here, so when this is merged I will add it to the spot I mentioned). Just FYI

@ktsaou ktsaou merged commit 9911045 into netdata:master Oct 2, 2023
141 of 142 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants