Investigate the best way to decide when to read system logs from files or journald

[belimawr]

Debian 12 has stopped writing system logs to traditional log files and now only uses journald by default (see release notes).

This makes the system integration unable to ingest some data because it expects to read direct from files.

We need to find the best way to detect the whether files or journald is used to store the system logs and configure the correct input (log/filestream or journald).

There is a similar issue in the Beats repository to handle the same situatin in Filebeat's system module: elastic/beats#40526.

[mauri870]

Since all Debian 12 installations use systemd-journald, maybe a condition like os == "debian" && version >= 12 is enough? Or this is a more general problem to detect if a Linux OS uses journald vs log files? If it is the later we could probe for some specific files on /var/log (ie dmesg, kern.log, etc) as well as checking if systemd-journald.service is running.

[belimawr]

A condition might be enough to star with, at this moment I'm not sure which information about the distros (like name and flavour) are available to use as conditions in the policy.

It is also a general problem of detecting it in all Linux hosts so we don't have to manually update it whenever a new distro/version starts (or stops) using journald for system logs.

The last bit of the challenge is (maybe not covered in this issue) is how to handle ingest pipelines and other assets that expect the event to be on a specific format (mostly the plain text form the traditional log files) that is different than what the journald input will create.

The ingest pipelines might just be a matter of updating them to also support the events from the journald input as they're capable of quite complex logics.