The systemd plugin is stalled reading events.
If you do in one shell: journalctl -f -o json and in other: fluent-bit -i systemd -prop=Read_From_Tail=true -p DB=journald.db -p tag='host.*' -o stdout -f 1 -v and you do for example sudo su - you don't see all the events in the fluent-bit output. If you restart the fluent-bit you see the missing events.
If you do the same without -p tag='host.*' it works.
The problem is in:
|
if (mp_sbuf.size > 1024000 || |
|
((last_tag_len != tag_len) || strncmp(last_tag, tag, tag_len) != 0)) { |
When you hit he condition, it's easy to hit the condition of different tags, it will break the loop of sd_journal_next and in_systemd_collect returns FLB_SYSTEMD_MORE.
The problem is that there is remaining data to read but you will not read it until new data arrives.
When fluent-bit starts it works well because in_systemd_collect is called from in_systemd_collect_archive and controls the returns codes of in_systemd_collect and generates a event when there is more data to read, but the first time that in_systemd_collect reached the end of the journal systemd_collect_archive register in_systemd_collect with flb_input_set_collector_event and in_systemd_collect_archive it's not called any more.
I hope I have explained the problem.
The systemd plugin is stalled reading events.
If you do in one shell:
journalctl -f -o jsonand in other:fluent-bit -i systemd -prop=Read_From_Tail=true -p DB=journald.db -p tag='host.*' -o stdout -f 1 -vand you do for examplesudo su -you don't see all the events in the fluent-bit output. If you restart the fluent-bit you see the missing events.If you do the same without
-p tag='host.*'it works.The problem is in:
fluent-bit/plugins/in_systemd/systemd.c
Lines 168 to 169 in 23b6171
When you hit he condition, it's easy to hit the condition of different tags, it will break the loop of
sd_journal_nextandin_systemd_collectreturnsFLB_SYSTEMD_MORE.The problem is that there is remaining data to read but you will not read it until new data arrives.
When fluent-bit starts it works well because
in_systemd_collectis called fromin_systemd_collect_archiveand controls the returns codes ofin_systemd_collectand generates a event when there is more data to read, but the first time thatin_systemd_collectreached the end of the journalsystemd_collect_archiveregisterin_systemd_collectwithflb_input_set_collector_eventandin_systemd_collect_archiveit's not called any more.I hope I have explained the problem.