Can't configure storage with the conf.arg parameter #109
Comments
|
So, the change is really small. Without much of the understanding, just mostly cut-n-paste: --- in_systemd.rb.orig 2023-11-03 17:37:20.875477097 +0000
+++ in_systemd.rb 2023-11-04 00:17:33.088467944 +0000
@@ -50,7 +50,8 @@ module Fluent
def configure(conf)
super
@journal = nil
- @pos_storage = storage_create(usage: 'positions')
+ config = conf.elements.find{|e| e.name == 'storage' }
+ @pos_storage = storage_create(usage: 'positions', conf: config, default_type: DEFAULT_STORAGE_TYPE)
@mutator = SystemdEntryMutator.new(**@entry_opts.to_h)
@mutator.warnings.each { |warning| log.warn(warning) }
endIf that seems legitimate, I can make an MR, although that diff is trivial to apply by hand as well. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I'm new to the
fluentdso it could be that I'm missing something. Trying to configure it to send all the busyjournaldlogs to the OpenSearch I've tried to configure it with multiple workers, so I could perform several parallel processes. Apparentlyfluent-plugin-systemdstill doesn't support multi-worker setup, so I've tried to assign input to one fixed worker.That seems to work, but the need to specify storage
pathlooked overwhelming(I wanted to keep all work files in the more appropriate/var/lib/fluentdir rather than/var/log/fluent). Especially, as the documentation refers to the storage plugin without any limitations.So, the configuration I tried first was:
That worked, producing a position file
/var/lib/fluent/systemd-cursor/worker0/storage.json. So far so good. But the need to specify/var/lib/fluentseems to be excessive.So, my next(2) try was:
That also worked, producing
/var/lib/fluent/worker0/systemd/storage.json. That already brings questions, as the name of the cursor file got hard coded to thestorage.json, as well as in the first variant, but there you at least get a customsystemd-cursorin the path which may make the difference in case there are two or more inputs with@type systemd(pretty common setup for K8s, apparently).Also, log got:
which explains the result above.
Ok, how about(3) the
conf.argfrom the documentation?That simply doesn't work, failing with the:
At this point, I thought that I was doing something wrong, and I re-read the documentation several times as well as tried to understand the Ruby source code(not in my portfolio).
In a few other plugins, I found the way how (local) storage is configured and modified the code accordingly.
Amusingly that helped and I got the desired
/var/lib/fluent/worker0/systemd/storage.systemd_cursor.json!Without
conf.argit gives the expected/var/lib/fluent/worker0/systemd/storage.json.Configuration (2) also works the same way with the message about redundant
@id(I was hoping it'll actually pick the parameter).I haven't tried though how all those permutations would work without the multiple workers.
The text was updated successfully, but these errors were encountered: