Repository navigation
YAML glob includes: extra pipeline.outputs are dropped if that file loads before the main pipeline file (5.1.2) #12745
Unanswered
udaydabbeta
asked this question in
Q&A
Replies: 1 comment 6 replies
|
I tried to reproduce this on |
6 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Question
If several YAML files each have
pipeline.outputs, and the main config loads them with a glob, are outputs from a file that loads before the main pipeline file supposed to be dropped?Fluent Bit loads that glob in filename order (A–Z) and merges into one pipeline.
extra-output.yml has only pipeline.outputs (extra stdout or kafka). Name starts with E, so it loads first.
main.yml has pipeline.inputs and pipeline.outputs. Name starts with M, so it loads second.
Filter-only files still load even if they come first.
What we see: only main.yml outputs run. Extra outputs never show in logs or /api/v1/metrics. No error. Config test still passes.
Proof it is order: rename extra-output.yml to zz-extra-output.yml (loads after main.yml) and restart. Same YAML. Both outputs are live (main_stdout and extra_stdout, or loki.0 plus kafka.5). Dummy/records go to both.
Hot reload does not bring the dropped outputs back.
Tiny repro
conf/apps/extra-output.yml (loads first):
conf/apps/main.yml:
Before rename: metrics show only main_stdout. After zz- rename + restart: both.
Docs show a literal include list, not *.yml, and do not say an earlier pipeline.outputs list can disappear.
Will a fix concatenate pipeline.outputs from every included file, in any order?
Full write-up is in the bug issue #12724. Thanks.
All reactions