Ability to add more outputs for logs

[tillig]

Is your feature request related to a problem? Please describe.

The current output.conf labels all logs as @NORMAL and then has the complete <label @NORMAL> block in it. While I'm not a Fluentd configuration expert, it appears that this groups all the logs coming in and routes them only to this <label> section which only has ElasticSearch in it.

The ElasticSearch configuration is actually pretty complex and generated (nicely!) by the Helm chart. I like that and don't want to replicate it.

However, I have a need to also route the logs to another location. I don't really have the ability to insert anything inside that <label @NORMAL> section, and if I add a configuration file alphabetically later than the ElasticSearch one (say I add output.other.conf, which will be loaded by Fluentd alphabetically after output.conf) then I don't get the opportunity to relabel the logs or also handle <label @NORMAL> - you can only define that block once.

Describe the solution you'd like

I'd like the ability to add a block of configuration inside the <label @NORMAL> block; or possibly not even have the labeling go on at all so there isn't any routing issue. I'd like this so I can continue to take advantage of the nicely generated ElasticSearch configuration without having to copy/paste that block into an override of the whole output.conf.

Describe alternatives you've considered

One workaround is to replace output.conf with a copy/paste version in your own values.yaml. And this does work, but you have to be "smart" about what values to put in the ElasticSearch configuration based on how ES is configured; and the Helm chart setup/values don't end up getting reflected in output.conf changes.

A second workaround is to name my config alphabetically before output.conf (like outpus.conf - one letter before the default - and do some interesting relabeling there:

  outpus.conf: |-
    <match **>
      @type relabel
      @label @MYOUTPUT
    </match>
    <label @MYOUTPUT>
    <match **>
      @type my_output_here
      some_configuration_and_then
    </match>
    <match **>
      @type relabel
      @label @NORMAL
    </match>
    </label>

This works, but it feels kinda hacky. Maybe that's OK? It definitely wasn't an obvious thing.

As noted I did try adding a second output.other.conf with my additional source but I can't also handle <label @NORMAL> - Fluentd doesn't like the duplicate block.