Feature: merge multiple "--config" and "--config-directory" flags

[gdunstone]

This adds a feature mentioned in this issue #7501

The only change in functionality is that telegraf will now merge multiple "--config" and "--config-directory" flags into the config, rather than only using value of the last flag.

Required for all PRs:

• Associated README.md updated.
• Has appropriate unit tests.

[telegraf-tiger]

🤝 ✅ CLA has been signed. Thank you!

[srebhan]

While the code looks good to me, you have to fix the CI errors for the windows build before this has a chance to be merged. The error is

# github.com/influxdata/telegraf/cmd/telegraf
cmd\telegraf\telegraf_windows.go:87:7: undefined: fConfig
cmd\telegraf\telegraf_windows.go:88:48: undefined: fConfig
cmd\telegraf\telegraf_windows.go:90:7: undefined: fConfigDirectory
cmd\telegraf\telegraf_windows.go:91:77: undefined: fConfigDirectory

[srebhan]

And make fmt plz! :-)

[gdunstone]

Sorry about that, I figured it was quicker to just fix it rather than reply.

Thanks for reviewing this.

[srebhan]

!retry-failed

[srebhan]

!retry-failed

[srebhan]

Looks good to me.

[Hipska]

Since, config-directory already walks recursively, I don't think it is needed to have the possibility to define multiple on the command-line?

[srebhan]

@Hipska well imagine a case where you have one central (directory) repository that defines the "used-everywhere" plugins and then a machine-local directory for holding the machine-individual configs. Those will likely not be nested. We currently approximate such a use case with symbolic links in one "machine-local" directory but it's... a workaround. :-)

[Hipska]

Both of these directories can be put in a parent directory and that one will be parsed recursively, so I still see no actual use case. It seems better to KISS as much as possible.

[srebhan]

@Hipska let me put an example. The central repository is located on a server and mounted e.g. to /myserver/configs/telegraf, while on some of your machines you additionally want to put a config e.g. for special hardware on this machine only. Let's assume those config files reside on /usr/local/configs/telegraf. Of course you could now merge those two directories on a filesystem level (e.g. merge by copying the files, or (as I do) symlink the two, etc) but how is that better than being able to say
telegraf ... --config-directory /myserver/configs/telegraf --config-directory /usr/local/configs/telegraf?

[gdunstone]

@Hipska Going through a directory recursively doesn't allow you to fetch and merge multiple config files over http.

[Hipska]

Of course you could now merge those two directories on a filesystem level (e.g. merge by copying the files, or (as I do) symlink the two, etc) but how is that better than being able to say
telegraf ... --config-directory /myserver/configs/telegraf --config-directory /usr/local/configs/telegraf?

@srebhan symlink would indeed be better. I would avoid adding 'complex' logic on the command line arguments.

Going through a directory recursively doesn't allow you to fetch and merge multiple config files over http.

@gdunstone Correct, that's why I have absolutely no objection towards multiple --config arguments. I'm only against multiple --config-directory flags.

[gdunstone]

@Hipska I am ambivalent about the --config-directory flag as I don't use it. My main objective with this PR was to allow merging multiple http configs with the --config flag, the --config-directory changes were just because it was convenient and I didn't see why not at the time.

However I can see the use case @srebhan provides.

[srebhan]

@Hipska I still don't see the complex logic point, but to cut a long story short: I can live with only one config directory option even though I'd prefer multiple ones. ;-)

[Hipska]

And I prefer not 😂 Best would be someone from Influx to give their thoughts.

If you want to include multiple different directories, I think it would be better to make that an option in the toml config rather than on the command line.

[srebhan]

The only difference is: Now only you is happy, with the change we can both be happy as nobody forces you to use the option multiple times.

[Hipska]

You are right 😛 But maybe put it on the table for the weekly meeting to know their view on it?

[ssoroka]

I'm okay with multiple --config and multiple --config-directory flags. It adds flexibility but doesn't harm the users who are not using it. This is good to go once my comment is resolved.