refactor(connectors): split connectors configs from the runtime config

[mmodzelewski]

No description provided.

[spetz]

Looks good overall! Just wondering, what if someone provides the same path for both, sink and source configs directory? Can we ensure, that it will also work?

[mmodzelewski]

Looks good overall! Just wondering, what if someone provides the same path for both, sink and source configs directory? Can we ensure, that it will also work?

It doesn't work with the current implementation. In such a case, each config is imported as both source and sink. In theory, if the connector binary implements both traits, then it should work, but the configuration would be shared. I think this is an unusual case.
If you think it should work under one dir, then probably I'll need to introduce some discriminator value to the config file itself or rely on a configuration file name convention, e.g. quickwit.sink.toml, random.source.toml

[spetz]

Looks good, thanks! Maybe let's also update the README for connectors runtime, with the new config_dir setting under " minimal viable configuration" info.