Expose validation in the API

[yoelk]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases

The way we use vector, is running in a k8s cluster and we have a separate service that configures vector, running in another pod.
It would be helpful if we could use the API to ask vector to validate a certain configuration, instead of putting it in the configuration folder and waiting for errors.

This is especially useful when dealing with S3 authentication (e.g. wrong ARN defined)

Attempted Solutions

No response

Proposal

No response

References

No response

Version

No response

[smitthakkar96]

@jszwedko do you see API supporting partial config validation at some point?

Our configuration might be spread out where base configuration for sinks and sources is managed by a different team, and the configuration for transforms (filter, sample etc) is democratized and is part of application code repos as those transforms are specific to particular application. It's good if it sits closer to the application repo. Ofcourse we could add an intermediate step in CI to fetch configuration and merge it before validation however being able to validate partial configuration against a running vector might simplify things a lot from a CI perspective.

[jszwedko]

@jszwedko do you see API supporting partial config validation at some point?

Hey! Yeah, I could see the API supporting this sort of thing in the future. Given it is just validation, rather than mutation, I think we could get away with adding it to the current API without needing to think about authentication.