Add Jsonnet mixin for deploying alongside kube-prometheus

[jrobersonaquent]

Description

This is a Jsonnet mixin designed to be used with kube-prometheus that sets up the remote write configs needed to send metrics to fluentd.

This also includes a quick doc on how to configure and use it with kube-prometheus.

Testing performed

• ci/build.sh
• Redeploy fluentd and fluentd-events pods
• Confirm events, logs, and metrics are coming in (tested and working in my company's dev k8s cluster)

[samjsong]

Thank you for the contribution! LGTM, but I've tagged members of the team so we can take a look

[frankreno]

LGTM to me, thanks for the PR!

[rvmiller89]

I'm fine with merging the doc about kube-prometheus and how to write the mixin.

However I'm nervous about merging the libsonnet file itself, since we currently have one source of truth for remote write metric configuration, and adding this file introduces the possibility for configuration drift. It is also non-trivial to verify the libsonnet file currently matches our remote write configuration.

For that reason, I'd ask that the libsonnet file either be omitted from this PR, or renamed so something like kube-prometheus-sumo-logic-mixin.libsonnet.example with the doc clarifying this file is only an example and may not be up-to-date with the remote write metric configuration. Ideally, this file could be generated as part of our CI based on our remote write configuration, so that it is always up-to-date.

[jrobersonaquent]

@rvmiller89 I renamed it like you suggested and I'm working on another PR to update the Jsonnet file automatically.

[rvmiller89]

Thanks for updating!