K8S diagnostics image with debugging tools

[pmm-sumo]

Description

This adds a diagnostics image that is a place to keep various tools helping to solve potential issues with K8S deployment quickly. It starts with K8S API tools but can be easily extended.

Testing performed

• ci/build.sh
• Redeploy fluentd and fluentd-events pods
• Confirm events, logs, and metrics are coming in

[sumo-drosiek]

LGTM, I like the idea :)

[samjsong]

LGTM I also like this idea. I think there were some ideas previously around a script that can check your environment for other prerequisites, so tagging @frankreno for his input as well

[frankreno]

In general I love this idea and it was something that @abhi-sumo built a protoype CLI for that went a bit farther than this. It analyze the clsuter and determine the optimimal settings to install with. The intention was to reduce the cognitive overload for users. it likely needs some imporvements but we might be able to learn from the concepts of this. - https://github.com/abhi-sumo/sumo-k8s-cli

[vsinghal13]

LGTM Thanks @pmm-sumo !

[pmm-sumo]

In general I love this idea and it was something that @abhi-sumo built a protoype CLI for that went a bit farther than this. It analyze the clsuter and determine the optimimal settings to install with. The intention was to reduce the cognitive overload for users. it likely needs some imporvements but we might be able to learn from the concepts of this. - https://github.com/abhi-sumo/sumo-k8s-cli

Thank you @frankreno , I think that @abhi-sumo's project serves a slightly different purpose. If I read the code right, it provides a wizard-like experience for the setup.

What I am proposing here is more like a standalone image that can be very easily executed to check what's the environment (no installation is even needed). It can be extended in a bunch of ways. I started with making it provide some answers that will help with figuring out why metadata tagging is not working for one of the alpha customers in a convenient way

[sumo-drosiek]

Does it make sense to merge both ideas?.
I believe the investigation and installation wizard can coexist in one tooling image :)

I know that this code is pretty urgent right now, but we can make some effort in the future to merge both solutions

[perk-sumo]

I like this idea but I think the name of the image should not be kubernetes-diagnostics but rather kubernetes-tools as this additional image might be used for many other side things (eg. generating helm templates etc.)

[perk-sumo]

(comment duplicate)

[sumo-drosiek]

Great. I would like to have kubectl run tools --generator=run-pod/v1 -it --rm --restart=Never --image sumologic/kubernetes-tools -- check, but I will take care about it while working on template generator :)

[frankreno]

LGTM

[perk-sumo]

LGTM :)