k8s-cluster-checker

k8s-cluster-checker is bundle of python scripts which can be used to analyse below configurations in a kubernetes cluster:

• OS version(supports flatcar OS, coreOS & Ubuntu only)
• Kubernetes version
• Docker version
• Admission Controllers
• security-context of workloads
• health-probes of workloads
• QoS of workloads
• types of services
• workload running with single replica
• rbac analysis
• stale namespaces with no workloads

Once the tool is run, it generates output in 3 ways:

1. stdout on the screen to visualise the analysis right away.
2. report in csv files are generated for each analysis. A combined report is generated in excel file. You can use it for your own custom analysis.
3. json output is generated for each analysis which can be consumed in down-stream scripts. JSON output can also be ingested in Splunk or any other log/data aggregation tool for dashboarding.

Compatible k8s versions: 1.14+

Running k8s-cluster-check on older k8s version 1.10.x to 1.13.x may result in missing results/exceptions. Tool do not support k8s version previous to 1.10.x.

This tool performs read-only operations on any k8s cluster. You can make a service account/kubeconfig with full read-only access to all k8s-objects and use the same to run the tool. Else, it will use the in-cluster kubeconfig when deployed in the cluster.

k8s-cluster-checker contains below scripts:

1. cluster.py: gives quick details of cluster and analyses configurations as below:

   • cluster name
   • node-details
   • node-roles
   • volumes used
   • OS, K8s and docker version. Also checks for latest versions.
   • overall and namespaced workload count
   • analysis of admision-controllers
   • analysis of security configs for workloads
   • analysis of health-check probes
   • analysis of resource limits/requests of workloads and their QoS
   • analysis of image-pull-policy
   • RBAC analysis
   • analysis of services in the cluster.

   above analysis is the collective result of following scripts.

2. nodes.py: gives details of nodes. Finds if docker, kubernetes and docker version are latest or not.

3. namespace.py: give details of namespace objects and analyses them

4. control_plane.py: analyses control-plane configuration and reports missing ones

5. deployments.py: gives detail for deployments in cluster and analyses them

6. daemonsets.py: gives detail for daemonsets in cluster and analyses them

7. statefulsets.py: gives detail for statefulsets in cluster and analyses them

8. services.py: gives detail for services in cluster and analyses them

9. jobs.py: gives detail for jobs in cluster and analyses them

10. pods.py: gives detail for pods in cluster in all namespaces and analyses them

11. ingress.py: gives detail for ingress in cluster and analyses them

12. rbac.py: gives detail for rbac in cluster and analyses them

13. images.py: gives detail for images used by workloads in cluster and reports back if any old images found.

Pre-requisites

1. python3 and packages

2. pip3 needs to be installed to get required packages. You need to install above packages with command:

   pip3 install <package-name>
   

   A docker image is available on dockerhub with all the dependencies installed. Follow this readme for docker image instructions.

3. KUBECONFIG for the cluster needs to be exported as env. It is read by k8s-cluster-checker scripts to connect to the cluster when output is generated on stdout.

How to run k8s-cluster-checker scripts

Once above pre-requisites are installed and configred, you are ready to run k8s-cluster-checker scripts as below:

1. Change dir: cd objects

2. Run scripts:

   python3 cluster.py
   

If you want a ready-made env to run k8s-cluster-checker, please build the docker image using below command:

docker build -t <image_name>:<tag_name> .

e.g.

docker build -t dguyhasnoname/k8s-cluster-checker:latest .

Running through docker image would be much easier than installing dependencies on your machine. The docker image being used is based on python:3.8-slim-buster which is a very light weight version of python in docker.

Alternatively, please check dockerhub for latest image, if you do not want to build your own image. You can download the latest image from dockerhub as the dockerhub image build is integrated with this repo and it polls this repo for update.

docker pull dguyhasnoname/k8s-cluster-checker:latest

Once your image is ready, run the docker container and export KUBECONFIG inside the container. You can get the kubeconfig inside the container by mapping dir inside the container from your local machine where your KUEBCONFIG file is stored:

docker run -it -v <location of kubeconfig on local machine>:/k8sconfig dguyhasnoname/k8s-cluster-checker:latest

Now you should be inside the container. Please export KUBECONFIG:

export KUBECONFIG=/k8sconfig/<your_kubeconfig_filename>

Now you are ready to run k8s-cluster-checker scripts:

cd /apps
python cluster.py

Flags available:

- `-n` namespace. If this is not given, it will return data for all namespaces.
- `-v` gives more details, this flag is valid for all scripts
- `-l` gives only JSON ouput on stdout. This data can be forwarded to splunk for dashboarding.

If you want the json data generated by this tool to be ingested in Splunk for dashboard, please user the CronJob.yaml to deploy k8s-cluster-checker in your cluster. Docker image used for this purpose runs the tool with -l flag which only generates JSON data on stdout.

kubectl apply  -f CronJob.yaml -n monitoring

Architecture

Sample run

Contributions/Issues

If you find any bug, please feel free to open an issue in this repo. If you want to contribute, PRs are welcome.