grpc health check endpoint

[achrefbensaad]

Feature Request
Short Description

Rely server should implement a health check grpc function and provide the grpc healmth probe utility in the container

https://kubernetes.io/blog/2018/10/01/health-checking-grpc-servers-on-kubernetes/
https://github.com/grpc-ecosystem/grpc-health-probe/

Is your feature request related to a problem? Please describe the use case.

fixes point 13 in kubearmor/KubeArmor#733

[Chinwendu20]

Hello, please is this newcomer friendly? Can I take this on?

[Chinwendu20]

Hello I can see a health check function implemented in this codebase. Is this still up?

Would you like me to implement the grpc health check probe in the deployments in kubeamor?

[TheRealSibasishBehera]

Hi @achrefbensaad I would like to work on this issue
I think here we need a side car container added to the default deployment, probing the grpc server in the relay-server container

[Chinwendu20]

Hi @TheRealSibasishBehera. I am currently working on this issue.

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    kubearmor-app: kubearmor-relay
  name: kubearmor-relay
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      kubearmor-app: kubearmor-relay
  template:
    metadata:
      annotations:
        kubearmor-policy: audited
      labels:
        kubearmor-app: kubearmor-relay
    spec:
      containers:
      - image: kubearmor/kubearmor-relay-server:latest
        name: kubearmor-relay-server
        ports:
        - containerPort: 32767
        livenessProbe:
          grpc:
               port: 32767
         initialDelaySeconds: 10
      nodeSelector:
        kubernetes.io/os: linux
      serviceAccountName: kubearmor

Here is my implementation on how we can carry out the probe @achrefbensaad please let me know if it is aligned. I would have gone ahead to modify the dockerfile of the container image but I read that from kubernetes 1.23 there is now a builtin grpc health checking capability. Also would it be better if this is implemented as readiness probe or startup probe

Link to my implementation reference:
https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#define-a-grpc-liveness-probe

[achrefbensaad]

Hi @Chinwendu20 , Thank you for your contribution.

We can go with the method you suggested, but we need to make it backward compatible.
In other words we need to still be able to generate yaml manifests without the grpc liviness probe to be able to support k8s<1.24.

[Chinwendu20]

Okay thanks that means I would just modify the container image grpc_health_probe binary in the dockerfile for kubeamor and replace this:

        livenessProbe:
          grpc:
               port: 32767

With this:

      livenessProbe:
        exec:
          command: ["/grpc_health_probe", "-addr=:32767"]

As this method would work with Kubernetes versions less than 1.24 and above it as well.

I plan on making these changes to the manifest files in the deployments folder in the kubeamor repo.

[achrefbensaad]

Hi @Chinwendu20 , we do not modify the deployments files manually, changes should be done via code under deployments/get then under deployments run make gen