Memory leak in container

[sebboer]

Bug describtion

I deployed kes to kubernetes using the minio operator and noticed that after some time the memory limits are reached for all replicas in the same sequence. This looks like a memory leak to me. Is anything already reported about this?

Additional context

Deployed by minio-operator
Tenant configuration (via kubectl describe tenants.minio.min.io ...):

 Kes:
    Annotations:
    Image:              minio/kes:2024-01-11T13-09-29Z
    Image Pull Policy:  IfNotPresent
    Kes Secret:
      Name:    dc-storage-kes-configuration
    Key Name:  default-minio-key
    Node Selector:
      kubernetes.io/arch:       arm64
      node.kubernetes.io/role:  agent
    Replicas:                   3
    Resources:
      Limits:
        Cpu:     300m
        Memory:  400Mi
      Requests:
        Cpu:     100m
        Memory:  100Mi
    Security Context:
      Fs Group:            1000
      Run As Group:        1000
      Run As Non Root:     true
      Run As User:         1000

Name:               dc-storage-kes
CreationTimestamp:  Tue, 09 Jan 2024 10:28:01 +0100
Selector:           v1.min.io/kes=dc-storage-kes
Labels:             <none>
Annotations:        <none>
Replicas:           3 desired | 3 total
Update Strategy:    RollingUpdate
Pods Status:        3 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
  Containers:
   kes:
    Image:      minio/kes:2024-01-11T13-09-29Z
    Port:       7373/TCP
    Host Port:  0/TCP
    Args:
      server
      --config=/tmp/kes/server-config.yaml
      --auth=off
    Limits:
      cpu:     300m
      memory:  400Mi
    Requests:
      cpu:     100m
      memory:  100Mi
    Environment:
      MINIO_KES_IDENTITY:  XXXX
    Mounts:
      /tmp/kes from dc-storage-kes (rw)
  Volumes:
   dc-storage-kes:
    Type:                Projected (a volume that contains injected data from multiple sources)
    SecretName:          dc-storage-kes-configuration
    SecretOptionalName:  <nil>
    SecretName:          dc-storage-kes-tls
    SecretOptionalName:  <nil>
Volume Claims:           <none>
Events:                  <none>

[aead]

Hi @sebboer such behavior has not been observed for 2024-01-11T13-09-29Z. Which KMS backend are you using Hashicorp Vault or something else?

[sebboer]

AWS SecretsManager / AWS-KMS