Keycloak derives memory settings from container limit (#719)

Relates to keycloak/keycloak#27078 Signed-off-by: Alexander Schwartz <aschwart@redhat.com> Co-authored-by: Kamesh Akella <kakella@redhat.com>
keycloak · Feb 19, 2024 · b209fae · b209fae
1 parent 082587a
commit b209fae
Show file tree

Hide file tree

Showing 13 changed files with 107 additions and 98 deletions.
diff --git a/.github/actions/keycloak-create-deployment/action.yml b/.github/actions/keycloak-create-deployment/action.yml
@@ -22,8 +22,6 @@ inputs:
     description: 'Keycloak pod CPU requests'
   podCpuLimit:
     description: 'Keycloak pod CPU limit'
-  heapMaxSizeMB:
-    description: 'Keycloak server maximum Java heap size (in MB)'
   otel:
     description: 'Enable OpenTelemetry'
     default: 'false'
@@ -56,7 +54,6 @@ runs:
         KC_MEMORY_LIMITS_MB: ${{ inputs.podMemoryLimit }}
         KC_CPU_REQUESTS: ${{ inputs.podCpuRequests }}
         KC_CPU_LIMITS: ${{ inputs.podCpuLimit }}
-        KC_HEAP_MAX_MB: ${{ inputs.heapMaxSizeMB }}
         KC_OTEL: ${{ inputs.otel }}
         KC_DATABASE: ${{ inputs.database }}
         KC_DATABASE_URL: ${{ inputs.databaseUrl }}

diff --git a/.github/workflows/keycloak-create-deployment.yml b/.github/workflows/keycloak-create-deployment.yml
@@ -28,9 +28,6 @@ on:
       podCpuLimit:
         description: 'Keycloak pod CPU limit'
         type: string
-      heapMaxSizeMB:
-        description: 'Keycloak server maximum Java heap size (in MB)'
-        type: string
       # Hack to workaround the max limit of 10 inputes
       additional_args:
         description: JSON of additional options
@@ -101,7 +98,6 @@ jobs:
           podMemoryLimit: ${{ inputs.podMemoryLimit }}
           podCpuRequests: ${{ inputs.podCpuRequests }}
           podCpuLimit: ${{ inputs.podCpuLimit }}
-          heapMaxSizeMB: ${{ inputs.heapMaxSizeMB }}
           database: ${{ env.DATABASE }}
           databaseUrl: ${{ env.AURORA_URL }}
           kcHostnameOverride: ${{ env.KC_HOSTNAME_OVERRIDE }}

diff --git a/.github/workflows/rosa-multi-az-cluster-create.yml b/.github/workflows/rosa-multi-az-cluster-create.yml
@@ -128,9 +128,8 @@ jobs:
           KC_CPU_REQUESTS: 6
           KC_INSTANCES: 3
           KC_DISABLE_STICKY_SESSION: true
-          KC_MEMORY_REQUESTS_MB: 4000
+          KC_MEMORY_REQUESTS_MB: 3000
           KC_MEMORY_LIMITS_MB: 4000
-          KC_HEAP_MAX_MB: 2048
           KC_DB_POOL_INITIAL_SIZE: 30
           KC_DB_POOL_MAX_SIZE: 30
           KC_DB_POOL_MIN_SIZE: 30

diff --git a/.github/workflows/rosa-scaling-benchmark.yml b/.github/workflows/rosa-scaling-benchmark.yml
@@ -95,9 +95,8 @@ env:
   KC_CPU_REQUESTS: 6
   KC_INSTANCES: 3
   KC_DISABLE_STICKY_SESSION: true
-  KC_MEMORY_REQUESTS_MB: 4000
+  KC_MEMORY_REQUESTS_MB: 3000
   KC_MEMORY_LIMITS_MB: 4000
-  KC_HEAP_MAX_MB: 2048
   KC_DB_POOL_INITIAL_SIZE: 30
   KC_DB_POOL_MAX_SIZE: 30
   KC_DB_POOL_MIN_SIZE: 30
@@ -163,7 +162,6 @@ jobs:
           podCpuRequests: ${{ env.KC_CPU_REQUESTS }}
           podMemoryRequests: ${{ env.KC_MEMORY_REQUESTS_MB }}
           podMemoryLimit: ${{ env.KC_MEMORY_LIMITS_MB }}
-          heapMaxSizeMB: ${{ env.KC_HEAP_MAX_MB }}
 
       - name: Get URLs
         uses: ./.github/actions/get-keycloak-url

diff --git a/doc/benchmark/modules/ROOT/pages/report/rosa-benchmark-key-results.adoc b/doc/benchmark/modules/ROOT/pages/report/rosa-benchmark-key-results.adoc
@@ -16,7 +16,7 @@ A full automation is pending to show repeatable results over different releases.
 
 == Setup
 
-* OpenShift 4.13.x deployed on AWS via ROSA.
+* OpenShift 4.14.x deployed on AWS via ROSA.
 * Machinepool with `m5.4xlarge` instances.
 * Keycloak 24 deployed with Operator and 3 pods.
 * Default user password hashing with PBKDF2(SHA512) 210,000 hash iterations.
@@ -37,9 +37,8 @@ Deploy OpenShift and ROSA as described in xref:kubernetes-guide::prerequisite/pr
 KC_CPU_REQUESTS=6
 KC_INSTANCES=3
 KC_DISABLE_STICKY_SESSION=true
-KC_MEMORY_REQUESTS_MB=4000
+KC_MEMORY_REQUESTS_MB=3000
 KC_MEMORY_LIMITS_MB=4000
-KC_HEAP_MAX_MB=2048
 KC_DB_POOL_INITIAL_SIZE=30
 KC_DB_POOL_MAX_SIZE=30
 KC_DB_POOL_MIN_SIZE=30
@@ -63,24 +62,25 @@ Summary:
 
 Observations:
 
-* The base memory usage for an inactive Pod is 1 GB of RAM.
-
-* Leave 1 GB extra head-room for spikes of RAM.
+* The base memory usage for an inactive Pod is 1000 MB of RAM.
 
 * For each 100,000 active user sessions, add 500 MB per Pod in a three-node cluster (tested with up to 200,000 sessions).
 +
 This assumes that each user connects to only one client.
 Memory requirements increase with the number of client sessions per user session (not tested yet).
 
+* In containers, Keycloak allocates 70% of the memory limit for heap based memory. It will also use approximately 300 MB of non-heap-based memory.
+To calculate the requested memory, use the calculation above. As limit, subtract the non-heap memory from the value above and divide the result by 0.7.
+
 * For each 30 user logins per second, 1 vCPU per Pod in a three-node cluster (tested with up to 300 per second).
 +
 Keycloak spends most of the CPU time hashing the password provided by the user.
 
-* For each 450 client credential grants per second, 1 vCPU per Pod in a three node cluster (tested with up to 2000 per second).
+* For each 450 client credential grants per second, 1 vCPU per Pod in a three-node cluster (tested with up to 2000 per second).
 +
 Most CPU time goes into creating new TLS connections, as each client runs only a single request.
 
-* For each 350 refresh token requests per second, 1 vCPU per Pod in a three node cluster (tested with up to 435 refresh token requests per second).
+* For each 350 refresh token requests per second, 1 vCPU per Pod in a three-node cluster (tested with up to 435 refresh token requests per second).
 
 * Leave 200% extra head-room for CPU usage to handle spikes in the load.
 This ensures a fast startup of the node, and sufficient capacity to handle failover tasks like, for example, re-balancing Infinispan caches, when one node fails.
@@ -104,13 +104,13 @@ Limits calculated:
 +
 (Allow for three times the CPU requested to handle peaks, startups and failover tasks, and also refresh token handling which we don't have numbers on, yet)
 
-* Memory requested: 1.25 GB
+* Memory requested: 1250 MB
 +
-(1 GB base memory plus 250 MB RAM for 50,000 active sessions)
+(1000 MB base memory plus 250 MB RAM for 50,000 active sessions)
 
-* Memory limit: 2.25 GB
+* Memory limit: 1360 GB
 +
-(adding 1 GB to the memory requested)
+(1250 MB expected memory usage minus 300 non-heap-usage, divided by 0.7)
 
 == Tests performed
 

diff --git a/doc/kubernetes/modules/ROOT/pages/customizing-deployment.adoc b/doc/kubernetes/modules/ROOT/pages/customizing-deployment.adoc
@@ -114,16 +114,6 @@ Sets memory limits in MB per Keycloak Pod.
 +
 Default value: `1024`
 
-KC_HEAP_INIT_MB::
-Sets the initial heap space size in MB per Keycloak JVM.
-+
-Default value: `64`.
-
-KC_HEAP_MAX_MB::
-Sets the maximum heap space size in MB per Keycloak JVM.
-+
-Default value: `512`.
-
 KC_METASPACE_INIT_MB::
 Sets the initial meta space size in MB per Keycloak JVM.
 +

diff --git a/provision/common/Taskfile.yaml b/provision/common/Taskfile.yaml
@@ -16,8 +16,6 @@ vars:
   KC_CPU_LIMITS: '{{default "" .KC_CPU_LIMITS}}'
   KC_MEMORY_REQUESTS_MB: '{{default "1024" .KC_MEMORY_REQUESTS_MB}}'
   KC_MEMORY_LIMITS_MB: '{{default "1024" .KC_MEMORY_LIMITS_MB}}'
-  KC_HEAP_INIT_MB: '{{default "64" .KC_HEAP_INIT_MB}}'
-  KC_HEAP_MAX_MB: '{{default "512" .KC_HEAP_MAX_MB}}'
   KC_METASPACE_INIT_MB: '{{default "96" .KC_METASPACE_INIT_MB}}'
   KC_METASPACE_MAX_MB: '{{default "256" .KC_METASPACE_MAX_MB}}'
   KC_CUSTOM_INFINISPAN_CONFIG: '{{default "true" .KC_CUSTOM_INFINISPAN_CONFIG}}'
@@ -64,8 +62,6 @@ tasks:
       - echo {{.KC_CPU_LIMITS}} > .task/var-KC_CPU_LIMITS
       - echo {{.KC_MEMORY_REQUESTS_MB}} > .task/var-KC_MEMORY_REQUESTS_MB
       - echo {{.KC_MEMORY_LIMITS_MB}} > .task/var-KC_MEMORY_LIMITS_MB
-      - echo {{.KC_HEAP_INIT_MB}} > .task/var-KC_HEAP_INIT_MB
-      - echo {{.KC_HEAP_MAX_MB}} > .task/var-KC_HEAP_MAX_MB
       - echo {{.KC_METASPACE_INIT_MB}} > .task/var-KC_METASPACE_INIT_MB
       - echo {{.KC_METASPACE_MAX_MB}} > .task/var-KC_METASPACE_MAX_MB
       - echo {{.KC_CUSTOM_INFINISPAN_CONFIG}} > .task/var-KC_CUSTOM_INFINISPAN_CONFIG
@@ -83,8 +79,6 @@ tasks:
           --argjson sticky_sessions "{{ .KC_DISABLE_STICKY_SESSION }}" \
           --arg mem_req_per_pod "{{ .KC_MEMORY_REQUESTS_MB }}" \
           --arg mem_limit_per_pod "{{ .KC_MEMORY_LIMITS_MB }}" \
-          --arg heap_init "{{ .KC_HEAP_INIT_MB }}" \
-          --arg heap_max "{{ .KC_HEAP_MAX_MB }}" \
           --arg db_pool_init "{{ .KC_DB_POOL_INITIAL_SIZE }}" \
           --arg db_pool_max "{{ .KC_DB_POOL_MAX_SIZE }}" \
           --arg db_pool_min "{{ .KC_DB_POOL_MIN_SIZE }}" \
@@ -100,10 +94,6 @@ tasks:
                              "stickySessionDisabled": ($sticky_sessions),
                              "memRequestsPerPod": ($mem_req_per_pod|tonumber),
                              "memLimitPerPod": ($mem_limit_per_pod|tonumber),
-                             "heap": {
-                               "init": ($heap_init|tonumber),
-                               "max": ($heap_max|tonumber)
-                             },
                              "dbPool": {
                                "init": ($db_pool_init|tonumber),
                                "min": ($db_pool_min|tonumber),
@@ -136,8 +126,6 @@ tasks:
       - test "{{.KC_CPU_LIMITS}}" == "$(cat .task/var-KC_CPU_LIMITS)"
       - test "{{.KC_MEMORY_REQUESTS_MB}}" == "$(cat .task/var-KC_MEMORY_REQUESTS_MB)"
       - test "{{.KC_MEMORY_LIMITS_MB}}" == "$(cat .task/var-KC_MEMORY_LIMITS_MB)"
-      - test "{{.KC_HEAP_INIT_MB}}" == "$(cat .task/var-KC_HEAP_INIT_MB)"
-      - test "{{.KC_HEAP_MAX_MB}}" == "$(cat .task/var-KC_HEAP_MAX_MB)"
       - test "{{.KC_METASPACE_INIT_MB}}" == "$(cat .task/var-KC_METASPACE_INIT_MB)"
       - test "{{.KC_METASPACE_MAX_MB}}" == "$(cat .task/var-KC_METASPACE_MAX_MB)"
       - test "{{.KC_CUSTOM_INFINISPAN_CONFIG}}" == "$(cat .task/var-KC_CUSTOM_INFINISPAN_CONFIG)"

diff --git a/provision/keycloak-tasks/Taskfile.yaml b/provision/keycloak-tasks/Taskfile.yaml
@@ -44,8 +44,6 @@ vars:
   KC_CPU_LIMITS: '{{default "" .KC_CPU_LIMITS}}'
   KC_MEMORY_REQUESTS_MB: '{{default "1024" .KC_MEMORY_REQUESTS_MB}}'
   KC_MEMORY_LIMITS_MB: '{{default "1024" .KC_MEMORY_LIMITS_MB}}'
-  KC_HEAP_INIT_MB: '{{default "64" .KC_HEAP_INIT_MB}}'
-  KC_HEAP_MAX_MB: '{{default "512" .KC_HEAP_MAX_MB}}'
   KC_METASPACE_INIT_MB: '{{default "96" .KC_METASPACE_INIT_MB}}'
   KC_METASPACE_MAX_MB: '{{default "256" .KC_METASPACE_MAX_MB}}'
   KC_CUSTOM_INFINISPAN_CONFIG: '{{default "true" .KC_CUSTOM_INFINISPAN_CONFIG}}'

diff --git a/provision/keycloak-tasks/Utils.yaml b/provision/keycloak-tasks/Utils.yaml
@@ -33,7 +33,7 @@ tasks:
     internal: true
     dir: ".task/keycloak"
     cmds:
-      - ./mvnw clean install -DskipTests -Poperator -am -pl quarkus/dist,operator -Dmaven.build.cache.enabled=true
+      - ./mvnw clean install -DskipTests -Poperator -am -pl quarkus/dist,operator
       - (git rev-parse HEAD && git diff) | sha256sum > ../status-build-keycloak-dist-source
     sources:
       - .task/subtask-{{.TASK}}.yaml
@@ -185,8 +185,6 @@ tasks:
         --set cpuLimits={{ .KC_CPU_LIMITS }}
         --set memoryRequestsMB={{ .KC_MEMORY_REQUESTS_MB }}
         --set memoryLimitsMB={{ .KC_MEMORY_LIMITS_MB }}
-        --set heapInitMB={{ .KC_HEAP_INIT_MB }}
-        --set heapMaxMB={{ .KC_HEAP_MAX_MB }}
         --set metaspaceInitMB={{ .KC_METASPACE_INIT_MB }}
         --set metaspaceMaxMB={{ .KC_METASPACE_MAX_MB }}
         --set infinispan.jgroupsTls={{ .KC_JGROUPS_TLS }}

diff --git a/provision/minikube/keycloak/templates/keycloak.yaml b/provision/minikube/keycloak/templates/keycloak.yaml
@@ -16,6 +16,13 @@ spec:
     {{- else }}
     hostname: keycloak-{{ .Values.namespace }}.{{ .Values.hostname }}
     {{- end }}
+  resources:
+    requests:
+      {{ if .Values.cpuRequests }}cpu: "{{ .Values.cpuRequests }}"{{end}}
+      {{ if .Values.memoryRequestsMB }}memory: "{{ .Values.memoryRequestsMB }}M"{{end}}
+    limits:
+      {{ if .Values.cpuLimits }}cpu: "{{ .Values.cpuLimits }}"{{end}}
+      {{ if .Values.memoryLimitsMB }}memory: "{{ .Values.memoryLimitsMB }}M"{{end}}
   db:
 {{ if or (eq .Values.database "aurora-postgres") (eq .Values.database "postgres") (eq .Values.database "postgres+infinispan") }}
     vendor: postgres
@@ -126,13 +133,12 @@ spec:
   http:
     tlsSecret: keycloak-tls-secret
   instances: {{ .Values.instances }}
+  # end::keycloak[]
   unsupported:
     podTemplate:
-      # end::keycloak[]
       metadata:
         annotations:
           checksum/config: {{ include (print $.Template.BasePath "/keycloak-providers-configmap.yaml") . | sha256sum }}-{{ include (print $.Template.BasePath "/postgres/postgres-deployment.yaml") . | sha256sum }}-{{ .Values.keycloakImage }}-{{ include (print $.Template.BasePath "/keycloak-infinispan-configmap.yaml") . | sha256sum }}-{{ .Values.otelVersion }}
-      # tag::keycloak[]
       spec:
         {{ if .Values.otel }}
         initContainers:
@@ -157,7 +163,6 @@ spec:
             imagePullPolicy: Never
 {{ end }}
             env:
-              # end::keycloak[]
               # We want to have an externally provided username and password, therefore, we override those two environment variables
               - name: KEYCLOAK_ADMIN
                 valueFrom:
@@ -171,7 +176,6 @@ spec:
                     name: keycloak-preconfigured-admin
                     key: password
                     optional: false
-                # tag::keycloak[]
 {{ if .Values.otel }}
               # Instrumentation for the HTTP/2 protocol doesn't work yet - no metrics will be available, probably also no tracing
               # As a workaround, force downgrade to HTTP/1.1
@@ -198,12 +202,6 @@ spec:
 {{ end }}
               - name: JAVA_OPTS_APPEND # <5>
                 value: >
-{{- if not (eq (int .Values.heapInitMB) 64) }}
-                  -Xms{{ .Values.heapInitMB }}m
-{{- end }}
-{{- if not (eq (int .Values.heapMaxMB) 512) }}
-                  -Xmx{{ .Values.heapMaxMB }}m
-{{- end }}
 {{- if not (eq (int .Values.metaspaceInitMB) 96) }}
                   -XX:MetaspaceSize={{ .Values.metaspaceInitMB }}m
 {{- end }}
@@ -219,7 +217,6 @@ spec:
 {{- if .Values.cryostat }}
                   -Dcom.sun.management.jmxremote.port=9091 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false
 {{- end }}
-# end::keycloak[]
             ports:
 {{ if .Values.otel }}
               - containerPort: 9464
@@ -236,14 +233,6 @@ spec:
                 protocol: TCP
                 name: jvm-debug
 {{ end }}
-# tag::keycloak[]
-            resources:
-              requests:
-                {{ if .Values.cpuRequests }}cpu: "{{ .Values.cpuRequests }}"{{end}}
-                {{ if .Values.memoryRequestsMB }}memory: "{{ .Values.memoryRequestsMB }}M"{{end}}
-              limits:
-                {{ if .Values.cpuLimits }}cpu: "{{ .Values.cpuLimits }}"{{end}}
-                {{ if .Values.memoryLimitsMB }}memory: "{{ .Values.memoryLimitsMB }}M"{{end}}
             # end::keycloak[]
             # readinessProbe:
             #   exec: