Make load shedding optional due to known problems

doc/kubernetes/modules/ROOT/pages/running/keycloak-deployment.adoc

@@ -39,15 +39,32 @@ Use a reverse proxy in front of Keycloak to filter out those URLs.
 The number of all Keycloak threads in the stateful set should not exceed the number of JGroup threads to avoid a JGroup thread pool exhaustion which could stall Keycloak request processing.
 It might be beneficial to limit the number of Keycloak threads even further, as too many concurrent threads will lead to throttling by Kubernetes once the requested CPU limit is reached.
 See xref:./concepts/threads.adoc[] for details.
-<4> This limits the number of queued Keycloak requests. All exceeding requests are served with an HTTP 500 error and logged.
-See xref:./concepts/threads.adoc#load-shedding[load shedding] for details.
-<5> The JVM options set additional parameters:
+<4> The JVM options set additional parameters:
 * `jgroups.thread_dumps_threshold` ensures that a log message "`thread pool is full`" appears once the JGroup thread pool is full for the first time. See xref:./concepts/threads.adoc[] for details.
 * Adjust the memory settings for the heap.
-<6> Probes will be delayed when requests queue up under load, and fail when a load is shed.
+<5> Probes will be delayed when requests queue up under load, and fail when a load is shed.
 So the best way to run Keycloak in Kubernetes would be to disable those probes, for now.
 See xref:./concepts/threads.adoc#probes[Probes] for details.
 
+== Optional: Load shedding
+
+Keycloak currently lacks a mechanism for load shedding.
+There are currently different methods in evaluation.
+
+The only way to do this in Keycloak 22 is to specify a Quarkus thread pool queue size. Unfortunately, this can cause side effects to block, for example, the health probes. See https://github.com/keycloak/keycloak/pull/23920[keycloak#23920] for the discussion.
+See below if you want to give it a try anyway.
+
+A more sophisticated solution is discussed in https://github.com/keycloak/keycloak/issues/23340[keycloak#23340] which migh arrive for Keycloak 23.
+
+.Load shedding with Quarkus thread pool size
+[source,yaml,indent=0]
+----
+            env:
+include::example$helm/keycloak.yaml[tag=keycloak-queue-size]
+----
+<1> This limits the number of queued Keycloak requests. All exceeding requests are served with an HTTP 500 error and logged.
+See xref:./concepts/threads.adoc#load-shedding[load shedding] for details.
+
 == Optional: Disable sticky sessions
 
 When running on OpenShift and the default passthrough Ingress setup as provided by the Keycloak Operator, the load balancing done by HAProxy is done using sticky sessions based on the source's IP address.

doc/kubernetes/modules/ROOT/pages/running/keycloak-with-external-infinispan.adoc

@@ -73,8 +73,8 @@ include::example$helm/keycloak-ispn.yaml[tag=keycloak-ispn]
 <1> Custom cache configuration XML file definition, which includes configuration for remote or embedded Infinispan store.
 <2> The hostname and port of the remote cache Infinispan cluster.
 <3> The credentials required, username and password, to access the remote cache Infinispan cluster.
-<4> Mounting the cache configuration Volume in Kubernetes.
-<5> `jboss.site.name` is an arbitrary Infinispan site name which Keycloak needs for its embedded Infinispan deployment when a remote store is used. This site name is related only to the embedded Infinispan and does not need to match any value from the external Infinispan deployment.
+<4> `jboss.site.name` is an arbitrary Infinispan site name which Keycloak needs for its embedded Infinispan deployment when a remote store is used. This site name is related only to the embedded Infinispan and does not need to match any value from the external Infinispan deployment.
+<5> Mounting the cache configuration Volume in Kubernetes.
 However, matching the `jboss.site.name` with the external Infinispan deployment site name helps debugging possible future issues.
 If you are using multiple sites for Keycloak in a cross-DC setup like xref::running/infinispan-crossdc-deployment.adoc[], the site name must be different in each site.
 <6> Defining the cache configuration Volume using the already created ConfigMap in Kubernetes.

provision/minikube/keycloak/templates/keycloak.yaml

@@ -155,9 +155,11 @@ spec:
         # end::keycloak-ispn[]
               - name: 'QUARKUS_THREAD_POOL_MAX_THREADS' # <3>
                 value: {{ div 200 .Values.instances | quote }}
-              - name: 'QUARKUS_THREAD_POOL_QUEUE_SIZE' # <4>
-                value: '1000'
               # end::keycloak[]
+              # tag::keycloak-queue-size[]
+              - name: 'QUARKUS_THREAD_POOL_QUEUE_SIZE' # <1>
+                value: '1000'
+              # end::keycloak-queue-size[]
               # We want to have an externally provided username and password, therefore, we override those two environment variables
               - name: KEYCLOAK_ADMIN
                 valueFrom:
@@ -202,7 +204,7 @@ spec:
                 value: "10000"
 {{ end }}
               # tag::keycloak-ispn[]
-              - name: JAVA_OPTS_APPEND # <5>
+              - name: JAVA_OPTS_APPEND # <4>
                 value: >
 {{- if not (eq (int .Values.heapInitMB) 64) }}
                   -Xms{{ .Values.heapInitMB }}m
@@ -255,19 +257,19 @@ spec:
               limits:
                 {{ if .Values.cpuLimits }}cpu: "{{ .Values.cpuLimits }}"{{end}}
                 {{ if .Values.memoryLimitsMB }}memory: "{{ .Values.memoryLimitsMB }}M"{{end}}
-            readinessProbe: # <6>
+            readinessProbe: # <5>
               exec:
                 command:
                   - 'true'
-            livenessProbe: # <6>
+            livenessProbe: # <5>
               exec:
                 command:
                   - 'true'
             # end::keycloak[]
             # tag::keycloak-ispn[]
             volumeMounts:
 {{ if .Values.infinispan.customConfig }}
-              - name: kcb-infinispan-cache-config # <4>
+              - name: kcb-infinispan-cache-config # <5>
                 mountPath: /opt/keycloak/conf/{{ base .Values.infinispan.configFile }}
                 subPath: {{ base .Values.infinispan.configFile }}
                 readOnly: true