Improve CC documentation about CPU capacity and utilization

documentation/modules/cruise-control/proc-generating-optimization-proposals.adoc

@@ -125,7 +125,23 @@ Status:
 ----
 +
 The properties in the `Optimization Result` section describe the pending cluster rebalance operation. 
-For descriptions of each property, see xref:contents-optimization-proposals[Contents of optimization proposals]. 
+For descriptions of each property, see xref:contents-optimization-proposals[Contents of optimization proposals].
+
+.Insufficient capacity for cpu
+
+If the Kafka cluster is overloaded in terms of CPU utilization and needs some additional brokers, you may see an error like the following in the `KafkaRebalance` status. This may happen because, while proposals exclude the `excludedTopics`, the CPU utilization still considers replicas of the `excludedTopics`.
+
+[source,shell,subs="+quotes"]
+com.linkedin.kafka.cruisecontrol.exception.OptimizationFailureException:
+        [CpuCapacityGoal] Insufficient capacity for cpu (Utilization 615.21,
+        Allowed Capacity 420.00, Threshold: 0.70). Add at least 3 brokers with
+        the same cpu capacity (100.00) as broker-0. Add at least 3 brokers with
+        the same cpu capacity (100.00) as broker-0.
+
+[NOTE]
+====
+These numbers do not make sense if you look at `Kafka.spec.kafka.resources` because Strimzi generates a CPU percentage Cruise Control configuration, instead of a CPU core configuration. It is like having a single "virtual" CPU which has the cycles of 1 or more CPUs (the number configured in `Kafka.spec.kafka.resources.limits.cpu`). This is counterintuitive, but it has no effect on the rebalance behavior since the ratio between the utilization and capacity remains the same.
+====
 
 .What to do next
 

documentation/modules/cruise-control/ref-cruise-control-configuration.adoc

@@ -94,6 +94,11 @@ Capacity limits can be set for the following broker resources, using the standar
 
 Because Strimzi Kafka brokers are homogeneous, Cruise Control applies the same capacity limits to every broker it is monitoring.
 
+[NOTE]
+====
+Cruise Control assumes brokers resources to be homogeneous. Therefore, it is recommended to set both `Kafka.spec.kafka.resources.limits.cpu` and `Kafka.spec.kafka.resources.limits.memory`, so that Kubernetes will ensure broker pods are limited to the same capacity. This is especially important when using nodes with heterogeneous resources (e.g. CPU cores).
+====
+
 .An example Cruise Control brokerCapacity configuration using bibyte units
 [source,yaml,subs="attributes+"]
 ----
@@ -103,6 +108,12 @@ metadata:
   name: my-cluster
 spec:
   # ...
+  kafka:
+    # ...
+    resources:
+      limits:
+        cpu: 1000m
+        memory: 1Gi
   cruiseControl:
     # ...
     brokerCapacity:

packaging/examples/cruise-control/kafka-cruise-control.yaml

@@ -23,6 +23,10 @@ spec:
       min.insync.replicas: 2
       log.message.format.version: "3.0"
       inter.broker.protocol.version: "3.0"
+    resources:
+      limits:
+        cpu: 1000m
+        memory: 1Gi
     storage:
       type: ephemeral
   zookeeper: