init draft of desc of resource limits (#1117)

* init draft of desc of resoruce limits

* revise based on feedback

* typo

* revise based on feedback

* revise based on feedback

* Update website/content/en/docs/tasks/set-resource-limits.md

Co-authored-by: Suket Sharma <suket22@gmail.com>

Co-authored-by: Cline <gcline@147dda767eb3.ant.amazon.com>
Co-authored-by: Suket Sharma <suket22@gmail.com>

website/content/en/preview/provisioner.md

@@ -47,6 +47,14 @@ spec:
     - key: "karpenter.sh/capacity-type" # If not included, the webhook for the AWS cloud provider will default to on-demand
       operator: In
       values: ["spot", "on-demand"]
+
+  # Resource limits constrain the total size of the cluster.
+  # Limits prevent Karpenter from creating new instances once the limit is exceeded.
+  limits:
+    resources:
+      cpu: 1000 
+      memory: 1000Gi
+
   # These fields vary per cloud provider, see your cloud provider specific documentation
   provider: {}
 ```
@@ -148,6 +156,21 @@ spec:
     clusterDNS: ["10.0.1.100"]
 ```
 
+## spec.limits
+
+The provisioner spec includes a limits section (`spec.limits.resources`), which constrains the maximum amount of resources that the provisioner will manage. 
+
+Presently, Karpenter supports `memory` and `cpu` limits. 
+
+CPU limits are described with a `DecimalSI` value, usually a natural integer. 
+
+Memory limits are described with a [`BinarySI` value, such as 1000Gi.](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#meaning-of-memory)
+
+Karpenter stops allocating resources once at least one resource limit is met/exceeded.
+
+Review the [resource limit task](../tasks/set-resource-limits) for more information.
+
+
 
 ## spec.provider
 

website/content/en/preview/tasks/set-resource-limits.md

@@ -0,0 +1,62 @@
+---
+title: "Set Resource Limits"
+linkTitle: "Set Resource Limits"
+weight: 10
+---
+
+Karpenter automatically provisions instances from the cloud provider. This often incurs hard costs. To control resource utilization and cluster size, use resource limits.
+
+The provisioner spec includes a limits section (`spec.limits.resources`), which constrains the maximum amount of resources that the provisioner will manage. 
+
+For example, setting "spec.limits.resources.cpu" to "1000" limits the provisioner to a total of 1000 CPU cores across all instances. This prevents unwanted excessive growth of a cluster. 
+
+At this time, Karpenter only supports:
+- CPU
+- Memory
+
+CPU limits are described with a `DecimalSI` value, usually a natural integer. 
+
+Memory limits are described with a [`BinarySI` value, such as 1000Gi.](https://github.com/kubernetes/apimachinery/blob/4427f8f31dfbac65d3a044d0168f84c51bfda440/pkg/api/resource/quantity.go#L31)
+
+You can view the current consumption of cpu and memory on your cluster by running:
+```
+kubectl get provisioner -o=jsonpath='{.items[0].status}'
+```
+
+Review the [Kubernetes core API](https://github.com/kubernetes/api/blob/37748cca582229600a3599b40e9a82a951d8bbbf/core/v1/resource.go#L23) (`k8s.io/api/core/v1`) for more information on `resources`.
+
+### Implementation
+
+Karpenter refuses to allocate new resources while at least one resource limit is *exceeded*. In other words, resource limits aren't hard limits, they only apply once Karpenter detects that a limit has been crossed.
+
+**Example:**
+
+A resource limit of 1000 CPUs is set. 996 CPU cores are currently allocated. The resource limit is not met.
+
+In response to pending pods, Karpenter calculates a new 6 core instance is needed. Karpenter *creates* the instance.
+
+1002 CPU cores are now allocated. The resource limit is in now met/exceeded. 
+
+In response to a new set of pending pods, Karpenter calculates another 6 core instance is needed. Karpenter *does not create* the instance, because the resource limit has been met.
+
+{{% alert title="Note" color="primary" %}}
+Karpenter provisioning is highly parallel. Because of this, limit checking is eventually consistent, which can result in overrun during rapid scale outs.
+{{% /alert %}}
+
+### Example Provisioner:
+
+```
+apiVersion: karpenter.sh/v1alpha5
+kind: Provisioner
+metadata:
+  name: default
+spec:
+  requirements:
+    - key: karpenter.sh/capacity-type
+      operator: In
+      values: ["spot"]
+  limits:
+    resources:
+      cpu: 1000 
+      memory: 1000Gi
+```