Skip to content

Commit

Permalink
promote docs version (#876)
Browse files Browse the repository at this point in the history
* remove github star buttons

* promote docs version

* use full version string

* fix broken links:
  • Loading branch information
geoffcline committed Nov 29, 2021
1 parent 8d6b89d commit ebcbf33
Show file tree
Hide file tree
Showing 27 changed files with 238 additions and 235 deletions.
7 changes: 4 additions & 3 deletions website/config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -136,9 +136,10 @@ pre = "<i class='fab fa-github'></i>"
url = "https://karpenter.sh/docs/"

[[params.versions]]
version = "v0.4.3"
version = "v0.5.0"
url = "https://karpenter.sh/docs/"

[[params.versions]]
version = "Pre-release"
url = "https://karpenter.sh/pre-docs/"
version = "v0.4.3"
url = "https://karpenter.sh/v0.4.3-docs/"

File renamed without changes.
File renamed without changes.
19 changes: 8 additions & 11 deletions website/content/en/docs/concepts/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ Here are some things to know about the Karpenter provisioner:
* **Provisioner CR**: Karpenter defines a Custom Resource called a Provisioner to specify provisioning configuration.
Each provisioner manages a distinct set of nodes, but pods can be scheduled to any provisioner that supports its scheduling constraints.
A provisioner contains constraints that impact the nodes that can be provisioned and attributes of those nodes (such timers for removing nodes).
See [Provisioner](/docs/provisioner-crd/) for a description of settings and the [Provisioning](/docs/tasks/provisioner.md) task for of provisioner examples.
See [Provisioner API](/docs/provisioner-crd/) for a description of settings and the [Provisioning](../tasks/provisioning-task) task for provisioner examples.

* **Well-known labels**: The provisioner can use well-known Kubernetes labels to allow pods to request only certain instance types, architectures, operating systems, or other attributes when creating nodes.
See [Well-Known Labels, Annotations and Taints](https://kubernetes.io/docs/reference/labels-annotations-taints/) for details.
Expand All @@ -67,16 +67,13 @@ Karpenter handles all clean-up work needed to properly delete the node.
* **Empty nodes**: When the last workload pod running on a Karpenter-managed node is gone, the node is annotated with an emptiness timestamp.
Once that "node empty" time-to-live (`ttlSecondsAfterEmpty`) is reached, finalization is triggered.

For more details on how Karpenter deletes nodes, see [Deleting nodes with Karpenter](/docs/tasks/delete-nodes.md) for details.
For more details on how Karpenter deletes nodes, see [Deleting nodes with Karpenter](../tasks/deprov-nodes.md) for details.

### Upgrading nodes

A straight-forward way to upgrade nodes is to set `ttlSecondsUntilExpired`.
Nodes will be terminated after a set period of time and will be replaced with newer nodes.

For details on upgrading nodes with Karpenter, see [Upgrading nodes with Karpenter](/docs/tasks/upgrade-nodes.md) for details.


Understanding the following concepts will help you in carrying out the tasks just described.

### Constraints
Expand Down Expand Up @@ -109,7 +106,7 @@ So, for example, to include a certain instance type, you could use the Kubernete
### Kubernetes cluster autoscaler
Like Karpenter, [Kubernetes Cluster Autoscaler](https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler) is
designed to add nodes when requests come in to run pods that cannot be met by current capacity.
Cluster autoscaler is part of the Kubenetes project, with implementations by most major Kubernetes cloud providers.
Cluster autoscaler is part of the Kubernetes project, with implementations by most major Kubernetes cloud providers.
By taking a fresh look at provisioning, Karpenter offers the following improvements:

* **Designed to handle the full flexibility of the cloud**:
Expand All @@ -134,11 +131,11 @@ Karpenter's job is to efficiently assess and choose compute assets based on requ
These can include basic Kubernetes features or features that are specific to the cloud provider (such as AWS).

Layered *constraints* are applied when a pod makes requests for compute resources that cannot be met by current capacity.
A pod can specify `nodeAffinity` (to run in a particular zone or instance type) or a `topologySpreadConstraints` spread (to cause a set of pods be balanced across multiple nodes).
A pod can specify `nodeAffinity` (to run in a particular zone or instance type) or a `topologySpreadConstraints` spread (to cause a set of pods to be balanced across multiple nodes).
The pod can specify a `nodeSelector` to run only on nodes with a particular label and `resource.requests` to ensure that the node has enough available memory.

The Kubernetes scheduler tries to match those constraints with available nodes.
If the pod is unschedulable, Karpenter created compute resources that match its needs.
If the pod is unschedulable, Karpenter creates compute resources that match its needs.
When Karpenter tries to provision a node, it analyzes scheduling constraints before choosing the node to create.

As long as the requests are not outside of the provisioner's constraints,
Expand All @@ -147,12 +144,12 @@ Note that if the constraints are such that a match is not possible, the pod will

So, what constraints can you use as an application developer deploying pods that could be managed by Karpenter?

Kubernetes features that Karpenters supports for scheduling nodes include node affinity based on [persistant volumes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#node-affinity) and [nodeSelector](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector).
Kubernetes features that Karpenters supports for scheduling nodes include nodeAffinity and [nodeSelector](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector).
It also supports [PodDisruptionBudget](https://kubernetes.io/docs/tasks/run-application/configure-pdb/) and [topologySpreadConstraints](https://kubernetes.io/docs/concepts/workloads/pods/pod-topology-spread-constraints/).

From the Kubernetes [Well-Known Labels, Annotations and Taints](https://kubernetes.io/docs/reference/labels-annotations-taints/) page,
you can see a full list of Kubernetes labels, annotations and taints that determine scheduling.
Only a small set of them are implemented in Karpenter, including:
Those that are implemented in Karpenter include:

* **kubernetes.io/arch**: For example, kubernetes.io/arch=amd64
* **node.kubernetes.io/instance-type**: For example, node.kubernetes.io/instance-type=m3.medium
Expand All @@ -164,4 +161,4 @@ Kubernetes SIG scalability recommends against these features and Karpenter doesn
Instead, the Karpenter project recommends `topologySpreadConstraints` to reduce blast radius and `nodeSelectors` and `taints` to implement colocation.
{{% /alert %}}

For more on how, as a developer, you can add constraints to your pod deployment, see [Running pods](/docs/tasks/running-pods.md) for details.
For more on how, as a developer, you can add constraints to your pod deployment, see [Running pods](../tasks/running-pods.md) for details.
50 changes: 11 additions & 39 deletions website/content/en/docs/getting-started/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,6 @@
title: "Getting Started with Karpenter on AWS"
linkTitle: "Getting Started"
weight: 10
menu:
main:
weight: 10
---

Karpenter automatically provisions new nodes in response to unschedulable
Expand Down Expand Up @@ -157,10 +154,9 @@ eksctl. Thus, we don't need the helm chart to do that.
helm repo add karpenter https://charts.karpenter.sh
helm repo update
helm upgrade --install karpenter karpenter/karpenter --namespace karpenter \
--create-namespace --set serviceAccount.create=false --version 0.4.3 \
--create-namespace --set serviceAccount.create=false --version 0.5.0 \
--set controller.clusterName=${CLUSTER_NAME} \
--set controller.clusterEndpoint=$(aws eks describe-cluster --name ${CLUSTER_NAME} --query "cluster.endpoint" --output json) \
--set defaultProvisioner.create=false \
--wait # for the defaulting webhook to install before creating a Provisioner
```

Expand All @@ -169,40 +165,6 @@ helm upgrade --install karpenter karpenter/karpenter --namespace karpenter \
kubectl patch configmap config-logging -n karpenter --patch '{"data":{"loglevel.controller":"debug"}}'
```

### Create Grafana dashboards (optional)

The Karpenter repo contains multiple [importable dashboards](https://github.com/aws/karpenter/tree/main/grafana-dashboards) for an existing Grafana instance. See the Grafana documentation for [instructions](https://grafana.com/docs/grafana/latest/dashboards/export-import/#import-dashboard) to import a dashboard.

#### Deploy a temporary Prometheus and Grafana stack (optional)

The following commands will deploy a Prometheus and Grafana stack that is suitable for this guide but does not include persistent storage or other configurations that would be necessary for monitoring a production deployment of Karpenter.

```sh
helm repo add grafana-charts https://grafana.github.io/helm-charts
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

kubectl create namespace monitoring

curl -fsSL https://karpenter.sh/docs/getting-started/prometheus-values.yaml
helm install --namespace monitoring prometheus prometheus-community/prometheus --values prometheus-values.yaml

curl -fsSL https://karpenter.sh/docs/getting-started/grafana-values.yaml
helm install --namespace monitoring grafana grafana-charts/grafana --values grafana-values.yaml
```

The Grafana instance may be accessed using port forwarding.

```sh
kubectl port-forward --namespace monitoring svc/grafana 3000:80
```

The new stack has only one user, `admin`, and the password is stored in a secret. The following command will retrieve the password.

```sh
kubectl get secret --namespace monitoring grafana -o jsonpath="{.data.admin-password}" | base64 --decode
```

### Provisioner

A single Karpenter provisioner is capable of handling many different pod
Expand All @@ -220,6 +182,8 @@ This behavior can be disabled by leaving the value undefined.
Review the [provisioner CRD](/docs/provisioner-crd) for more information. For example,
`ttlSecondsUntilExpired` configures Karpenter to terminate nodes when a maximum age is reached.

Note: This provisioner will create capacity as long as the sum of all created capacity is less than the specified limit.

```bash
cat <<EOF | kubectl apply -f -
apiVersion: karpenter.sh/v1alpha5
Expand All @@ -231,6 +195,9 @@ spec:
- key: karpenter.sh/capacity-type
operator: In
values: ["spot"]
limits:
resources:
cpu: 1000
provider:
instanceProfile: KarpenterNodeInstanceProfile-${CLUSTER_NAME}
ttlSecondsAfterEmpty: 30
Expand Down Expand Up @@ -262,6 +229,7 @@ spec:
labels:
app: inflate
spec:
terminationGracePeriodSeconds: 0
containers:
- name: inflate
image: public.ecr.aws/eks-distro/kubernetes/pause:3.2
Expand Down Expand Up @@ -309,3 +277,7 @@ aws ec2 describe-launch-templates \
| xargs -I{} aws ec2 delete-launch-template --launch-template-name {}
eksctl delete cluster --name ${CLUSTER_NAME}
```

---

## Next Steps:
112 changes: 103 additions & 9 deletions website/content/en/docs/provisioner-crd.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
---
title: "Provisioner CRD"
linkTitle: "Provisioner CRD"
weight: 40
title: "Provisioner API"
linkTitle: "Provisioner API"
weight: 70
date: 2017-01-05
description: >
Provisioner API reference page
---

## Example Provisioner Resource
Expand Down Expand Up @@ -33,21 +35,113 @@ spec:
# These requirements are combined with pod.spec.affinity.nodeAffinity rules.
# Operators { In, NotIn } are supported to enable including or excluding values
requirements:
- key: "node.kubernetes.io/instance-type" # If not included, all instance types are considered
- key: "node.kubernetes.io/instance-type"
operator: In
values: ["m5.large", "m5.2xlarge"]
- key: "topology.kubernetes.io/zone" # If not included, all zones are considered
- key: "topology.kubernetes.io/zone"
operator: In
values: ["us-west-2a", "us-west-2b"]
- key: "kubernetes.io/arch" # If not included, all architectures are considered
- key: "kubernetes.io/arch"
operator: In
values: ["arm64", "amd64"]
- key: "kubernetes.io/os" # If not included, all operating systems are considered
operator: In
values: ["linux"]
- key: "karpenter.sh/capacity-type" # If not included, the webhook for the AWS cloud provider will default to on-demand
operator: In
values: ["spot", "on-demand"]
# These fields vary per cloud provider, see your cloud provider specific documentation
provider: {}
```

## spec.requirements

Kubernetes defines the following [Well-Known Labels](https://kubernetes.io/docs/reference/labels-annotations-taints/), and cloud providers (e.g., AWS) implement them. They are defined at the "spec.requirements" section of the Provisioner API.

These well known labels may be specified at the provisioner level, or in a workload definition (e.g., nodeSelector on a pod.spec). Nodes are chosen using the both the provisioner's and pod's requirements. If there is no overlap, nodes will not be launched. In other words, a pod's requirements must be within the provisioner's requirements. If a requirement is not defined for a well known label, any value available to the cloud provider may be chosen.

For example, an instance type may be specified using a nodeSelector in a pod spec. If the instance type requested is not included in the provisioner list and the provisioner has instance type requirements, Karpenter will not create a node or schedule the pod.

📝 None of these values are required.

### Instance Types

- key: `node.kubernetes.io/instance-type`

Generally, instance types should be a list and not a single value. Leaving this field undefined is recommended, as it maximizes choices for efficiently placing pods.

☁️ **AWS**

Review [AWS instance types](https://aws.amazon.com/ec2/instance-types/).

The default value includes all instance types with the exclusion of metal
(non-virtualized),
[non-HVM](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/virtualization_types.html),
and GPU instances.

View the full list of instance types with `aws ec2 describe-instance-types`.

**Example**

*Set Default with provisioner.yaml*

```yaml
spec:
requirements:
- key: node.kubernetes.io/instance-type
operator: In
values: ["m5.large", "m5.2xlarge"]
```

*Override with workload manifest (e.g., pod)*

```yaml
spec:
template:
spec:
nodeSelector:
node.kubernetes.io/instance-type: m5.large
```

### Availability Zones

- key: `topology.kubernetes.io/zone`
- value example: `us-east-1c`

☁️ **AWS**

- value list: `aws ec2 describe-availability-zones --region <region-name>`

Karpenter can be configured to create nodes in a particular zone. Note that the Availability Zone `us-east-1a` for your AWS account might not have the same location as `us-east-1a` for another AWS account.

[Learn more about Availability Zone
IDs.](https://docs.aws.amazon.com/ram/latest/userguide/working-with-az-ids.html)

### Architecture

- key: `kubernetes.io/arch`
- values
- `amd64` (default)
- `arm64`

Karpenter supports `amd64` nodes, and `arm64` nodes.


### Capacity Type

- key: `karpenter.sh/capacity-type`

☁️ **AWS**

- values
- `spot` (default)
- `on-demand`

Karpenter supports specifying capacity type, which is analogous to [EC2 purchase options](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-purchasing-options.html).


## spec.provider

This section is cloud provider specific. Reference the appropriate documentation:

- [AWS](../AWS/constraints.md)



File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -24,14 +24,14 @@ If you want to modify or add provisioners to Karpenter, do the following:

1. Review the following Provisioner documents:

* [Provisioner](../getting-started/#provisioner) in the Getting Started guide for a sample default Provisioner
* [Provisioner API](../provisioner-crd) for descriptions of Provisioner API values
* [Provisioning Configuration](../AWS/constraints) for cloud-specific settings
* [Provisioner](../../getting-started/#provisioner) in the Getting Started guide for a sample default Provisioner
* [Provisioner API](../../provisioner-crd) for descriptions of Provisioner API values
* [Provisioning Configuration](../../AWS/constraints) for cloud-specific settings

2. Apply the new or modified Provisioner to the cluster.

The following examples illustrate different aspects of Provisioners.
Refer to [Running pods](running-pods) to see how the same features are used in Pod specs to determine where pods run.
Refer to [Running pods](../running-pods) to see how the same features are used in Pod specs to determine where pods run.

## Example: Requirements

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,8 @@ Its limits are set to 256MiB of memory and 1 CPU.
Instance type selection math only uses `requests`, but `limits` may be configured to enable resource oversubscription.


See [Managing Resources for Containers](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) for details on resource types supported by Kubernetes, [Specify a memory request and a memory limit](https://kubernetes.io/docs/tasks/configure-pod-container/assign-memory-resource/#specify-a-memory-request-and-a-memory-limit) for examples of memory requests, and [Provisioning COnfiguration](../aws/constraints) for a list of supported resources.
See [Managing Resources for Containers](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) for details on resource types supported by Kubernetes, [Specify a memory request and a memory limit](https://kubernetes.io/docs/tasks/configure-pod-container/assign-memory-resource/#specify-a-memory-request-and-a-memory-limit) for examples of memory requests, and [Provisioning Configuration](../../aws/constraints) for a list of supported resources.


## Selecting nodes (`nodeSelector` and `nodeAffinity`)

Expand Down

0 comments on commit ebcbf33

Please sign in to comment.