Add docs for feature gates

control-plane-operator/hostedclusterconfigoperator/controllers/featuregate/karpenter/karpeneter.go

@@ -93,6 +93,7 @@ func (r *Reconciler) Reconcile(ctx context.Context, req reconcile.Request) (reco
 	}
 
 	// Inject userData so we keep the ignition authorization token up todate in the ec2Class.
+	// TODO(alberto): Reconcile the amiID as well.
 	{
 		labelSelector := labels.SelectorFromSet(labels.Set{"featuregate": "autoprovision"})
 		listOptions := &client.ListOptions{
@@ -146,6 +147,7 @@ func (r *Reconciler) Reconcile(ctx context.Context, req reconcile.Request) (reco
 	return reconcile.Result{
 		Requeue: false,
 		// Requeue every 1 seconds to go over EC2NodeClassList so we don't have to vendor and watch types.
+		// TODO(alberto): make this event driven.
 		RequeueAfter: 1 * time.Second,
 	}, nil
 }

docs/content/how-to/automated-machine-management/node-autoprovisioning.md

@@ -0,0 +1,80 @@
+---
+title: Node Autoprovisioning
+---
+
+# Node Autoprovisoning
+
+## Context and Considerations
+
+Node Autoprovisoning is an experimental feature in Hypershift and it is enabled behind a feature gate.
+To use it the Hypershift Operator needs to run with --feature-gates=Autoprovision=true.
+
+We implement autoprovision via Karpenter. When the feature is enabled we handle everything for the cluster consumer so a cluster admin can just create regular karpenter resources against their guest cluster at any time.
+
+```yaml
+apiVersion: karpenter.sh/v1beta1
+kind: NodePool
+metadata:
+  name: spot-example
+  annotations:
+    kubernetes.io/description: "NodePool to run spot instances"
+spec:
+  template:
+    spec:
+      requirements:
+        - key: kubernetes.io/arch
+          operator: In
+          values: ["amd64"]
+        - key: kubernetes.io/os
+          operator: In
+          values: ["linux"]
+        - key: karpenter.sh/capacity-type
+          operator: In
+          values: ["spot"]
+        - key: karpenter.k8s.aws/instance-category
+          operator: In
+          values: ["c", "m", "r"]
+        - key: karpenter.k8s.aws/instance-generation
+          operator: Gt
+          values: ["4"]
+        - key: karpenter.k8s.aws/instance-size
+          operator: In
+          values: ["xlarge",]
+      nodeClassRef:
+        apiVersion: karpenter.k8s.aws/v1beta1
+        kind: EC2NodeClass
+        name: default
+---
+apiVersion: karpenter.k8s.aws/v1beta1
+kind: EC2NodeClass
+metadata:
+  name: default
+  annotations:
+    kubernetes.io/description: "General purpose EC2NodeClass for running Amazon Linux 2 nodes"
+spec:
+  amiFamily: Custom
+  amiSelectorTerms:
+  - id: ami-00722494a4a3fa2af
+  subnetSelectorTerms:
+    - tags:
+        karpenter.sh/discovery: "your-infra-tag" # replace with your known tag
+  securityGroupSelectorTerms:
+    - tags:
+        karpenter.sh/discovery: "your-infra-tag" # replace with your known tag
+  blockDeviceMappings:
+    - deviceName: /dev/xvda
+      rootVolume: true
+      ebs:
+        volumeType: gp3
+        volumeSize: 120Gi
+        deleteOnTermination: true 
+```
+Make sure to tag your securityGroups and subnets.
+
+
+### Limitations and Caveats
+- We automatically reconcile the userData against the EC2NodeClass resources.
+- We automatically reconcile the amiID against the EC2NodeClass resources.
+- We automatically approve CSRs.
+!!! warning
+- The underlying implementation and APIs exposure to consumer for Autoprovisiong might change in the future.

docs/content/how-to/experimental-features.md

@@ -0,0 +1,22 @@
+---
+title: Experimental features
+---
+
+# Experimental features
+
+Hypershift ships with a "featuregate" package where you can specify new experimental features behind a feature gate.
+
+The Hypershift Operator has a featuregate/ reconciler where experimental features that affect the operator can be added.
+
+The Control Plane Operator has featuregate/ folder where experimental components reconciliation can be added.
+
+The Hosted Cluster Config Operator has a featuregate/ reconciler where experimental features that affect the operator can be added.
+
+Additional changes that need to interfere within the stable controllers logic can be added behind a check such as
+
+```go
+if featuregateConfig.Gates.Enabled(featuregateConfig.AutoProvision) {}
+```
+
+TODO(alberto): Elaborate experimental APIs policy.
+

docs/mkdocs.yml

@@ -62,13 +62,15 @@ nav:
   - how-to/pause-reconciliation.md
   - how-to/per-hostedcluster-dashboard.md
   - how-to/metrics-sets.md
+  - how-to/experimental-features.md
   - how-to/troubleshooting-general.md
   - 'Disaster Recovery':
     - how-to/disaster-recovery/index.md
     - how-to/disaster-recovery/backup-and-restore-oadp.md
     - how-to/disaster-recovery/etcd-recovery.md
   - 'Automated Machine Management':
     - how-to/automated-machine-management/index.md
+    - how-to/automated-machine-management/node-autoprovisioning.md
     - how-to/automated-machine-management/scale-to-zero-dataplane.md
     - how-to/automated-machine-management/nodepool-lifecycle.md
     - how-to/automated-machine-management/node-tuning.md