Skip to content

Commit

Permalink
馃尡 Update hetznerbaremetalhost_types.go (#1171)
Browse files Browse the repository at this point in the history
Update hetznerbaremetalhost_types.go
  • Loading branch information
Sayanta66 authored and kranurag7 committed Mar 29, 2024
1 parent 6146bc3 commit 4ff99a3
Show file tree
Hide file tree
Showing 10 changed files with 270 additions and 70 deletions.
13 changes: 10 additions & 3 deletions api/v1beta1/hetznerbaremetalhost_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,10 @@ const (

// RootDeviceHints holds the hints for specifying the storage location
// for the root filesystem for the image. Need to specify either WWN or raid
// to provision host machine successfully.
// to provision the host machine successfully. It is important to find the correct root device.
// If none are specified, the host will stop provisioning in between to wait for
// the details to be specified. HardwareDetails in the host's status can be used to find the correct device.
// Currently, you can specify one disk or a raid setup.
type RootDeviceHints struct {
// Unique storage identifier. The hint must match the actual value
// exactly.
Expand Down Expand Up @@ -172,6 +175,7 @@ type RebootAnnotationArguments struct {
// HetznerBareMetalHostSpec defines the desired state of HetznerBareMetalHost.
type HetznerBareMetalHostSpec struct {
// ServerID defines the ID of the server provided by Hetzner.
// Find it on your Hetzner robot dashboard.
ServerID int `json:"serverID"`

// Provide guidance about how to choose the device for the image
Expand All @@ -188,11 +192,14 @@ type HetznerBareMetalHostSpec struct {
// and won't be selected by any Hetzner bare metal machine.
MaintenanceMode *bool `json:"maintenanceMode,omitempty"`

// Description is a human-entered text used to help identify the host
// Description is a human-entered text used to help identify the host.
// It can be used to store some valuable information about the host.
// +optional
Description string `json:"description,omitempty"`

// Status contains all status information. DO NOT EDIT!!!
// Status contains all status information. The controller writes this status.
// As some cannot be regenerated during any reconcilement, the status
// is in the specs of the object - not the actual status. DO NOT EDIT!!!
// +optional
Status ControllerGeneratedStatus `json:"status,omitempty"`
}
Expand Down
4 changes: 0 additions & 4 deletions docs/topics/advanced-caph.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,6 @@ For more information, see:
* https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-certs/
* https://kubernetes.io/docs/reference/access-authn-authz/kubelet-tls-bootstrapping/#client-and-serving-certificates

## Rate Limits

Hetzner Cloud and Hetzner Robot both implement rate limits. As a brute-force method, we implemented some logic that prevents the controller from reconciling a specific object for some defined time period if a rate limit was hit during reconcilement of that object. We set the condition on true, that a rate limit was hit. Of course, this only affects one object so that another `HCloudMachine` still reconciles normally, even though one hits the rate limit. There is a chance that it will also hit the rate limit (which is defined per function so that it does not necessarily need to happen). In that case, the controller also stops reconciling this object for some time.

## Multi-tenancy

We support multi-tenancy. You can start multiple clusters in one Hetzner project at the same time. As the resources all have a label with the cluster name, the controller is able to handle them perfectly.
Expand Down
66 changes: 66 additions & 0 deletions docs/topics/hetzner-baremetal.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
#### Preparing Hetzner Robot

1. Create a new web service user. [Here](https://robot.your-server.de/preferences/index), you can define a password and copy your user name.
1. Generate an SSH key. You can either upload it via Hetzner Robot UI or just rely on the controller to upload a key that it does not find in the robot API. This is possible, as you have to store the public and private key together with the SSH key's name in a secret that the controller reads.

## Hetzner Dedicated / Bare Metal Server

If you want to create a cluster with bare metal servers, you will also need to set up the robot credentials in the preparation step. As described in the [reference](/docs/reference/hetzner-bare-metal-machine-template.md), you need to buy bare metal servers beforehand manually. To use bare metal servers for your deployment, you should choose one of the following flavors:

| Flavor | What it does |
| -------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
| hetzner-baremetal-control-planes-remediation | Uses bare metal servers for the control plane nodes - with custom remediation (try to reboot machines first) |
| hetzner-baremetal-control-planes | Uses bare metal servers for the control plane nodes - with normal remediation (unprovision/recreate machines) |
| hetzner-hcloud-control-planes | Uses the hcloud servers for the control plane nodes and the bare metal servers for the worker nodes |

Next, you need to create a `HetznerBareMetalHost` object for each bare metal server that you bought and specify its server ID in the specs. Refer to an example [here](/docs/reference/hetzner-bare-metal-host.md). Add the created objects to your `my-cluster.yaml` file. If you already know the WWN of the storage device you want to choose for booting, specify it in the `rootDeviceHints` of the object. If not, you can apply the workload cluster, start the provisioning without specifying the WWN, and then wait for the bare metal hosts to show an error.

After that, look at the status of `HetznerBareMetalHost` by running `kubectl describe hetznerbaremetalhost` in your management cluster. There you will find `hardwareDetails` of all of your bare metal hosts, in which you can see a list of all the relevant storage devices as well as their properties. You can copy+paste the WWN:s of your desired storage device into the `rootDeviceHints` of your `HetznerBareMetalHost` objects.

### Create a secret for Hetzner (Hcloud + Robot)

In order for the provider integration hetzner to communicate with the Hetzner API ([HCloud API](https://docs.hetzner.cloud/) + [Robot API](https://robot.your-server.de/doc/webservice/en.html#preface)), we need to create a secret with the access data. The secret must be in the same namespace as the other CRs.

```shell
export HCLOUD_TOKEN="<YOUR-TOKEN>" \
export HETZNER_ROBOT_USER="<YOUR-ROBOT-USER>" \
export HETZNER_ROBOT_PASSWORD="<YOUR-ROBOT-PASSWORD>" \
export HETZNER_SSH_PUB_PATH="<YOUR-SSH-PUBLIC-PATH>" \
export HETZNER_SSH_PRIV_PATH="<YOUR-SSH-PRIVATE-PATH>"
```

- HCLOUD_TOKEN: The project where your cluster will be placed. You have to get a token from your HCloud Project.
- HETZNER_ROBOT_USER: The User you have defined in Robot under settings/web.
- HETZNER_ROBOT_PASSWORD: The Robot Password you have set in Robot under settings/web.
- HETZNER_SSH_PUB_PATH: The Path to your generated Public SSH Key.
- HETZNER_SSH_PRIV_PATH: The Path to your generated Private SSH Key. This is needed because CAPH uses this key to provision the node in Hetzner Dedicated.

```shell
kubectl create secret generic hetzner --from-literal=hcloud=$HCLOUD_TOKEN --from-literal=robot-user=$HETZNER_ROBOT_USER --from-literal=robot-password=$HETZNER_ROBOT_PASSWORD

kubectl create secret generic robot-ssh --from-literal=sshkey-name=cluster --from-file=ssh-privatekey=$HETZNER_SSH_PRIV_PATH --from-file=ssh-publickey=$HETZNER_SSH_PUB_PATH
```

Patch the created secrets so that they get automatically moved to the target cluster later. The following command helps you do that:

```shell
kubectl patch secret hetzner -p '{"metadata":{"labels":{"clusterctl.cluster.x-k8s.io/move":""}}}'
kubectl patch secret robot-ssh -p '{"metadata":{"labels":{"clusterctl.cluster.x-k8s.io/move":""}}}'
```

The secret name and the tokens can also be customized in the cluster template.

### Deploying the Hetzner Cloud Controller Manager

> This requires a secret containing access credentials to both Hetzner Robot and HCloud.
`make install-manifests-ccm-hetzner PRIVATE_NETWORK=false`

```shell
helm repo add syself https://charts.syself.com
helm repo update syself

KUBECONFIG=$CAPH_WORKER_CLUSTER_KUBECONFIG helm upgrade --install ccm syself/ccm-hetzner --version 1.1.10 \
--namespace kube-system \
--set privateNetwork.enabled=false
```
Empty file added docs/topics/introduction.md
Empty file.
10 changes: 10 additions & 0 deletions docs/topics/managing-ssh-keys.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,15 @@
## Managing SSH keys

This section provides details about SSH keys and its importance with regards to CAPH.

### What are SSH keys?

SSH keys are a crucial component of secured network communication. They provide a secure and convenient method for authenticating to and communicating with remote servers over unsecured networks. They are used as an access credential in the SSH (Secure Shell) protocol, which is used for logging in remotely from one system to another. SSH keys come in pairs with a public and a private key and its strong encryption is used for executing remote commands and remotely managing vital system components.

### SSH keys in CAPH

In CAPH, SSH keys help in establishing secure communication remotely with Kubernetes nodes running on Hetzner cloud. They help you get complete access to the underlying Kubernetes nodes that are machines provisioned in Hetzner cloud and retrieve required information related to the system. With the help of these keys, you can SSH into the nodes in case of troubleshooting.

### In Hetzner Cloud
In pure HCloud clusters, without bare metal servers, there is no need for SSH keys. All keys that exist in HCloud API and are specified in ```HetznerCluster``` properties are included when provisioning machines. Therefore, they can be used to access those machines via SSH. Note that you have to upload those keys via Hetzner UI or API beforehand.

Expand Down
15 changes: 15 additions & 0 deletions docs/topics/node-image.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,23 @@
# Node Images

## What are node-images?

Node-images are pre-configured operating system (OS) images for setting up nodes within a Kubernetes cluster. A Kubernetes cluster consists of multiple nodes that are physical or virtual machines. To run the necessary Kubernetes components for a fully functional cluster, each node runs an operating system that hosts these components. These OS images should be compatible with Kubernetes and ensure that the node has the required environment to join and perform in the cluster. The images often comes with necessary components that are pre-installed or can be easy installable facilitating a smooth setup process for Kubernetes nodes.

## Node-images in CAPH

Node-image is necessary for using CAPH in production and as a user, there can be some specific needs as per your requirements that needs to be in the linux instance. The popular linux distributions might not contain all of those specifics. In such cases, the user need to build a node-image. These images can be uploaded to the Hetzner cloud as a snapshot and then the user can use these node-images for cluster creation.

## Creating a viable Node Image

For using cluster-API with the bootstrap provider kubeadm, we need a server with all the necessary binaries and settings for running Kubernetes.
There are several ways to achieve this. In the quick-start guide, we use `pre-kubeadm` commands in the KubeadmControlPlane and KubeadmConfigTemplate objects. These are propagated from the bootstrap provider kubeadm and the control plane provider kubeadm to the node as cloud-init commands. This way is usable universally also in other infrastructure providers.
For Hcloud, there is an alternative way of doing this using Packer. It creates a snapshot to boot from. This makes it easier to version the images, and creating new nodes using this image is faster. The same is possible for Hetzner Bare Metal, as we could use installimage and a prepared tarball, which then gets installed.

To use CAPH in production, it needs a node image. In Hetzner Cloud, it is not possible to upload your own images directly. However, a server can be created, configured, and then snapshotted.
For this, Packer could be used, which already has support for Hetzner Cloud.
In this repository, there is also an example `Packer node-image`. To use it, do the following:

```shell
export HCLOUD_TOKEN=<your-token>

Expand Down
7 changes: 0 additions & 7 deletions docs/topics/preparation.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,11 +138,4 @@ kubectl patch secret robot-ssh -p '{"metadata":{"labels":{"clusterctl.cluster.x-

The secret name and the tokens can also be customized in the cluster template.


### Creating a viable Node Image

For using cluster-API with the bootstrap provider kubeadm, we need a server with all the necessary binaries and settings for running Kubernetes.
There are several ways to achieve this. In the quick-start guide, we use `pre-kubeadm` commands in the KubeadmControlPlane and KubeadmConfigTemplate objects. These are propagated from the bootstrap provider kubeadm and the control plane provider kubeadm to the node as cloud-init commands. This way is usable universally also in other infrastructure providers.
For Hcloud, there is an alternative way of doing this using Packer. It creates a snapshot to boot from. This makes it easier to version the images, and creating new nodes using this image is faster. The same is possible for Hetzner Bare Metal, as we could use installimage and a prepared tarball, which then gets installed.

See [node-image](./node-image.md) for more information.
7 changes: 7 additions & 0 deletions docs/topics/private-network.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
You can also use different flavors, e.g., to create a cluster with the private network:

```shell
clusterctl generate cluster my-cluster --kubernetes-version v1.28.4 --control-plane-machine-count=3 --worker-machine-count=3 --flavor hcloud-network > my-cluster.yaml
```

All pre-configured flavors can be found on the [release page](https://github.com/syself/cluster-api-provider-hetzner/releases). The cluster-templates start with `cluster-template-`. The flavor name is the suffix.

0 comments on commit 4ff99a3

Please sign in to comment.