Autoscaler multi architecture

[mysticaltech]

Added architecture auto selection for the autoscaler. But the way it is now, it's the same architecture for all autoscaler nodepools, I do not see a way out of that constraint.

Also made other small tweaks, especially to the packer file to make it easier to edit, and also made the createkh script better.

FYI @aleksasiriski @ifeulner

[aleksasiriski]

Hmm, is it possible to have 2 autoscalers on one cluster? I'll look into this when I get a chance

[ifeulner]

if I understand this correctly kubernetes/autoscaler@6e94d1a it should be possible now to use a node group with a ca* instance and the implementation chooses the right image. So it should then work out of the box.

[mysticaltech]

@ifeulner @aleksasiriski Very interesting, if we use autoscaler node groups, then it should work. PR welcome folks!

[ifeulner]

@ifeulner @aleksasiriski Very interesting, if we use autoscaler node groups, then it should work. PR welcome folks!

No PR needed, if that is implemented. We already support multiple node groups in the autoscaler setup.

[mysticaltech]

@ifeulner Have you seen the code for this very PR, no way this is going to work for multiple autoscaler nodepools of different types. It is designed for one type, based on checking out the server_type of the first autoscaler nodepool.

[ifeulner]

@mysticaltech Maybe I am wrong, but look here: https://github.com/kubernetes/autoscaler/blob/1009797f5585d7bf778072ba59fd12eb2b8ab83c/cluster-autoscaler/cloudprovider/hetzner/hetzner_node_group.go#L390

The updated findImage function is called within the context of a nodeGroup. So it should work from my understanding. Will try later.

func createServer(n *hetznerNodeGroup) error {
	serverType, err := n.manager.cachedServerType.getServerType(n.instanceType)
	if err != nil {
		return err
	}

	image, err := findImage(n, serverType)
	if err != nil {
		return err
	}

Comment from findImage:

// findImage searches for an image ID corresponding to the supplied
// HCLOUD_IMAGE env variable. This value can either be an image ID itself (an
// int), a name (e.g. "ubuntu-20.04"), or a label selector associated with an
// image snapshot. In the latter case it will use the most recent snapshot.
// It also verifies that the returned image has a compatible architecture with
// server.

[mysticaltech]

Very interesting, we may need to delete lines. Maybe only the selector is needed indeed.

[ifeulner]

Will check, need to find out if these patches are already deployed officially...

[ifeulner]

@mysticaltech so now I see our misunderstanding. Our implementation here

terraform-hcloud-kube-hetzner/autoscaler-agents.tf

Line 3 in 0c7e9d6

first_nodepool_snapshot_id = length(var.autoscaler_nodepools) == 0 ? "" : (

is blocking now the freshly added multi-arch support by the provider. We need to refer to the image name here (which must be the same for ARM and x86), and let the autoscaler-provider choose the right image.

So there is a PR needed for our code ;)

[aleksasiriski]

Couldn't we just use the same named label selector? Like how it is currenty?

[ifeulner]

should work also according to the documentation above, yes.

Will try and provide a PR if it is working.

[ifeulner]

it works with the label-selector but the Hetzner cluster-autoscaler changes mentioned above are not yet released.
(latest version currently cluster-autoscaler:v1.26.2)