Enforcing immutability of Spec variables

[wangdrew]

I'm writing an operator to manage RabbitMQ queues and would like to know if there's a way for k8s to enforce immutability of particular spec fields. I have the following struct which represents a rabbitMQ queue and some parameters to have it bind to a rabbitMQ exchange:

type RmqQueueSpec struct {
	VHost string `json:"vhost,required"`
	Exchange string `json:"exchange,required"`
	RoutingKey string `json:"routingKey"`
	SecretConfig map[string]string `json:"secretConfig"`
}

The reason why I want immutability, specifically for VHost, is because it's a parameter that's used to namespace a queue in rabbitMQ. If it were changed for an existing deployed queue, the reconciler will fail to query rabbitMQ for the intended queue since it will be querying with a different vhost (effectively a different namespace), which could cause the creation of a new queue or an update of the wrong queue.

There are a few alternatives that I'm considering such as using the ObjectMeta.Name field to contain both the concatenated vhost and the queuename to ensure that they are immutable for a deployed queue. Or caching older spec parameters (haven't figured out exactly how to do this yet) and do a comparison in the reconciler in order to return an error. However neither of these approaches seem ideal.

Please bear with me as I'm also new to Kubernetes and appreciate any advice/feedback. Thanks!

[hasbro17]

Hi @wangdrew, that's an interesting use case.

The ideal way to do this would be via the CRD's OpenAPI Validation by marking certain fields as immutable.
The good news is this is being worked on upstream, but you'll have to wait a while for that.
kubernetes/kubernetes#65973
kubernetes/enhancements#1099

In the meanwhile you'll have to use a ValidatingAdmissionWebhook to validate your CR and check that certain fields are not changed by looking at the AdmissionRequest and comparing the old and new objects.

We're still sorting out our proposal to scaffold Validating and Mutating AdmissionWebhooks in the SDK.
#1455
Parent issue: #1217

However if you want to get a head start and check out the APIs that we'll eventually be using, you can check out this Validating webhook example in the upstream controller-runtime library that the SDK uses.
https://github.com/kubernetes-sigs/controller-runtime/blob/master/examples/builtins/validatingwebhook.go

[hasbro17]

If you have questions on the approach we can discuss them here.
But for tracking the feature request for validating admission webhooks, we should probably consolidate that to #1217

[hasbro17]

Closing in favor of #1217