Why does Crossplane use Patches?

[isihu]

Hi everybody,

thanks for your great work. I feel like Crossplane was a missing piece in the Cloud World. I am curious about a design decision, though:
Patches seem to make compositions very bloated.

For comparison, I've created an EKS Setup including the networking and iam stuff and some installations into the cluster
once with Terraform and once with Crossplane without using libraries.
Terraform ended up with about 450 lines of code.
The crossplane composition itself was about 1100 loc long.
Now, this isn't scientific by any means, the composition could arguably be trimmed down to the very necessary parts.
But even then, it would probably still be about twice as long as the Terraform version.

I think the culprit here are the patches.

Basic Case: XR -> MR

In the very basic case (xr-field -> mr-field) there isn't much of a difference.

Terraform

resource "aws_eks_cluster" "test_cluster" {
    version = var.kubernetes_version
}

Crossplane

- name: testCluster
  base:
    apiVersion: eks.aws.crossplane.io/v1beta1
    kind: Cluster
  patches:
    - fromFieldPath: spec.kubernetesVersion
      toFieldPath: spec.forProvider.version

Terraform takes one line, while Crossplane takes two lines to set the field.

Connecting MR -> MR

When connecting two MRs, Crossplane usally allows reference by name or label, which is a common pattern in kubernetes. Either way, it takes way more lines, especially when keeping the labels dynamic, which is important when XRs are claimed multiple times (if I'm not mistaken here).

Terraform

resource "aws_eks_node_group" "test_nodes" {
    cluster_name = aws_eks_cluster.test_cluster.name
}

Crossplane

- name: testCluster
  base:
    apiVersion: eks.aws.crossplane.io/v1beta1
    kind: Cluster
  patches:
    - fromFieldPath: metadata.generateName
      toFieldPath: metadata.labels.type
      transforms:
        - type: string
          string:
            fmt: "%scluster"
            
- name: testNodeGroup
  base:
    apiVersion: eks.aws.crossplane.io/v1alpha1
    kind: NodeGroup
  patches:
    - fromFieldPath: metadata.generateName
      toFieldPath: spec.forProvider.clusterNameSelector.matchLabels.type
      transforms:
        - type: string
          string:
            fmt: "%scluster"

While Terraform still only needs one line, Crossplane needs 12 to connect the NodeGroup to the Cluster.

Question

Maybe this was already answered somewhere, but I don't quite get why Crossplane doesn't use this terraformy-templaty-way of connecting the resources like so:

- name: testCluster
  base:
    apiVersion: eks.aws.crossplane.io/v1beta1
    kind: Cluster
    metadata:
      labels:
        type: ${XR.metadata.generateName}cluster
            
- name: testNodeGroup
  base:
    apiVersion: eks.aws.crossplane.io/v1alpha1
    kind: NodeGroup
    spec:
      forProvider:
        clusterNameSelector:
          matchLabels:
            type: ${XR.metadata.generateName}cluster

It takes less lines and is more readable imo.
Propagating values to the XR could be achieved like so:

- name: testCluster
  toCompositeFields:
    - "XR.status.eks.endpoint": ${MR.status.atProvider.endpoint}
            

Base64

Even more complex functions can work this way.
Take base64-encoding for example and pretend Kubernetes Secrets didn't have the field stringData.
Currentrly, it's implemented like this:

- name: test-secret
  base:
    apiVersion: kubernetes.crossplane.io/v1alpha1
    kind: Object
    spec:
      forProvider:
        manifest:
          apiVersion: v1
          kind: Secret
  patches:
    - fromFieldPath: spec.data.foo
      toFieldPath: spec.forProvider.manifest.data.foo
      transforms:
        - type: string
          string:
            type: Convert
            convert: ToBase64

Consider now this terraformy alternative:

- name: test-secret
  base:
    apiVersion: kubernetes.crossplane.io/v1alpha1
    kind: Object
    spec:
      forProvider:
        manifest:
          apiVersion: v1
          kind: Secret
          data:
            foo: ${base64(spec.data.foo)}

Or why not:

foo: ${spec.data.foo.toBase64()}

Now everyone's tastes are different, but for me the latter is more readable.
Especially when there are 5+ patches in a resource, the patches array becomes a mess.
Of course, this would require yet another templating language for yaml to implement and learn.
But honestly, Crossplane patches also have to be grasped and aren't super intuitive, in my personal opinion.

Even using helm charts of managed resources instead of compositions is considered by some.

There is no perfect solution, it's yaml after all. But I'm curious as to why the Crossplane Team chose to use patches.

Thanks in advance:)

[negz]

Hi @crimbler2,

There's some amount of context for this decision in the design document of the upcoming Composition Functions feature. See the Background section. In particular I recommend reading Brian Grant's Declarative application management in Kubernetes, which explores some of the pitfalls of configuration DSLs.

The quick answer to "why patches" is that we wanted to model everything as schemafied Kubernetes API resources that could be represented as JSON or YAML. This allows the API server to validate the syntax of the document for us. Given we're Kubernetes based, if we instead used a DSL (similar to HCL) it would presumably need to be opaque to the API (i.e. logic inside a string). For example you suggest foo: ${spec.data.foo.toBase64()} - something would need to be aware of the ${} syntax, check for missing curly braces, provide useful debugging information, etc. We'd need to build tooling for this not only inside Crossplane itself but eventually for editors (syntax highlighting etc), linters, etc.

Rather than inventing a new DSL for Crossplane we opted to use an intentionally limited configuration object (the array of resources in a Composition) for simpler use cases. For more advanced cases we feel it's better to use a tool or general purpose programming language of your choice that already has all the tooling you need to edit, lint, test, etc. To that end we're working on the aforementioned Composition Functions feature to let you specify composition logic however you like.

[isihu]

Thank you for your response.

So, to rephrase it, Terraform had the liberty to create a language however it liked to model infrastructure,
while Crossplane also had to be a nice Kubernetes Citizen and not reinvent things Kubernetes already solved?
That makes sense.

Thank you very much:)

[negz]

That's pretty close. I'd say that we actively did not want to create another DSL to describe infrastructure. Less because we were constrained by Kubernetes, more for the reasons outlined in Brian's document.