In crossplane composition cannot patch spec.forProvider.manifest.metadata.namesapce

[uluzox]

Motivation

I want to give a simple API for a complicated kubernetes resource manifest.
For better demonstration, the "complicated" k8s resource will be a Secret.

What happened?

I created the following XRD and Composition

---
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: compositeexternalsecrets.my-domain.io
spec:
  group: my-domain.io
  names:
    kind: CompositeSecret
    plural: compositesecrets
  claimNames:
    kind: MySecret
    plural: mysecrets
  versions:
    - name: v1alpha1
      served: true
      referenceable: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                dataInject:
                  type: string
---
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: mysecret
spec:
  compositeTypeRef:
    apiVersion: my-domain.io/v1alpha1
    kind: CompositeSecret
  resources:
    - base:
        apiVersion: kubernetes.crossplane.io/v1alpha1
        kind: Object
        spec:
          providerConfigRef:
            name: crossplane-provider-kubernetes-config
          forProvider:
            manifest:
              apiVersion: v1
              kind: Secret
              type: Opaque
              data:
                default: default
      patches:
        - fromFieldPath: "metadata.namespace"
          toFieldPath: "spec.forProvider.manifest.metadata.namespace"
        - fromFieldPath: "spec.dataInject"
          toFieldPath: "spec.forProvider.manifest.data[injected]"

And claim it with

---
apiVersion: my-domain.io/v1alpha1
kind: MySecret
metadata:
  name: secret
  namespace: some-namespace
spec:
  dataInject: injected

The idea is to created a k8s resource of kind MySecret with as little data as possible and by that create a more complex and importantly a different k8s resource.
In this example I would like to see a Secret to be created that looks like this:

apiVersion: v1
kind: Secret
type: Opaque
metadata:
  name: secret-hashed-12379 # generated
  namespace: some-namespace # identical to MySecret's namespace
data:
  default: default
  injected: injected

What happens instead is that the Secret does not get created. Instead I see the following error
an empty namespace may not be set when a resource name is provided.

A kubectl describe object secret-f7jbk-n52l5 returns:

Name:         secret-f7jbk-n52l5
Namespace:
Labels:       crossplane.io/claim-name=secret
              crossplane.io/claim-namespace=sms-infra
              crossplane.io/composite=secret-f7jbk
Annotations:  crossplane.io/external-name: secret-f7jbk-n52l5
API Version:  kubernetes.crossplane.io/v1alpha1
Kind:         Object
Metadata:
  ...
  Owner References:
    API Version:     my-domain.io/v1alpha1
    Controller:      true
    Kind:            CompositeSecret
    Name:            secret-f7jbk
    UID:             c48be853-eb31-4f60-9f78-13ade20dc7c8
  Resource Version:  1267707672
  UID:               c7b56c56-8578-4095-a2d1-6d6b165ff750
Spec:
  For Provider:
    Manifest:
      API Version:  v1
      Data:
        Default:      default
        Injected:     injected
      Kind:           Secret
      Type:           Opaque
  Management Policy:  Default
  Provider Config Ref:
    Name:  crossplane-provider-kubernetes-config
Status:
  At Provider:
  Conditions:
    Last Transition Time:  2022-04-19T13:51:05Z
    Message:               observe failed: cannot get object: an empty namespace may not be set when a resource name is provided
    Reason:                ReconcileError
    Status:                False
    Type:                  Synced
Events:
  Type     Reason                         Age                  From                                     Message
  ----     ------                         ----                 ----                                     -------
  Warning  CannotObserveExternalResource  92s (x11 over 7m7s)  managed/object.kubernetes.crossplane.io  cannot get object: an empty namespace may not be set when a resource name is provided

I also tried other resources instead of Secrets. Patching the namespace does not work. I am using provider-kubernetes for this because of Crossplane's Composite Resource Limitation for cluster scoped resources crossplane/crossplane#1730

What environment did it happen in?

Crossplane version: v1.6.0
provider-kubernetes version: v0.3.0

kubectl version

Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.5", GitCommit:"c285e781331a3785a7f436042c65c5641ce8a9e9", GitTreeState:"clean", BuildDate:"2022-03-16T15:51:05Z", GoVersion:"go1.17.8", Compiler:"gc", Platform:"darwin/arm64"}
Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.5", GitCommit:"aea7bbadd2fc0cd689de94a54e5b7b758869d691", GitTreeState:"clean", BuildDate:"2021-09-15T21:04:16Z", GoVersion:"go1.16.8", Compiler:"gc", Platform:"linux/amd64"}
```

[turkenh]

@uluzox for patches in composition, the source is the CompositeResource(XR) not the Claim(XRC), e.g.

      patches:
        - fromFieldPath: "metadata.namespace"
          toFieldPath: "spec.forProvider.manifest.metadata.namespace"

This instructs to patch metadata.namespace of XR to spec.forProvider.manifest.metadata.namespace of provider-kubernetes Object.

The problem here is that XRs are not namespace resources hence their metadata.namespace is just empty. Couldn't remember if there were a reference back to the claim in XR including the claim reference which could then be used as fromFieldPath, if not, you may consider getting namespace as a spec in XR and patching from there.

[bobh66]

The Composite Resource has labels with the claim name and claim namespace, so you can pull the information you want from those labels:

labels:
      crossplane.io/claim-name: myclaimname
      crossplane.io/claim-namespace: myclaimnamespace

use the format:

fromFieldPath: metadata.labels["crossplane.io/claim-name"]  

[uluzox]

Ok that works. Thank you so much!
What I do not understand though is that if metadata.labels[crossplane.io/claim-namespace] comes from the CompositeResource(XR) (in this case the Object) and not the Claim (the MySecret), how is it possible that the

spec:
  dataInject: injected

part of the Claim can successfully be used for patching the XR with

patches:
  - fromFieldPath: "spec.dataInject"
     toFieldPath: "spec.forProvider.manifest.data[injected]"

?

Clearly the spec.dataInject comes from the MySecret, right?

[bobh66]

Your CompositeResourceDefinition defines two Custom Resource Definitions (CRDs), which you have called "CompositeSecret" and "MySecret". Those two CRDs contain the exact same set of input parameters, so when you create a Claim for a MySecret, you also get a CompositeSecret with the exact same inputs, which is where all of the Composition references take place.

Your "FromCompositeFieldPath" patches retrieve information from the CompositeSecret instance for use in the Managed Resources (Object in this case), and your "ToCompositeFieldPath" patches will retrieve information from the Managed Resource and place it in the CompositeSecret instance.

So yes, the spec.dataInhect comes from the MySecret instance, but it is copied into the CompositeSecret instance and that's the one the Composition is working with.

Hopefully that makes sense?

[uluzox]

Yes, that makes total sense and I could verify this on my example.
Thank you so much @bobh66! And also thank you @turkenh