cannot run composition function with crossplane/xfn:v1.11.1

[duizabojul]

What happened?

After upgrading to v1.11.1 (using latest helm chart) to avoid flapping issues, I got errors when i am running my functions:

cannot compose resources: cannot run Composition Function pipeline:
cannot run function "xnf-api": cannot run container: rpc error: code =
Unknown desc = exit status 1: xfn: error: spark.Command.Run(): OCI
runtime error: exit status 1

I thought it was a bug in my container script but even if I forward stdin to stdout, I got the above error.

Reverting to crossplane/xfn:v1.11.0 fix this error.

How can we reproduce it?

Use composition functions with xfn container running crossplane/xfn:v1.11.1.

What environment did it happen in?

Crossplane version:
crossplane/xfn:v1.11.1

Kubernetes:
EKS 1.24

[duizabojul]

I digged a bit in this issue and i found crossplane/xfn:v1.11.0-10.g99ce14c1 is working but crossplane/xfn:v1.11.0-18.ge62deea5 is not. The difference between them is crun version: first one is running crun 1.5 and the failing one crun 1.8.

[negz]

@duizabojul Thanks for the report! Are you able to share your Composition Function? I seem to be able to run functions in general using v1.11.1:

$ cat ~/xp/fnio.yaml |docker run -i --security-opt=seccomp=unconfined crossplane/xfn:v1.11.1 run -c /tmp negz/xfn-quotable:latest -
apiVersion: apiextensions.crossplane.io/v1alpha1
config:
  apiVersion: database.example.org/v1alpha1
  kind: Config
  metadata:
    name: cloudsql
  spec:
    version: POSTGRES_9_6
kind: FunctionIO
observed:
  composite:
    connectionDetails:
    - name: uri
      value: postgresql://db.example.org:5432
    resource:
      apiVersion: database.example.org/v1alpha1
      kind: XPostgreSQLInstance
      metadata:
        name: my-db
      spec:
        compositionSelector:
          matchLabels:
            provider: gcp
        parameters:
          storageGB: 20
      status:
        conditions:
        - status: true
          type: Ready

[negz]

As a side-note, it would be ideal to pin the crun version we're using.

[duizabojul]

I just make a http call to a service inside my cluster. I tested again to use 1.11.1 and my XRs sync all failed, came back to 1.11.0 and all went green with my xfn doing its job. I tested with your command and with another image doing only local stuff and everything went fine with 1.11.1 so maybe it is a network related issue. I hope I'm not bothering you with a misconfiguration on my side but again the only difference between working and failing in my cluster is a matter of switching crossplane/xfn version 😭

FROM alpine
RUN apk add --update --no-cache curl

ENTRYPOINT [ "curl" ]
CMD [ "-X", "POST", "-H", "Content-Type: application/yaml", "--data-binary", "@-", "http://....svc.cluster.local:3000/..." ]

My function configuration is the following:

- name: xfn-api      
  type: Container
  container:
    image: public.ecr.aws/.../xfn-runner:0.0.1
    imagePullPolicy: IfNotPresent
    resources:
      limits:
        memory: 10Mi
        cpu: 30m
    timeout: 20s
    network:
      policy: Runner
  config:
    apiVersion: crossplane.airnity.net/v1alpha1
    kind: XfnConfig
    ...

btw composition functions are great stuff, it's unleashing the power of crossplane 🚀

[garreeoke]

I get this issue as well, but it doesn't work regardless of which tag of xfn I use. @negz Trying to use your example above and get the same results. Is it possibly environmental?

[Kidswiss]

I get this, too.

Interestingly using yq -o=json '.' redisFunctionIO.yaml |docker run -i --security-opt=seccomp=unconfined crossplane/xfn:v1.11.1 run -c /tmp kidswiss/myfunc:latest - it works. But as soon as I want to run it within Crossplane it fails.

Here's the code I tested with: https://github.com/Kidswiss/compositionFunctionsTest

For the composition to work fully the cluster needs a cert-manager installed.

EDIT: okay figured it out, the output needs to be YAML, not JSON.

[AtzeDeVries]

having the same issue as @duizabojul. Reverting to 1.11.0 seems to fix it for me.

[bobh66]

I'm curious if this is a regression caused by using a later version of crun in the xfn container. If you deploy the 1.11.1 Crossplane chart but override the xfn.image.tag to 1.11.0 do you still see the error?

If that fixes it we should probably pin crun to a known-good version as well is investigate why it's failing on later versions of crun.

[duizabojul]

@bobh66 yes this is my exact setting, 1.11.1 chart with xfn.image.tag to 1.11.0

[jbw976]

EDIT: okay figured it out, the output needs to be YAML, not JSON.

@Kidswiss does this mean you were able to successfully run your XFN with v1.11.1+? I'm trying to figure how many folks v1.11.1+ is working for. I have some hope since it was working for @negz with #3807 (comment), but @bobh66 theory about new version of crun could have validity 🤔

[Kidswiss]

@jbw976 yeah the python and go version in my link are both working. Although the go version has some bug and creates a reconcilation loop. But that's not a problem of crossplane itself :)

I did a lot more testing with it last week and I found that the

cannot compose resources: cannot run Composition Function pipeline:
cannot run function "xnf-api": cannot run container: rpc error: code =
Unknown desc = exit status 1: xfn: error: spark.Command.Run(): OCI
runtime error: exit status 1

Happens if the function container exits with a non 0 exit status. So I feel like the error is slightly misleading. As far as I was able to see the container actually run but failed.

[duizabojul]

@jbw976

I could be wrong but people having xfn working in 1.11.1 do not use runner network policy.

[AndrewChubatiuk]

I've tried to build docker image with all crun versions from 1.5 till 1.8, it works fine till 1.7.2 and fails starting from 1.8

[AndrewChubatiuk]

in my case crun 1.8 fails with this error
containers/crun#1131
I've fixed it in PR #3869