Promote Composition Functions to beta

[negz]

What problem are you facing?

We added alpha Composition Function support to Crossplane v1.11 in #3465. Eventually we need to either drop support for the feature, or promote it to beta. This issue tracks any work that would prevent us promoting the feature to beta.

Since alpha, we've tweaked the design per #4306. With that design in mind, I believe we'll need to do the following to enable the base Function functionality.

Tasks

Beta Give feedback

1. Add support for installing a Function to the package manager. #4418
   enhancement functions +2
   
2. Remove the FunctionIO type and update the RunFunctionRequest and RunFunctionResponse RPC types #4419
   functions +1
   
3. Move xfn out of tree - e.g. to crossplane/function-runtime-oci - and remove it from the Helm chart. #4432
   composition functions +2
   
4. Refactor the PTFComposer to accommodate long-running Functions #4339
   composition enhancement functions +3
   
5. Implement encryption and authentication for gRPC communications with the function runner.
6. Dry-run apply should happen after xfn pipeline run #3896
   bug composition functions +3
   
7. FunctionIO desired resources array is not bootstrapped with all observed resources #4288
   bug composition docs functions +4
   
8. cannot read from secret store: cannot get secret after enabling Composition Functions #4191
   bug composition functions +3
   
9. Can't use Composition Function to update XR status #3812
   bug composition functions +3
   
10. Make Patch & Transform and Functions in a Composition mutually exclusive #4558
    composition enhancement functions +3
    

We'll also need to do the following things, which we might want to move into a more developer-experience focused tracking issue:

Tasks

Beta Give feedback

1. Document how to write a Composition Function docs#579
   docs enhancement +2
   
2. Implement a dummy Function (e.g. function-nop) for testing.
3. Create a Go SDK for building Functions #4574
   composition enhancement functions +3
   
4. Implement Patch & Transform as a Composition Function #4571
   composition enhancement functions +3
   
5. Implement Go Templates as a Composition Function #4573
   composition enhancement functions +3
   
6. Create a GitHub template repository for Go SDK based Functions #4576
   composition enhancement functions +3
   

   

How could Crossplane help solve your problem?

If and when we feel ready, we should promote Composition Functions to beta. To do so we should:

• Move the feature flag to the "beta features" group, and have it be on by default.
• Update the "alpha feature" warnings on the Composition type to beta feature warnings.
• Bump the relevant protobuf messages (RunFunctionRequest etc) to v1beta1

[negz]

I feel like promoting these to beta in 1.12 is within the realm of possibility, but I'm not ready to commit to that just yet. I'd like to take at least another month to gather feedback.

[jpoley]

please add in #3899

[negz]

@jpoley @jbw976 I see that you added #3899, but I've removed it. Presuming the issue is that Bottlerocket doesn't support user namespaces I don't think that will be fixable without a complete redesign of xfn. User namespaces are fundamental to how it works. Similarly, we know xfn won't work on gVisor (which won't let us make the unshare syscall).

[bobh66]

@negz - ok to add #4116 to this list?

[jbw976]

@bobh66 i've added it to the list now for tracking - we can always remove it if it's not a good fit 😉

[negz]

Breadcrumbs to #4138. Not ready to commit to this as something we should definitely do, but I would like us to consider it. If folks are onboard with the direction we would need to make the breaking API change before promoting the feature to beta.

[negz]

@ezgidemirel and @pedjak are starting to ramp up on functions. Here's what we're planning to tackle first:

• xfn should support custom CA bundles #4041
• XFN container needs a default registry to pull images from #4116
• /tmp isn't accessible for xfn functions #4182

We picked these in part because they're impactful and in part because they're good starter issues. After these, the next priorities are:

• Support passing credentials to Composition Functions #3718
• Improve rootless container Function debugging function-runtime-oci#35

I'm also personally working on this one, which requires a bit deeper understanding of how everything works in the Composition engine:

• Dry-run apply should happen after xfn pipeline run #3896

If we make good progress on all of these with plenty of time for v1.13 we can bring some more things into scope.

[negz]

I removed the following things from this issue, since these are all specific to the xfn rootless containers implementation. Once we have a new repo for that project, we should raise these as issues there:

• Improve rootless container Function debugging function-runtime-oci#35
• Determine whether we need to do anything to support multi-platform OCI images.
• Refactor spark.Run for reduced complexity, add tests
• Cannot start function container when container user does not exist within crossplane-xfn container function-runtime-oci#30

[negz]

These shipped as beta today! 🎉