Add a design document for 'Composition Functions' #2886
Conversation
|
We're currently prototyping two approaches to actually implement this.
The second approach also involves #2958. Each approach has tradeoffs and we may pick either one or allow a choice. The first approach would likely result in
The second approach is roughly the inverse:
|
negz
force-pushed
the
cc
branch
2 times, most recently
from
dd79c11
to
fc87ef3
Compare
negz
changed the title
negz
force-pushed
the
cc
branch
5 times, most recently
from
fb3a7e2
to
68f3b68
Compare
|
This is now ready for review. I'd like to get approvals from @sergenyalcin and @turkenh - perhaps @sergenyalcin could do a first pass since he has the most context on this design. |
|
I noticed while testing #3062 that the default seccomp profile in Docker and containerd reject the I'm not too worried about this - I believe that even if |
|
@negz @st3v @turkenh folks.. I hope my comments here are not a problem. I am hoping to be able to help with this as I am quite passionate about functions. As I've mentioned in one of my comments, as a user I believe that the metacontroller approach is the way to go: https://metacontroller.github.io/metacontroller/guide/create.html, and I will be more than happy to build a PoC around that with Crossplane.. because at least in my head adding function references to compositions to run custom logic shouldn't be that complicated :) |
|
@salaboy Thanks for your comments. I don't immediately share your concerns - my feeling is that what's proposed here should make it easier for folks to build and deploy functions vs requiring them to do so as separate long running processes. Consider especially:
I'm just getting back around to working on this - happy to chat with you about it some time. |
Fixes crossplane#3001 Fixes crossplane#2959 Fixes crossplane#2958 This commit adds a design document for 'Composition Functions', which we've previously referred to as 'Custom Compositions'. See crossplane#2524 for the full context. This is the Signed-off-by: Nic Cope <nicc@rk0n.org>
Signed-off-by: Nic Cope <nicc@rk0n.org>
As well as a new `type: Chroot` alternative that could be used if compatibility becomes an issue. Signed-off-by: Nic Cope <nicc@rk0n.org>
Signed-off-by: Nic Cope <nicc@rk0n.org>
Signed-off-by: Nic Cope <nicc@rk0n.org>
Including timeouts, network isolation, and resource limits. Signed-off-by: Nic Cope <nicc@rk0n.org>
This is just a fresh take on what was already there after a few months of discussion and thought. Signed-off-by: Nic Cope <nicc@rk0n.org>
Signed-off-by: Nic Cope <nicc@rk0n.org>
Signed-off-by: Nic Cope <nicc@rk0n.org>
The design of the ResourceList type was heavily inspired by KRM functions, but the implementation proposed by the Composition Functions design was already incompatible (see "Using KRM Spec Compatible Functions" under "Alternatives Considered"). It also recently occurred to me that Crossplane may need information that isn't easily modelled as part of a simple array of Kubernetes resources - for example it may need to know how to derive an XR's connection details from its composed resources'. To this end it seemed better to break away completely from ResourceList and design a similar type that was purpose built for Composition Functions. Some improvements in this type include: * Better schema for `composite` - we can ensure exactly one is required, and easily find it without iterating through the list of resources. * We can use real (required) fields in the resources array, not special annotations, for things like the name of the array entry. * We can use fields in the resources array to return connection details (or instructions on how to derive connection details) Signed-off-by: Nic Cope <nicc@rk0n.org>
This approach is a nice compromise between webhooks and 'built-in' support for running containers. Containerized functions have less operational overhead than a webhook function - they allow the function author to simply write and push the function rather than needing to deploy, maintain, and secure it. The downside of Containerized functions is that some other system must do all of these things. Previously this document proposed that Crossplane be that system - that it would run Composition Functions itself. Given that there are many different ways to run containers, each with their own tradeoffs, breaking out the function runner into a separate implementation offers extra flexibility. Signed-off-by: Nic Cope <nicc@rk0n.org>
This could allow us to specify extra runner config (e.g. credentials) in future, if required. Signed-off-by: Nic Cope <nicc@rk0n.org>
GitHub doesn't recognise the latter. Signed-off-by: Nic Cope <nicc@rk0n.org>
|
https://gist.github.com/negz/c5a5b4f82b06882906b08af377a609a4 Drawing attention to the above gist. I used this to sketch out the |
crossplane#2886 (comment) The above comment thread contains the discussion that lead to this change. The primary motiviation for this change is that sometimes we'll want to provide functions with context (observed state) that they can't mutate, and we want to clearly distinguish that from state that they can mutate. A simple example is the status of composed resources - functions may use this status as an input, but can't mutate it. Connection details are a more complex example, in that the format used to specify desired connection details differs from the format used to surface existing, observed connection details. This approach has some additional benefits: * All functions are able to know the observed state of the world, which may differ from the accumulated desired state. * Desired state may be returned in a more succinct 'overlay' style. This may be more or less onerous than mutating observed state depending on the scenario. * Passing observed state (without accidentally mutating it) is likely to be less error prone. Signed-off-by: Nic Cope <nicc@rk0n.org>
Description of your changes
Fixes #3001
Fixes #2959
Fixes #2958
This commit adds a design document for 'Composition Functions', which
we've previously referred to as 'Custom Compositions'. See #2524 for the
full context.
I have:
make reviewableto ensure this PR is ready for review.backport release-x.ylabels to auto-backport this PR if necessary.How has this code been tested
@sergenyalcin and I have built a few proof of concepts, notably #3001
and #2959 to inform this design.