A draft one-pager around resource usage

[negz]

Description of your changes

This PR seeks review of an initial draft of a one-pager (four-pager?) around the concept of a "resource usage", i.e. an association between a resource claim and a Kubernetes object consuming said claim. This is an early draft, in which I expect there to be a lot of holes, but I'd like to further the conversation around this concept.

Checklist

I have:

• Run make reviewable to ensure this PR is ready for review.
• Ensured this PR contains a neat, self documenting set of commits.
• Updated any relevant documentation, examples, or release notes.
• Updated the RBAC permissions in clusterrole.yaml to include any new types.

[upbound-bot]

[upbound-bot]

[upbound-bot]

[negz]

@lukeweber @bassam @ichekrygin @prasek I'd like to expedite the feedback process on this one pager. There's some pretty fundamental changes proposed (e.g. making resource classes and managed resources cluster scoped). Would it help to setup a call to discuss?

[hasheddan]

@negz would love to be included on call if it happens

[displague]

Do managed sub-resources warrant another design document? I am interested in the possibility that resource claims may need to be resolved beyond cloud provider APIs. A MySQL User claim could be resolved through a provider API or through MySQL commands (requiring either a shared MySQL claim or direct access to the secret). Sub-resources that require parent resource claims may affect the design of how non-root MySQL credentials are bound, for example, and could affect this overall design.

This comment is a bit off-topic, responding only to one the Open Questions.

The overall draft is compelling to me, but I worry about added complexity. I would hope that your other Open Question about merging the constructs could be realized.

[negz]

Do managed sub-resources warrant another design document?

Quite possibly. They're related to this document and also likely related to any proposals around security connectivity, given a subset of hypothetical security and connectivity resources (e.g. Azure and GCP firewall rules) fall within the set of resources that could potentially be created in order to 'solve' a usage.

A MySQL User claim could be resolved through a provider API or through MySQL commands (requiring either a shared MySQL claim or direct access to the secret).

Exactly. If we begin to model users and databases it will vary from provider to provider as to whether we can do so simply via API calls (e.g. GCP), or whether we'll need to use SQL client libraries (e.g. AWS, Azure). We might consider whether such resources that are implemented using (for example) SQL client libraries belong in the same binary as resources implemented using cloud provider client libraries.

Sub-resources that require parent resource claims may affect the design of how non-root MySQL credentials are bound, for example, and could affect this overall design.

I imagine the credentials would be read from the 'parent' managed resource, with a relationship formed between the parent and child managed resources (e.g. PostgreSQLInstance and PostgreSQLUser) by the resource claim that spawns them (presuming they're spawned by one claim and not two separate ones). A variant of this pattern exists in the Azure storage container controller, which treats the container's associated storage account resource as its 'provider' (instead of a provider.azure.crossplane.io) for credential purposes.

The overall draft is compelling to me, but I worry about added complexity.

Agreed. If others find the broad direction compelling we should determine how to reach the goal iteratively. At the very least I believe we could make the foundational changes (e.g. cluster scoped managed resources, classes, and secrets) first, then introduce some resource usage (or reworked resource claim?) construct in a future change.

[displague]

We discussed offline that the existing KubernetesCluster to KubernetesApplication / KubernetesApplicationResource relationship may serve as a design model for how secrets (cluster credentials) and virtual resources (Kubernetes applications, MySQL users and databases outside of GCP) are treated throughout this project. Both represent managed resources that are not controlled by a Cloud provider managed API, but rather an independent API accessed using the parent resources' Kubernetes secret.

@negz you were able to read more into my conjecture. Would it be beneficial to log your thoughts here?

[negz]

@displague I broadly agree with the thoughts you've captured here. Nothing more to add right now.

[negz]

@bassam You mentioned you wanted to discuss this design. Should we setup some time to do so in person, or would you prefer to leave feedback on this PR?

[negz]

I synced with @bassam today to help sanity check this proposal. Some notes:

1. Cluster scoped managed resources and resource classes seem like a reasonable direction. The pattern has symmetry with PersistentVolume and StorageClass, which are cluster scoped. It helps model the Crossplane separation of concerns, in that the operator persona deals in cluster scoped resources, while the developer persona deals in namespace scoped resources (claims, usages).
2. We won't introduce cluster scoped secrets - if we really wanted this it would fit better upstream. Instead we'll use a variant of an existing pattern in which cluster scoped resources write any sensitive connection details to plain old namespaced secrets.
3. We will require, at least by convention, a resource claim to exist in order to consume a managed resource. This means managed resource secrets will be consumed only by other Crossplane controllers, not humans, and their API/UX will reflect this.
4. It may be preferable to 'embed' resource usages in some cases, for example it may be preferable for a KubernetesApplication to specify its usages inline rather than via a distinct resource. Embedding resource usages ties the usage's lifecycle to the type using it, but requires more thought. This pattern is not possible for any consumer that is not part of the Crossplane ecosystem, such as a Deployment. It may also prevent us writing controllers to act upon specific kinds of usage.

I've updated the one pager to reflect everything except for point 4, which needs more thought.

[upbound-bot]

[upbound-bot]

[negz]

We've made no progress on this PR in several months, so I am going to close it.