Allow existing resources to be "imported" into Crossplane to be managed

[ichekrygin]

Overview

One of the key Crossplane functionality is support for provisioning and life-cycle management of the public cloud managed resources. Crossplane offers two modes of resource creation:

• Static: Resource is created using a dedicated resource type (CR)
• Dynamic: Resource is created using ResourceClaim type (CR)
  In either case, crossplane takes "full lifecycle" ownership of the created resource, and based on the specified Reclaim Policy, can delete managed resource upon the clean-up,

There are, however, use cases when the user would like to represent existing resources (i.e. resources provisioned outside the Crossplane) as Crossplane Resource instances. To achieve that, Crossplane must support resource import functionality.

Further Consideration

In order for Crossplane to provide resource import functionality following questions need to be addressed:

Resource Lifecycle

Will the crossplane become a "full" owner of the managed resource lifecycle.

Deletion

If crossplane imports a resource, can this resource be specified with ReclaimPolicy: Delete. Initial thinking, no, it cannot, i.e. all imported resources automatically get assigned (and hopefully at some point enforced) Reclaim Policy: Retain ("cannot delete something you didn't create")

Update

Can crossplane perform manage resource update, i.e. changing resource state/properties with the potential impact on any/all existing downstream resource consumers?

Cloud Provider compatibility

There should be a special concideration that the imported resource could be "fully" managed by the existing cloud provider (service account scopes/permissions, etc)

[negz]

crossplane/crossplane#399 (comment)

Per the comment above, I'm curious about how the ability to import an existing KubernetesCluster might affect our design for workloads.

[ichekrygin]

I don't recall any specific details of this debate. As for:

whether installing a Workload in an existing Kubernetes cluster and/or namespace is a use case we should design for.

My $0.02, yes, absolutely.

[jbw976]

@negz opened a very related issue with crossplane/crossplane#497 that we've closed as a duplicate.

As brought up in that issue, importing an existing resource such as a Kubernetes cluster could even be on-premises (bare-metal) resources.

Overall, there seem to be many existing resources that once imported into Crossplane could easily be consumed by new resources in Crossplane. Examples:

• existing Kubernetes cluster is imported into Crossplane and then new workloads can be scheduled to run on it
• existing database is imported into Crossplane and then new applications can consume it

[negz]

Cross linking this to #21, which tracks the work we're doing around reclaim policies. I could imagine this "import" work could affect that design and work. I've been increasingly thinking about this as "attaching" and "detaching" external resources from management as a Crossplane managed resource.

[negz]

In #27 (inspired by a suggestion from @muvaf) I proposed a new .spec.externalResourcePolicy that would configure, amongst other things, the ability to import existing external resources.

I like that idea @muvaf. Perhaps we could expand on it with something like:

spec:
  # externalResourcePolicy controls how the managed resource's lifecycle
  # relates to the lifecycle of its associated external resource.
  externalResourcePolicy:
    create: Attach  # Valid values are Attach, Create, and AttachOrCreate.
    update: Observe  # Valid values are Observe, or Update.
    delete: Detach  # Valid values are Detach or Delete.

This would allow:

# The default policy. Returns an error if the named external resource already
# exists at creation time.
externalResourcePolicy:
  create: Create
  update: Update
  delete: Delete

# A "read only" policy, similar to a Terraform data source. Exposes the state
# of an existing external resource via the managed resource status object.
externalResourcePolicy:
  create: Attach
  update: Observe
  delete: Detach

# The ability to attach an existing external resource to Crossplane reconciliation.
externalResourcePolicy:
  create: Attach
  update: Update
  delete: Delete

# The equivalent of Crossplane's current reclaim policy implementation
externalResourcePolicy:
  create: CreateOrAttach
  update: Update
  delete: Detach

[muvaf]

I think #45 makes it pretty easy to import a resource. Just tried with new CloudSQL controller that incorporated the late initialize pattern from https://github.com/crossplaneio/crossplane/blob/master/design/one-pager-managed-resource-api-design.md#high-fidelity

All you have to do:

• Have your resource class Spec.ForProvider struct empty so that after acquiring the resource, you don't try to make updates immediately. All fields would get late-initialized with the values received from the provider.
• Create a claim with annotation crossplane.io/external-name: <your resource name>

Then it will acquire the resource spec (into managed resource), bind it to the claim and propagate its credentials to claim's namespace. That goes for a resource (CloudSQL) that allows you to retrieve the credentials even after the creation though. So, for resources that send you the credentials only during creation there would be some manual steps getting them into managed resource's secret.

I think putting reclaimPolicy between Claim<>Managed and having an externalPolicy between Managed<>External Resource is the remaining block of work to have a nice story around importing. Though it's not blocking, it'd feel much safer to state the external policy for a resource that Crossplane didn't create in the first place.

[muvaf]

We had some discussions regarding this in #87 I didn't want to block that PR, so, writing my answer here:

In general, I believe following PVC/PV semantics is a good direction for Crossplane. However, we also need to see that there are valid cases where a cloud resource should be treated differently than how a volume is treated. For example, one can say that a volumes are generally used by 1 application, indeed in most of the cases 1 pod. So, it's safe to restrain its lifecycle management to 1 kubernetes cluster, hence volume controller. However, a cloud resource may be consumed by 1 application or N applications. Indeed, we can use it in N kubernetes clusters. It may not make sense for a volume to be consumed in that way but let's say a MySQL DB or Amazon SNS or X cloud resource should be able to be consumed by different environments.

From #87 (comment)

I'm quite strongly convinced these days that Crossplane should always be the source of truth for the managed resources it manages.

I'm negative on this for two reasons;

• User may run multiple Crossplane instances each in different Kubernetes clusters and they might want to consume the same resources from those different clusters. For example, an Azure resource group is a logical grouping mechanism of Azure and user wants to accumulate all the resources provisioned in a group of Crossplane instances in the same resource group or an SQL server could be consumed by different applications in different Kubernetes clusters using different databases in the same server. I'd like to set one Crossplane instance to manage the full lifecycle of resource X while other instance would just import and not touch anything except Observe and make it available in its cluster.
• It restricts the adoption of Crossplane into existing infrastructure. Let's say I have an application that consumes 5 different cloud services and they are all stateful, like database. I'd like to import those resources in another cluster via Crossplane and consume them there. If everything goes well I may go further, change external policy and shut down the legacy infrastructure provisioning pipelines. I am not sure we want to tell people that you're all in or out.

What I'd like to see is to have an external policy field that decides what operations that Crossplane instance is allowed to conduct on the given resource. This doesn't exist in PVC/PV model but for aforementioned reasons, our use case drifts from that model. Though I don't think it should be as fine-grained as suggested in #22 (comment)

Something even as simple as Mutable (all operations are allowed) and Immutable(only Observe is called) could be a good start.

[negz]

I'm going to close this. As @muvaf mentioned in #22 (comment) we have rudimentary support for bringing an existing external resource into Crossplane management thanks to the managed resource reconciler and the external name annotation.

This currently requires the Crossplane user to submit a managed resource that matches the existing external resource configuration wise - at least any required fields. I think we can raise a new issue if we want to track improving that process.

[kornelione]

I think #45 makes it pretty easy to import a resource. Just tried with new CloudSQL controller that incorporated the late initialize pattern from https://github.com/crossplaneio/crossplane/blob/master/design/one-pager-managed-resource-api-design.md#high-fidelity

All you have to do:

• Have your resource class Spec.ForProvider struct empty so that after acquiring the resource, you don't try to make updates immediately. All fields would get late-initialized with the values received from the provider.
• Create a claim with annotation crossplane.io/external-name: <your resource name>

Then it will acquire the resource spec (into managed resource), bind it to the claim and propagate its credentials to claim's namespace. That goes for a resource (CloudSQL) that allows you to retrieve the credentials even after the creation though. So, for resources that send you the credentials only during creation there would be some manual steps getting them into managed resource's secret.

I think putting reclaimPolicy between Claim<>Managed and having an externalPolicy between Managed<>External Resource is the remaining block of work to have a nice story around importing. Though it's not blocking, it'd feel much safer to state the external policy for a resource that Crossplane didn't create in the first place.

Is it necessary to set spec.forProvider to empty ?
If spec.forProvider is empty, I do not see I can use crossplane to scale up cpu on my azure managed database ?

[janwillies]

You have to fill in the mandatory/required fields in spec.forProvider (and the external-name annotation), then lateInitialize will query the upstream API and fill in the rest of the values for you