Develop Component resource for: kubernetes:apiextensions.k8s.io:CustomResource

[mjeffryes]

CustomResource is a generic resource for creating a Kubernetes custom resource; see example. Today it is implemented as an 'overlay' resource and has two major limitations:

• not supported in Pulumi YAML; see Implement a blocklist and improve errors when unsupported resources are created pulumi-yaml#158
• not supported in Pulumi Java SDK

Conceptually,CustomResource represents a single Kubernetes object akin to ConfigMap or Deployment. It's not a component resource and should continue to benefit from the usual resource lifecycle (e.g. diff).

For most use-cases, a reasonable alternative to CustomResource is ConfigGroup with its objs input property, which allows for literal Kubernetes objects. That said, a component resource is an awkward substitute for a custom resource.

Note that the Pulumi Converter for Kubernetes (kube2pulumi and pulumi-converter-kubernetes) doesn't support custom resources (see Limitations).

Resolves

Beta Give feedback

1. Inconsistency in code gen for constructor params of Python overlay resources #1082
   area/sdk kind/bug language/python mro1 +4
2. CustomResource with input kind #1115
   impact/panic kind/bug +2
   

[EronWright]

I was curious as to why Pulumi YAML is unable to simply use the CustomResource as provided by the kubernetes provider without an overlay. Imagine we removed the special-case logic (here) and allowed the YAML engine to register the resource. We see the essential problem is a limitation of Pulumi schema: lack of support for open-ended types w/ extra properties.

For example, given a YAML program:

resources:
  myCustomResource:
    type: kubernetes:apiextensions.k8s.io:CustomResource
    properties:
      apiVersion: stable.example.com/v1
      kind: CronTab
      metadata:
        name: my-new-cron-object
      spec:
        cronSpec: "* * * * */5"

The YAML engine obtains schema info from the provider, and is unable to handle the spec property.

    Error: Property spec does not exist on 'kubernetes:apiextensions.k8s.io:CustomResource'
      on Pulumi.yaml line 16:
      16:       spec:
    Cannot assign '{apiVersion: string, kind: string, metadata: {name: string}, spec: {cronSpec: string, image: string}}' to 'kubernetes:apiextensions.k8s.io:CustomResource':
      Existing properties are: kind, others, apiVersion, metadata

Observe that the schema for CustomResource (overlays.go) has the standard kind, apiVersion, and metadata properties. Then there's a others property (a map type), which is how the magic happens. The overlay code packs the spec into others to convey it to the provider. See how the SDK type is defined in a way that Pulumi schema cannot represent (ref):

export interface CustomResourceArgs {
    apiVersion: pulumi.Input<string>;

    kind: pulumi.Input<string>;

    metadata?: pulumi.Input<inputs.meta.v1.ObjectMeta>;

    [othersFields: string]: pulumi.Input<any>;
}

In other words, the SDKs simulate support for open-ended schemas.