[yaml/v2] Support for resource ordering (implicit and explicit) (#2894)

<!--Thanks for your contribution. See [CONTRIBUTING](CONTRIBUTING.md)
    for Pulumi's contribution guidelines.

    Help us merge your changes more quickly by adding more details such
    as labels, milestones, and reviewers.-->

### Proposed changes

<!--Give us a brief description of what you've done and what it solves.
-->

This PR adds support for resource ordering within a `ConfigGroup` or
`ConfigFile`. Two approaches are supported (and work in combination):
1. implicit dependencies: the provider uses heuristics to install CRDs
and namespaces first.
2. explicit dependencies: the provider understands the
`config.kubernetes.io/depends-on` annotation to explicitly declare a
dependency on a given resource.

The implementation is based on
[kubernetes-sigs/cli-utils](https://github.com/kubernetes-sigs/cli-utils)
and its support for resource ordering
([documentation](https://github.com/kubernetes-sigs/cli-utils?tab=readme-ov-file#resource-ordering)).

To be clear, ordering _across_ `ConfigGroup` resources is supported
already, simply using the `dependsOn` option. This PR adds a more
granular ordering _within_ the group.

### Testing
New test cases were added to verify the new behavior. Many tests rely on
a common manifest file, and it is updated to include a CRD and a
namespace.

Manual testing was performed with the following well-known manifests:
1. knative-serving-core
([ref](https://github.com/knative/serving/releases/download/knative-v1.13.1/serving-core.yaml))
2. cert-manager
([ref](https://github.com/cert-manager/cert-manager/releases/download/v1.14.4/cert-manager.yaml))

### Example
Here's an example that installs cert-manager then provisions a
certificate issuer and a TLS certificate. An explicit dependency is
drawn between the `Certificate` and `Issuer`.

```yaml
name: issue-2881-cert-manager
runtime: yaml
description: Installs cert-manager.  See https://cert-manager.io/docs/installation/kubectl/ for details.
variables: {}
resources:
  install:
    type: kubernetes:yaml/v2:ConfigGroup
    properties:
      files:
      - https://github.com/cert-manager/cert-manager/releases/download/v1.14.4/cert-manager.yaml
  test:
    type: kubernetes:yaml/v2:ConfigGroup
    options:
      dependsOn:
      - ${install}
    properties:
      yaml: |
        apiVersion: v1
        kind: Namespace
        metadata:
          name: cert-manager-test
        ---
        apiVersion: cert-manager.io/v1
        kind: Issuer
        metadata:
          name: test-selfsigned
          namespace: cert-manager-test
        spec:
          selfSigned: {}
        ---
        apiVersion: cert-manager.io/v1
        kind: Certificate
        metadata:
          name: selfsigned-cert
          namespace: cert-manager-test
          annotations:
            config.kubernetes.io/depends-on: cert-manager.io/namespaces/cert-manager-test/Issuer/test-selfsigned
        spec:
          dnsNames:
            - example.com
          secretName: selfsigned-cert-tls
          issuerRef:
            name: test-selfsigned
```

Within the stack state, one sees dependencies, e.g. on the `Certificate`
resource.
```json
{
    "urn": "urn:pulumi:dev::issue-2881-cert-manager::kubernetes:yaml/v2:ConfigGroup$kubernetes:cert-manager.io/v1:Certificate::test-cert-manager-test/selfsigned-cert",
    "id": "cert-manager-test/selfsigned-cert",
    "type": "kubernetes:cert-manager.io/v1:Certificate",
    "dependencies": [
        "urn:pulumi:dev::issue-2881-cert-manager::kubernetes:yaml/v2:ConfigGroup$kubernetes:cert-manager.io/v1:Issuer::test-cert-manager-test/test-selfsigned",
        "urn:pulumi:dev::issue-2881-cert-manager::kubernetes:yaml/v2:ConfigGroup$kubernetes:core/v1:Namespace::test-cert-manager-test"
    ]
}
```

### Related issues (optional)

<!--Refer to related PRs or issues: #1234, or 'Fixes #1234' or 'Closes
#1234'.
Or link to full URLs to issues or pull requests in other GitHub
repositories. -->

Closes #2881

CHANGELOG.md

@@ -3,6 +3,7 @@
 - ConfigGroup V2 (https://github.com/pulumi/pulumi-kubernetes/pull/2844)
 - ConfigFile V2 (https://github.com/pulumi/pulumi-kubernetes/pull/2862)
 - Bugfix for ambiguous kinds (https://github.com/pulumi/pulumi-kubernetes/pull/2889)
+- [yaml/v2] Support for resource ordering (https://github.com/pulumi/pulumi-kubernetes/pull/2894)
 
 ### New Features
 

provider/pkg/gen/examples/overlays/configFileV2.md

@@ -1,5 +1,39 @@
 ConfigFile creates a set of Kubernetes resources from a Kubernetes YAML file.
 
+## Dependency ordering
+Sometimes resources must be applied in a specific order. For example, a namespace resource must be
+created before any namespaced resources, or a Custom Resource Definition (CRD) must be pre-installed.
+
+Pulumi uses heuristics to determine which order to apply and delete objects within the ConfigFile.  Pulumi also
+waits for each object to be fully reconciled, unless `skipAwait` is enabled.
+
+### Explicit Dependency Ordering
+Pulumi supports the `config.kubernetes.io/depends-on` annotation to declare an explicit dependency on a given resource.
+The annotation accepts a list of resource references, delimited by commas. 
+
+Note that references to resources outside the ConfigFile aren't supported.
+
+**Resource reference**
+
+A resource reference is a string that uniquely identifies a resource.
+
+It consists of the group, kind, name, and optionally the namespace, delimited by forward slashes.
+
+| Resource Scope   | Format                                         |
+| :--------------- | :--------------------------------------------- |
+| namespace-scoped | `<group>/namespaces/<namespace>/<kind>/<name>` |
+| cluster-scoped   | `<group>/<kind>/<name>`                        |
+
+For resources in the “core” group, the empty string is used instead (for example: `/namespaces/test/Pod/pod-a`).
+
+### Ordering across ConfigFiles
+The `dependsOn` resource option creates a list of explicit dependencies between Pulumi resources.
+Use it on another resource to make it dependent on the ConfigFile and to wait for the resources within
+the group to be deployed.
+
+A best practice is to deploy each application using its own ConfigFile, especially when that application
+installs custom resource definitions.
+
 {{% examples %}}
 ## Example Usage
 {{% example %}}

provider/pkg/gen/examples/overlays/configGroupV2.md

@@ -6,6 +6,40 @@ may be supplied using any of the following methods:
 3. Using a literal string containing YAML, or a list of such strings:
 4. Any combination of files, patterns, or YAML strings:
 
+## Dependency ordering
+Sometimes resources must be applied in a specific order. For example, a namespace resource must be
+created before any namespaced resources, or a Custom Resource Definition (CRD) must be pre-installed.
+
+Pulumi uses heuristics to determine which order to apply and delete objects within the ConfigGroup.  Pulumi also
+waits for each object to be fully reconciled, unless `skipAwait` is enabled.
+
+### Explicit Dependency Ordering
+Pulumi supports the `config.kubernetes.io/depends-on` annotation to declare an explicit dependency on a given resource.
+The annotation accepts a list of resource references, delimited by commas. 
+
+Note that references to resources outside the ConfigGroup aren't supported.
+
+**Resource reference**
+
+A resource reference is a string that uniquely identifies a resource.
+
+It consists of the group, kind, name, and optionally the namespace, delimited by forward slashes.
+
+| Resource Scope   | Format                                         |
+| :--------------- | :--------------------------------------------- |
+| namespace-scoped | `<group>/namespaces/<namespace>/<kind>/<name>` |
+| cluster-scoped   | `<group>/<kind>/<name>`                        |
+
+For resources in the “core” group, the empty string is used instead (for example: `/namespaces/test/Pod/pod-a`).
+
+### Ordering across ConfigGroups
+The `dependsOn` resource option creates a list of explicit dependencies between Pulumi resources.
+Use it on another resource to make it dependent on the ConfigGroup and to wait for the resources within
+the group to be deployed.
+
+A best practice is to deploy each application using its own ConfigGroup, especially when that application
+installs custom resource definitions.
+
 {{% examples %}}
 ## Example Usage
 {{% example %}}

provider/pkg/provider/yaml/v2/configfile_test.go

@@ -88,7 +88,9 @@ var _ = Describe("ConfigFile.Construct", func() {
 				Expect(err).ShouldNot(HaveOccurred())
 				outputs := unmarshalProperties(GinkgoTB(), resp.State)
 				Expect(outputs).To(MatchProps(IgnoreExtras, Props{
-					"resources": MatchArrayValue(HaveExactElements(
+					"resources": MatchArrayValue(ConsistOf(
+						MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigFile$kubernetes:core/v1:Namespace::test-my-namespace", "test-my-namespace"),
+						MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigFile$kubernetes:apiextensions.k8s.io/v1:CustomResourceDefinition::test-crontabs.stable.example.com", "test-crontabs.stable.example.com"),
 						MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigFile$kubernetes:core/v1:ConfigMap::test-my-map", "test-my-map"),
 						MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigFile$kubernetes:stable.example.com/v1:CronTab::test-my-new-cron-object", "test-my-new-cron-object"),
 					)),
@@ -104,7 +106,9 @@ var _ = Describe("ConfigFile.Construct", func() {
 					Expect(err).ShouldNot(HaveOccurred())
 					outputs := unmarshalProperties(GinkgoTB(), resp.State)
 					Expect(outputs).To(MatchProps(IgnoreExtras, Props{
-						"resources": MatchArrayValue(HaveExactElements(
+						"resources": MatchArrayValue(ConsistOf(
+							MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigFile$kubernetes:core/v1:Namespace::prefixed-my-namespace", "prefixed-my-namespace"),
+							MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigFile$kubernetes:apiextensions.k8s.io/v1:CustomResourceDefinition::prefixed-crontabs.stable.example.com", "prefixed-crontabs.stable.example.com"),
 							MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigFile$kubernetes:core/v1:ConfigMap::prefixed-my-map", "prefixed-my-map"),
 							MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigFile$kubernetes:stable.example.com/v1:CronTab::prefixed-my-new-cron-object", "prefixed-my-new-cron-object"),
 						)),
@@ -121,7 +125,9 @@ var _ = Describe("ConfigFile.Construct", func() {
 					Expect(err).ShouldNot(HaveOccurred())
 					outputs := unmarshalProperties(GinkgoTB(), resp.State)
 					Expect(outputs).To(MatchProps(IgnoreExtras, Props{
-						"resources": MatchArrayValue(HaveExactElements(
+						"resources": MatchArrayValue(ConsistOf(
+							MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigFile$kubernetes:core/v1:Namespace::my-namespace", "my-namespace"),
+							MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigFile$kubernetes:apiextensions.k8s.io/v1:CustomResourceDefinition::crontabs.stable.example.com", "crontabs.stable.example.com"),
 							MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigFile$kubernetes:core/v1:ConfigMap::my-map", "my-map"),
 							MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigFile$kubernetes:stable.example.com/v1:CronTab::my-new-cron-object", "my-new-cron-object"),
 						)),

provider/pkg/provider/yaml/v2/configgroup.go

@@ -102,7 +102,11 @@ func (k *ConfigGroupProvider) Construct(ctx *pulumi.Context, typ, name string, i
 			return pulumi.ArrayOutput{}, err
 		}
 		for _, obj := range objects {
-			objs = append(objs, unstructured.Unstructured{Object: obj})
+			expanded, err := Expand([]unstructured.Unstructured{{Object: obj}})
+			if err != nil {
+				return pulumi.ArrayOutput{}, err
+			}
+			objs = append(objs, expanded...)
 		}
 
 		// Register the objects as Pulumi resources.

provider/pkg/provider/yaml/v2/configgroup_test.go

@@ -78,7 +78,9 @@ var _ = Describe("Construct", func() {
 			Expect(err).ShouldNot(HaveOccurred())
 			outputs := unmarshalProperties(GinkgoTB(), resp.State)
 			Expect(outputs).To(MatchProps(IgnoreExtras, Props{
-				"resources": MatchArrayValue(HaveExactElements(
+				"resources": MatchArrayValue(ConsistOf(
+					MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigGroup$kubernetes:core/v1:Namespace::test-my-namespace", "test-my-namespace"),
+					MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigGroup$kubernetes:apiextensions.k8s.io/v1:CustomResourceDefinition::test-crontabs.stable.example.com", "test-crontabs.stable.example.com"),
 					MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigGroup$kubernetes:core/v1:ConfigMap::test-my-map", "test-my-map"),
 					MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigGroup$kubernetes:stable.example.com/v1:CronTab::test-my-new-cron-object", "test-my-new-cron-object"),
 				)),
@@ -94,7 +96,9 @@ var _ = Describe("Construct", func() {
 				Expect(err).ShouldNot(HaveOccurred())
 				outputs := unmarshalProperties(GinkgoTB(), resp.State)
 				Expect(outputs).To(MatchProps(IgnoreExtras, Props{
-					"resources": MatchArrayValue(HaveExactElements(
+					"resources": MatchArrayValue(ConsistOf(
+						MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigGroup$kubernetes:core/v1:Namespace::prefixed-my-namespace", "prefixed-my-namespace"),
+						MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigGroup$kubernetes:apiextensions.k8s.io/v1:CustomResourceDefinition::prefixed-crontabs.stable.example.com", "prefixed-crontabs.stable.example.com"),
 						MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigGroup$kubernetes:core/v1:ConfigMap::prefixed-my-map", "prefixed-my-map"),
 						MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigGroup$kubernetes:stable.example.com/v1:CronTab::prefixed-my-new-cron-object", "prefixed-my-new-cron-object"),
 					)),
@@ -111,7 +115,9 @@ var _ = Describe("Construct", func() {
 				Expect(err).ShouldNot(HaveOccurred())
 				outputs := unmarshalProperties(GinkgoTB(), resp.State)
 				Expect(outputs).To(MatchProps(IgnoreExtras, Props{
-					"resources": MatchArrayValue(HaveExactElements(
+					"resources": MatchArrayValue(ConsistOf(
+						MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigGroup$kubernetes:core/v1:Namespace::my-namespace", "my-namespace"),
+						MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigGroup$kubernetes:apiextensions.k8s.io/v1:CustomResourceDefinition::crontabs.stable.example.com", "crontabs.stable.example.com"),
 						MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigGroup$kubernetes:core/v1:ConfigMap::my-map", "my-map"),
 						MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigGroup$kubernetes:stable.example.com/v1:CronTab::my-new-cron-object", "my-new-cron-object"),
 					)),
@@ -130,19 +136,42 @@ var _ = Describe("Construct", func() {
 	})
 
 	Describe("objs", func() {
-		Context("when the input is a valid object", func() {
+		decodeObjects := func(manifest string) []resource.PropertyValue {
+			// decode the manifest to Unstructured objects, then convert to input properties
+			resources, err := yamlDecode(manifest, nil)
+			Expect(err).ShouldNot(HaveOccurred())
+			var objs []resource.PropertyValue
+			for _, res := range resources {
+				objs = append(objs, resource.NewPropertyValue(res.Object))
+			}
+			return objs
+		}
+
+		Context("when the input is a valid object literal", func() {
 			BeforeEach(func() {
-				// decode the manifest to Unstructured objects, then convert to input properties
-				resources, err := yamlDecode(manifest, nil)
-				Expect(err).ShouldNot(HaveOccurred())
-				var objs []resource.PropertyValue
-				for _, res := range resources {
-					objs = append(objs, resource.NewPropertyValue(res.Object))
-				}
-				inputs["objs"] = resource.NewArrayProperty(objs)
+				inputs["objs"] = resource.NewArrayProperty(decodeObjects(manifest))
 			})
 			commonAssertions()
 		})
+
+		Context("when the object is a list", func() {
+			BeforeEach(func() {
+				inputs["objs"] = resource.NewArrayProperty(decodeObjects(list))
+			})
+
+			It("should expand the list", func(ctx context.Context) {
+				resp, err := pulumiprovider.Construct(ctx, req, tc.EngineConn(), k.Construct)
+				Expect(err).ShouldNot(HaveOccurred())
+				outputs := unmarshalProperties(GinkgoTB(), resp.State)
+				Expect(outputs).To(MatchProps(IgnoreExtras, Props{
+					"resources": MatchArrayValue(HaveExactElements(
+						MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigGroup$kubernetes:core/v1:ConfigMap::test-map-1", "test-map-1"),
+						MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigGroup$kubernetes:core/v1:ConfigMap::test-map-2", "test-map-2"),
+						MatchResourceReferenceValue("urn:pulumi:stack::project::kubernetes:yaml/v2:ConfigGroup$kubernetes:core/v1:ConfigMap::test-map-3", "test-map-3"),
+					)),
+				}))
+			})
+		})
 	})
 
 	Describe("files", func() {