Develop Component resource for: kubernetes:kustomize:Directory (#3036)

<!--Thanks for your contribution. See [CONTRIBUTING](CONTRIBUTING.md)
    for Pulumi's contribution guidelines.

    Help us merge your changes more quickly by adding more details such
    as labels, milestones, and reviewers.-->

## Proposed changes

This PR implements the Kustomize `Directory` resource (v2) as a
multi-language component resource.

The `Directory` resource creates child resources based on a
kustomization directory. The input is a path to a directory containing
'kustomization.yaml', or a git repository URL with a path suffix and
other qualifiers.

### Related issues (optional)

<!--Refer to related PRs or issues: #1234, or 'Fixes #1234' or 'Closes
#1234'.
Or link to full URLs to issues or pull requests in other GitHub
repositories. -->

Closes #2786 

### API

|property|description|
|-|-| 
| `directory` | The directory or git URL containing the kustomization to
apply. |
| `namespace` | Overrides the default namespace. |
| `skipAwait` | Applies the skipAwait annotation. |
| `resourcePrefix` | Prefix for child resources, defaults to the
component name. |

### Feature: Remote Targets

This implementation supports remote kustomization targets as described
[here](https://github.com/kubernetes-sigs/kustomize/blob/master/examples/remoteBuild.md).

Note: the v1 implementation also supports git references, but resolves
them using the Pulumi SDK
([`RetrieveGitFolder`](https://github.com/pulumi/pulumi/blob/v3.117.0/sdk/go/common/workspace/templates.go#L392))
rather than using Kustomize's own implementation.

### Feature: Enable Alpha Plugins

This implementation always enables Kustomize's plugin support, akin to
`kustomize build --enable-alpha-plugins`.

Plugins are drawn from `KUSTOMIZE_PLUGIN_HOME` (default:
`~/.config/kustomize/plugin/`). Note: the kustomize library doesn't
allow for easy customization of the plugin home.

### Feature: Unrestricted Loading

Kustomize has a strict and a relaxed mode with respect to path
references outside the kustomization base directory
(`--load-restrictor`, default is `strict`). The feature seems intended
to encourage portability, similarly to paths in Dockerfiles. This
implementation simply enables the relaxed mode.

### Feature: Enable Helm Charts

This implementation enables [Helm chart support in
Kustomize](https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/helmcharts/),
which is an experimental feature of Directory v1 (see:
#2470). Note that
chart support in Kustomize is limited and evolving; see
kubernetes-sigs/kustomize#4401 for the
long-term support plan.

The `helm` binary is assumed to be on the path.

### Feature: Namespace Override

New to v2 is support for overriding the default namespace (default is
from provider configuration), as a convenience. Kustomize itself has a
similar facility (see:
[namespace](https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/namespace/))
but it makes sense to support it natively.

### Feature: Resource Ordering

The Directory resource uses Pulumi's engine to install the resources
produced by kustomize. It automatically detects dependencies between
some resources. For example, it knows to install namespaces and Custom
Resource Definitions (CRDs) first.

Use the `config.kubernetes.io/depends-on` annotation to declare an
explicit resource dependency. See [blog
post](https://www.pulumi.com/blog/kubernetes-yaml-v2/#resource-ordering)
for more info.

### Limitation: Kubernetes-Style Transformations

The older v1 implementation provides the ability to transform the
Kubernetes objects produced by kustomize before being sent to the
server. That feature isn't available in Directory v2 at this time. You
may still use Pulumi's [transform
option](https://www.pulumi.com/docs/concepts/options/transformations/)
to modify the child resources.

### Tests

The PR includes unit tests covering the provider implementation code.

### Example

Here's an example of deploying a couple of kustomizations. The `local`
kustomization (see:
[helloWorld](https://github.com/kubernetes-sigs/kustomize/tree/master/examples/helloWorld)
for details) shows how to use a variable to select a kustomization
overlay.

```yaml
name: issue-2786-yaml
runtime: yaml
description: |
  Demonstrates the Directory resource.
  Reference: https://github.com/kubernetes-sigs/kustomize/tree/master/examples/helloWorld
config:
  variant:
    type: string
    default: staging
outputs:
  name: ${local.resources}
resources:
  ns:
    type: kubernetes:core/v1:Namespace
  local:
    type: kubernetes:kustomize/v2:Directory
    properties:
      namespace: ${ns.metadata.name}
      directory: ./config/${variant}
      skipAwait: true
  remote:
    type: kubernetes:kustomize/v2:Directory
    properties:
      namespace: ${ns.metadata.name}
      directory: https://github.com/kubernetes-sigs/kustomize//examples/helloWorld/?ref=v3.3.1
      skipAwait: true
```

CHANGELOG.md

@@ -4,6 +4,7 @@
 - Fixed a panic that occurs when diffing Job resources containing `replaceUnready` annotations and an unreachable cluster connection. (https://github.com/pulumi/pulumi-kubernetes/pull/3024)
 - CustomResource for Java SDK (https://github.com/pulumi/pulumi-kubernetes/pull/3020)
 - Fixed spurious diffing for updates when in renderYaml mode (https://github.com/pulumi/pulumi-kubernetes/pull/3030)
+- Kustomize Directory v2 resource (https://github.com/pulumi/pulumi-kubernetes/pull/3036) 
 
 ## 4.12.0 (May 21, 2024)
 

provider/cmd/pulumi-gen-kubernetes/main.go

@@ -175,6 +175,7 @@ func generateSchema(swaggerPath string) schema.PackageSpec {
 var resourcesToFilterFromTemplate = codegen.NewStringSet(
 	"kubernetes:helm.sh/v3:Release",
 	"kubernetes:helm.sh/v4:Chart",
+	"kubernetes:kustomize/v2:Directory",
 	"kubernetes:yaml/v2:ConfigFile",
 	"kubernetes:yaml/v2:ConfigGroup",
 )

provider/pkg/gen/examples/overlays/kustomizeDirectoryV2.md

@@ -0,0 +1,173 @@
+Directory is a component representing a collection of resources described by a kustomize directory (kustomization).
+
+{{% examples %}}
+## Example Usage
+{{% example %}}
+### Local Kustomize Directory
+
+```yaml
+name: example
+runtime: yaml
+resources:
+  helloWorldLocal:
+    type: kubernetes:kustomize/v2:Directory
+    properties:
+      directory: ./helloWorld
+```
+```typescript
+import * as k8s from "@pulumi/kubernetes";
+
+const helloWorld = new k8s.kustomize.v2.Directory("helloWorldLocal", {
+    directory: "./helloWorld",
+});
+```
+```python
+from pulumi_kubernetes.kustomize.v2 import Directory
+
+hello_world = Directory(
+    "hello-world-local",
+    directory="./helloWorld",
+)
+```
+```csharp
+using System.Threading.Tasks;
+using Pulumi;
+using Pulumi.Kubernetes.Kustomize.V2;
+
+class KustomizeStack : Stack
+{
+    public KustomizeStack()
+    {
+        var helloWorld = new Directory("helloWorldLocal", new DirectoryArgs
+        {
+            Directory = "./helloWorld",
+        });
+    }
+}
+```
+```go
+package main
+
+import (
+	"github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/kustomize/v2"
+	"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
+)
+
+func main() {
+	pulumi.Run(func(ctx *pulumi.Context) error {
+		_, err := kustomize.NewDirectory(ctx, "helloWorldLocal",
+			kustomize.DirectoryArgs{
+				Directory: pulumi.String("./helloWorld"),
+			},
+		)
+		if err != nil {
+			return err
+		}
+
+		return nil
+	})
+}
+```
+```java
+package myproject;
+
+import com.pulumi.Pulumi;
+import com.pulumi.kubernetes.kustomize.v2.Directory;
+import com.pulumi.kubernetes.kustomize.v2.DirectoryArgs;
+
+public class App {
+    public static void main(String[] args) {
+        Pulumi.run(ctx -> {
+            var helloWorld = new Directory("helloWorldLocal", DirectoryArgs.builder()
+                    .directory("./helloWorld")
+                    .build());
+        });
+    }
+}
+```
+{{% /example %}}
+{{% example %}}
+### Kustomize Directory from a Git Repo
+
+```yaml
+name: example
+runtime: yaml
+resources:
+  helloWorldRemote:
+    type: kubernetes:kustomize/v2:Directory
+    properties:
+      directory: https://github.com/kubernetes-sigs/kustomize/tree/v3.3.1/examples/helloWorld
+```
+```typescript
+import * as k8s from "@pulumi/kubernetes";
+
+const helloWorld = new k8s.kustomize.v2.Directory("helloWorldRemote", {
+    directory: "https://github.com/kubernetes-sigs/kustomize/tree/v3.3.1/examples/helloWorld",
+});
+```
+```python
+from pulumi_kubernetes.kustomize.v2 import Directory
+
+hello_world = Directory(
+    "hello-world-remote",
+    directory="https://github.com/kubernetes-sigs/kustomize/tree/v3.3.1/examples/helloWorld",
+)
+```
+```csharp
+using System.Threading.Tasks;
+using Pulumi;
+using Pulumi.Kubernetes.Kustomize.V2;
+
+class KustomizeStack : Stack
+{
+    public KustomizeStack()
+    {
+        var helloWorld = new Directory("helloWorldRemote", new DirectoryArgs
+        {
+            Directory = "https://github.com/kubernetes-sigs/kustomize/tree/v3.3.1/examples/helloWorld",
+        });
+    }
+}
+```
+```go
+package main
+
+import (
+	"github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/kustomize/v2"
+	"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
+)
+
+func main() {
+	pulumi.Run(func(ctx *pulumi.Context) error {
+		_, err := kustomize.NewDirectory(ctx, "helloWorldRemote",
+			kustomize.DirectoryArgs{
+				Directory: pulumi.String("https://github.com/kubernetes-sigs/kustomize/tree/v3.3.1/examples/helloWorld"),
+			},
+		)
+		if err != nil {
+			return err
+		}
+
+		return nil
+	})
+}
+```
+```java
+package myproject;
+
+import com.pulumi.Pulumi;
+import com.pulumi.kubernetes.kustomize.v2.Directory;
+import com.pulumi.kubernetes.kustomize.v2.DirectoryArgs;
+
+public class App {
+    public static void main(String[] args) {
+        Pulumi.run(ctx -> {
+            var helloWorld = new Directory("helloWorldRemote", DirectoryArgs.builder()
+                    .directory("https://github.com/kubernetes-sigs/kustomize/tree/v3.3.1/examples/helloWorld")
+                    .build());
+        });
+    }
+}
+```
+{{% /example %}}
+{{% /examples %}}

provider/pkg/gen/overlays.go

@@ -1230,6 +1230,55 @@ var kustomizeDirectoryResource = pschema.ResourceSpec{
 	},
 }
 
+//go:embed examples/overlays/kustomizeDirectoryV2.md
+var kustomizeDirectoryV2MD string
+
+var kustomizeDirectoryV2Resource = pschema.ResourceSpec{
+	IsComponent: true,
+	ObjectTypeSpec: pschema.ObjectTypeSpec{
+		IsOverlay:   false,
+		Description: kustomizeDirectoryV2MD,
+		Properties: map[string]pschema.PropertySpec{
+			"resources": {
+				TypeSpec: pschema.TypeSpec{
+					Type: "string",
+				},
+				Description: "Resources created by the Directory resource.",
+			},
+		},
+		Type: "object",
+	},
+	InputProperties: map[string]pschema.PropertySpec{
+		"directory": {
+			TypeSpec: pschema.TypeSpec{
+				Type: "string",
+			},
+			Description: "The directory containing the kustomization to apply. The value can be a local directory or a folder in a\ngit repository.\nExample: ./helloWorld\nExample: https://github.com/kubernetes-sigs/kustomize/tree/master/examples/helloWorld",
+		},
+		"namespace": {
+			TypeSpec: pschema.TypeSpec{
+				Type: "string",
+			},
+			Description: "The default namespace to apply to the resources. Defaults to the provider's namespace.",
+		},
+		"resourcePrefix": {
+			TypeSpec: pschema.TypeSpec{
+				Type: "string",
+			},
+			Description: "A prefix for the auto-generated resource names. Defaults to the name of the Directory resource. Example: A resource created with resourcePrefix=\"foo\" would produce a resource named \"foo:resourceName\".",
+		},
+		"skipAwait": {
+			TypeSpec: pschema.TypeSpec{
+				Type: "boolean",
+			},
+			Description: "Indicates that child resources should skip the await logic.",
+		},
+	},
+	RequiredInputs: []string{
+		"directory",
+	},
+}
+
 //go:embed examples/overlays/configFile.md
 var configFileMD string
 
@@ -1615,6 +1664,7 @@ func init() {
 	resourceOverlays["kubernetes:helm.sh/v4:Chart"] = helmV4ChartResource
 	resourceOverlays["kubernetes:helm.sh/v3:Release"] = helmV3ReleaseResource
 	resourceOverlays["kubernetes:kustomize:Directory"] = kustomizeDirectoryResource
+	resourceOverlays["kubernetes:kustomize/v2:Directory"] = kustomizeDirectoryV2Resource
 	resourceOverlays["kubernetes:yaml:ConfigFile"] = yamlConfigFileResource
 	resourceOverlays["kubernetes:yaml/v2:ConfigFile"] = yamlConfigFileV2Resource
 	resourceOverlays["kubernetes:yaml:ConfigGroup"] = yamlConfigGroupResource

provider/pkg/gen/schema.go

@@ -223,15 +223,17 @@ func PulumiSchema(swagger map[string]any) pschema.PackageSpec {
 		"helm.sh/v2":    "Helm.V2",
 		"helm.sh/v3":    "Helm.V3",
 		"helm.sh/v4":    "Helm.V4",
+		"kustomize/v2":  "Kustomize.V2",
 		"yaml":          "Yaml",
 		"yaml/v2":       "Yaml.V2",
 		"":              "Provider",
 	}
 	javaPackages := map[string]string{
-		"helm.sh/v2": "helm.v2",
-		"helm.sh/v3": "helm.v3",
-		"helm.sh/v4": "helm.v4",
-		"yaml/v2":    "yaml.v2",
+		"helm.sh/v2":   "helm.v2",
+		"helm.sh/v3":   "helm.v3",
+		"helm.sh/v4":   "helm.v4",
+		"kustomize/v2": "kustomize.v2",
+		"yaml/v2":      "yaml.v2",
 	}
 	modToPkg := map[string]string{
 		"apiextensions.k8s.io": "apiextensions",
@@ -241,9 +243,10 @@ func PulumiSchema(swagger map[string]any) pschema.PackageSpec {
 		"helm.sh/v4":           "helm/v4",
 	}
 	pkgImportAliases := map[string]string{
-		"github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/helm/v3": "helmv3",
-		"github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/helm/v4": "helmv4",
-		"github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/yaml/v2": "yamlv2",
+		"github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/helm/v3":      "helmv3",
+		"github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/helm/v4":      "helmv4",
+		"github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/kustomize/v2": "kustomizev2",
+		"github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/yaml/v2":      "yamlv2",
 	}
 
 	definitions := swagger["definitions"].(map[string]any)

provider/pkg/provider/README.md

@@ -0,0 +1,21 @@
+
+# Provider Package
+
+This package implements the Kubernetes provider. It provides Pulumi custom resources representing Kubernetes kinds
+and Pulumi component resources for working with Kubernetes manifests and with tools such as Helm and Kustomize. 
+It also provides some invokes to support various "overlay" resources that are implemented at the SDK level.
+
+## Custom Resources
+
+## Component Resources
+
+Each component is implemented via a sub-provider as defined by the `resource.ResourceProvider` interface.
+
+Steps to add a new component provider:
+1. Create a new package for the implementation, e.g. `provider/pkg/provider/helm/v4`.
+2. Add an alias to the package for each SDK in `provider/pkg/gen/schema.go`. You want your package name to be idiomatic to the SDK.
+3. Add an exclusion to `resourcesToFilterFromTemplate` in `provider/cmd/provider-gen-kubernetes/main.go`.
+4. Define a resource schema in `provider/pkg/gen/overlays.go`. The term *overlay* here means "not a Kubernetes kind".
+5. Create a documentation file for the resource in `provider/pkg/gen/examples/overlays/kustomizeDirectory.md` and link from `overlays.go`.
+6. Create an implementation of `resource.ResourceProvider` into your new implementation package.
+7. Register the implementation into `resourceProviders` in `provider/pkg/provider/provider_construct.go`.