Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add terraform mapping to k8s provider #2457

Merged
merged 1 commit into from
Sep 1, 2023
Merged

Conversation

Frassle
Copy link
Member

@Frassle Frassle commented Jun 14, 2023

Proposed changes

This returns a basic mapping description for the terraform converter to map terraform programs using https://registry.terraform.io/providers/hashicorp/kubernetes to our kubernetes provider, even though it's not bridged.

I grabbed a list of all terraform kubernetes resources by running terraform providers schema -json in a tiny terraform program that just defined a kubernetes provider. I've then hooked that up to gen-kubernetes to do a best guess of mapping everything (well just resources so far) from terraform to our provider. Most things look to map pretty well by convention, but we'll probably need some manual fixups to cover everything (already got one for the "container" to containers" rename).

Related issues (optional)

@github-actions
Copy link

Does the PR have any schema changes?

Looking good! No breaking changes found.
No new resources/functions.

provider/pkg/provider/provider.go Outdated Show resolved Hide resolved
provider/pkg/provider/provider.go Outdated Show resolved Hide resolved
@Frassle Frassle force-pushed the fraser/getMappingTf branch 4 times, most recently from ee9f9f4 to d68bd83 Compare June 16, 2023 15:41
@Frassle
Copy link
Member Author

Frassle commented Jun 16, 2023

With this mapping in place the following terraform:

provider "kubernetes" {
}

resource "kubernetes_deployment" "nginx" {
  metadata {
    name = "scalable-nginx-example"
    labels = {
      App = "ScalableNginxExample"
    }
  }

  spec {
    replicas = 2
    selector {
      match_labels = {
        App = "ScalableNginxExample"
      }
    }
    template {
      metadata {
        labels = {
          App = "ScalableNginxExample"
        }
      }
      spec {
        container {
          image = "nginx:1.7.8"
          name  = "example"

          port {
            container_port = 80
          }

          resources {
            limits = {
              cpu    = "0.5"
              memory = "512Mi"
            }
            requests = {
              cpu    = "250m"
              memory = "50Mi"
            }
          }
        }
      }
    }
  }
}

resource "kubernetes_service" "nginx" {
  metadata {
    name = "nginx-example"
  }
  spec {
    selector = {
      App = kubernetes_deployment.nginx.spec.0.template.0.metadata[0].labels.App
    }
    port {
      node_port   = 30201
      port        = 80
      target_port = 80
    }

    type = "NodePort"
  }
}

Get converted to the following TypeScript:

import * as pulumi from "@pulumi/pulumi";
import * as kubernetes from "@pulumi/kubernetes";

const nginx = new kubernetes.apps.v1beta1.Deployment("nginx", {
    metadata: {
        name: "scalable-nginx-example",
        labels: {
            app: "ScalableNginxExample",
        },
    },
    spec: {
        replicas: 2,
        selector: {
            matchLabels: {
                app: "ScalableNginxExample",
            },
        },
        template: {
            metadata: {
                labels: {
                    app: "ScalableNginxExample",
                },
            },
            spec: {
                containers: [{
                    image: "nginx:1.7.8",
                    name: "example",
                    ports: [{
                        containerPort: 80,
                    }],
                    resources: {
                        limits: {
                            cpu: "0.5",
                            memory: "512Mi",
                        },
                        requests: {
                            cpu: "250m",
                            memory: "50Mi",
                        },
                    },
                }],
            },
        },
    },
});
const nginxResource = new kubernetes.core.v1.Service("nginxResource", {
    metadata: {
        name: "nginx-example",
    },
    spec: {
        selector: {
            app: nginx.spec.apply(spec => spec?.template?.metadata?.labels?.app),
        },
        ports: [{
            nodePort: 30201,
            port: 80,
            targetPort: 80,
        }],
        type: "NodePort",
    },
});

@lblackstone
Copy link
Member

lblackstone commented Jun 16, 2023

That conversion looks good except for the selected resource version. v1beta1.Deployment is deprecated and is only retained for backward compat on very old cluster versions.

It should use v1.Deployment instead

@Frassle
Copy link
Member Author

Frassle commented Jun 16, 2023

Ah cool I wasn't sure because TF had both kubernetes_deployment_v1 and kubernetes_deployment. I wasn't sure if the non versioned one should go to the first version released or the latest.

@lblackstone
Copy link
Member

Ah cool I wasn't sure because TF had both kubernetes_deployment_v1 and kubernetes_deployment. I wasn't sure if the non versioned one should go to the first version released or the latest.

I think picking the latest version should generally be safe/backward compatible. Lets do that for now, and can revise if we run into problems.

@Frassle Frassle force-pushed the fraser/getMappingTf branch 3 times, most recently from cf7aa2b to ef81c00 Compare June 16, 2023 19:33
Copy link
Member

@lblackstone lblackstone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes LGTM, but is there any testing for this?

@Frassle
Copy link
Member Author

Frassle commented Aug 31, 2023

Changes LGTM, but is there any testing for this?

No, I wasn't sure what the best way to test this was?
We can write a .tf program and call convert on it, but should we convert to PCL and check the output of that or would y'all rather we pick a language (like typescript) and check it coverts all the way to that?

@lblackstone
Copy link
Member

No, I wasn't sure what the best way to test this was?

cc @t0yv0 @iwahbe @rquitales for thoughts

@iwahbe iwahbe self-requested a review August 31, 2023 18:32
Copy link
Member

@iwahbe iwahbe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

YAML is fast, easy to target and aggressively type checked. HCL -> YAML and then previewing the yaml sounds like a great test.

Comment on lines 532 to 533
type TerraformAttributeSchema struct {
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
type TerraformAttributeSchema struct {
}
type TerraformAttributeSchema struct {}

@Frassle
Copy link
Member Author

Frassle commented Aug 31, 2023

previewing the yaml sounds like a great test.

As in pulumi preview? Does that not need a k8s cluster spun up to run against?

@iwahbe
Copy link
Member

iwahbe commented Aug 31, 2023

previewing the yaml sounds like a great test.

As in pulumi preview? Does that not need a k8s cluster spun up to run against?

Why would pulumi preview require existing infrastructure? I'm not saying we should deploy anything, just use pulumi preview to perform a type check.

@Frassle
Copy link
Member Author

Frassle commented Aug 31, 2023

Why would pulumi preview require existing infrastructure?

Same reason an aws preview complains if your not yet logged into aws.

@Frassle Frassle force-pushed the fraser/getMappingTf branch 7 times, most recently from 39e6470 to 1e78c5b Compare August 31, 2023 22:01
This returns a basic mapping description for the terraform converter to
map terraform programs using
https://registry.terraform.io/providers/hashicorp/kubernetes to our
kubernetes provider, even though it's not bridged.

I grabbed a list of all terraform kubernetes resources by running
`terraform providers schema -json` in a tiny terraform program that just
defined a kubernetes provider. I've then hooked that up to
gen-kubernetes to do a best guess of mapping everything (well just
resources so far) from terraform to our provider. Most things look to
map pretty well by convention, but we'll probably need some manual
fixups to cover everything (already got one for the "container" to
"containers" rename).
@Frassle Frassle merged commit 77ba176 into master Sep 1, 2023
20 checks passed
@Frassle Frassle deleted the fraser/getMappingTf branch September 1, 2023 10:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants