Enable Schema Linking via Terraform

[linouk23]

Context

https://www.youtube.com/watch?v=xpKvLeeT_Qc

What

This is a feature request from another issue: #124 (comment)

[mihdih]

Thank you for creating the feature request :)

Just wondering, how far down is this on the roadmap? I guess just wanted to know if when will this be implemented?

[linouk23]

👋 thanks for asking @mihdih!

We can't promising anything but most likely we'll add it next quarter.

[minionOfZuul]

It would be really incredible timing for my org if this feature were getting close to ready for production. There's one more month left in the quarter. Any updates?

[linouk23]

@minionOfZuul I can't promise anything but it's currently in a code review stage.

[linouk23]

@mihdih @minionOfZuul @brunodominici @aaronphilip

We've got a quick question for you: would you prefer option #1 or option #2:

# Option #1
resource "confluent_schema_exporter" "main" {
  schema_registry_cluster {
    id = confluent_schema_registry_cluster.essentials.id
  }
  rest_endpoint = confluent_schema_registry_cluster.essentials.rest_endpoint
  credentials {
    key    = "<Schema Registry API Key for confluent_schema_registry_cluster.essentials>"
    secret = "<Schema Registry API Secret for confluent_schema_registry_cluster.essentials>"
  }
  
  name = "test-exporter"
  context = "test-context"
  context_type = "CUSTOM"
  subjects = ["foo"]
  config_nonsensitive = {
      "basic.auth.credentials.source" = "USER_INFO",
      "schema.registry.url" = "<destination sr url>"
  }
  config_sensitive = {
      "basic.auth.user.info" = "<destination api key>:<destination api secret>"
  }
  status = "RUNNING"
  reset_on_update = true
}

# Option #2 to mirror cluster link resource
resource "confluent_schema_exporter" "main" {
  schema_registry_cluster {
    id = confluent_schema_registry_cluster.essentials.id
  }
  rest_endpoint = confluent_schema_registry_cluster.essentials.rest_endpoint
  credentials {
    key    = "<Schema Registry API Key for confluent_schema_registry_cluster.essentials>"
    secret = "<Schema Registry API Secret for confluent_schema_registry_cluster.essentials>"
  }
  
  name = "test-exporter"
  context = "test-context"
  context_type = "CUSTOM"
  subjects = ["foo"]
  destination_schema_registry_cluster {
    id                 = ...
    rest_endpoint = ...
    credentials {
      key    = ...
      secret = ...
    }
  }
  status = "RUNNING"
  reset_on_update = true
  config = {
      # any SL specific settings if there're any
      "retention.ms" = "123"
  }
}

[brunodomenici]

@mihdih @minionOfZuul @brunodominici @aaronphilip

We've got a quick question for you: would you prefer option #1 or option #2:

Option 2 for me 👍

[minionOfZuul]

Same, Option 2 makes more sense to me.

[linouk23]

Update: we

Added new confluent_schema_exporter resource in a Preview lifecycle stage (see schema-linking example)
in our latest 1.52.0 version of TF Provider.

cc @mihdih @minionOfZuul @brunodominici @aaronphilip

Please let us know if there are any issues or improvements we can make. Thank you!

[mihdih]

@linouk23 Apologies for a very late response. Thank again for all the work and for adding this in. 🙇

[linouk23]

Thanks @mihdih!

@zhenli00 did all the work by himself 😁 and he also implemented a fix: #321 that we'll release in our next release.

[linouk23]

fyi we're closing this issue as we added a corresponding resource last November in the 1.52.0 version of TF Provider: https://registry.terraform.io/providers/confluentinc/confluent/latest/docs/resources/confluent_schema_exporter