[Enhancement]: Add support for Google BigQuery glue connection in terraform

[justin-hong1]

Description

Add support for the new Google BigQuery glue connector in terraform.

https://aws.amazon.com/about-aws/whats-new/2023/10/aws-glue-apache-spark-connectivity-google-bigquery/

Affected Resource(s) and/or Data Source(s)

aws_glue_connection

Potential Terraform Configuration

resource "aws_glue_connection" "bigquery_connection" {
  connection_type = "bigquery"

  connection_properties = {
    SECRET_ID           = aws_secretsmanager_secret.example_secret.name
  }

  name = "bigquery_connection"
}

References

blog: https://aws.amazon.com/about-aws/whats-new/2023/10/aws-glue-apache-spark-connectivity-google-bigquery/

docs: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/glue_connection

Would you like to implement a fix?

None

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[bschaatsbergen]

I'll pick this up! (probably just bumping SDK version, docs and checking whether there's a validateFunc on input values)

[DrFaust92]

Similar to #32663

[dponkin]

The connection can be configured manually and then imported into Terraform, provided that you ignore the type and the properties:

resource "aws_glue_connection" "bigquery_connection" {
  name = "bigquery-connection"

  lifecycle {
    ignore_changes = [connection_type, connection_properties]
  }
}

The actual state of the connection, as generated by Terraform when you remove ignore_changes, looks like this:

resource "aws_glue_connection" "bigquery_connection" {
  name = "bigquery-connection"

  connection_type = "BIGQUERY"
  connection_properties = {
    "SparkProperties" = jsonencode({
      "secretId" = "${var.secretid}"
    })
  }
}

[dponkin]

@bschaatsbergen Do you mind if I submit a PR for this?

[bschaatsbergen]

@bschaatsbergen Do you mind if I submit a PR for this?

Go for it! 😎

[dponkin]

This connection type is not supported by aws-sdk-go itself.

Considering that GA for v1 ends in a few months, I feel like it'd make more sense to use the opportunity to upgrade the connection module to aws-sdk-go-v2.

[cylik93]

@dponkin
Is this implemented yet?

[ksauzz]

According to my investigation, even if we upgrade the connection module to aws-sdk-go-v2, it seems any aws SDKs don't support BIGQuery connection type because the API model file is too old.