feat: Add support for third-party connection types to aws_glue_connection

[acwwat]

Description

This PR is to add support for the following connection types: AZURECOSMOS, AZURESQL, BIGQUERY, OPENSEARCH, SNOWFLAKE. These types were confirmed by AWS support to be supported but not fully documented. They are also not available in types and enums in various SDKs, including the AWS Go SDK v2 which the Terraform AWS Provider uses.

The approach to add these connection types is to check whether they are in the SDK enum, and add them if they are not. This provides some level of compatibility for when the enum eventually gets updated before the code is removed for good. In addition, a new value SparkProperties is needed for the enum for connection_properties. It is also not documented but supported by the API. Each connection type make require different properties within a JSON object that is provided as the value for SparkProperties. I am not sure how to best document them, so I've added a full example for each type in the documentation.

Relations

Closes #37233
Closes #34575
Closes #32663

References

Examined network traces for CreateConnection requests made in the Console to determine what properties are needed in the JSON object for SparkProperties.

Output from Acceptance Testing

$ make testacc TESTS=TestAccGlueConnection_ PKG=glue
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go1.22.2 test ./internal/service/glue/... -v -count 1 -parallel 20 -run='TestAccGlueConnection_'  -timeout 360m
=== RUN   TestAccGlueConnection_basic
=== PAUSE TestAccGlueConnection_basic
=== RUN   TestAccGlueConnection_tags
=== PAUSE TestAccGlueConnection_tags
=== RUN   TestAccGlueConnection_mongoDB
=== PAUSE TestAccGlueConnection_mongoDB
=== RUN   TestAccGlueConnection_kafka
=== PAUSE TestAccGlueConnection_kafka
=== RUN   TestAccGlueConnection_network
=== PAUSE TestAccGlueConnection_network
=== RUN   TestAccGlueConnection_description
=== PAUSE TestAccGlueConnection_description
=== RUN   TestAccGlueConnection_matchCriteria
=== PAUSE TestAccGlueConnection_matchCriteria
=== RUN   TestAccGlueConnection_physicalConnectionRequirements
=== PAUSE TestAccGlueConnection_physicalConnectionRequirements
=== RUN   TestAccGlueConnection_disappears
=== PAUSE TestAccGlueConnection_disappears
=== RUN   TestAccGlueConnection_azureCosmos
=== PAUSE TestAccGlueConnection_azureCosmos
=== RUN   TestAccGlueConnection_azureSQL
=== PAUSE TestAccGlueConnection_azureSQL
=== RUN   TestAccGlueConnection_bigQuery
=== PAUSE TestAccGlueConnection_bigQuery
=== RUN   TestAccGlueConnection_openSearch
=== PAUSE TestAccGlueConnection_openSearch
=== RUN   TestAccGlueConnection_snowflake
=== PAUSE TestAccGlueConnection_snowflake
=== CONT  TestAccGlueConnection_basic
=== CONT  TestAccGlueConnection_physicalConnectionRequirements
=== CONT  TestAccGlueConnection_network
=== CONT  TestAccGlueConnection_matchCriteria
=== CONT  TestAccGlueConnection_mongoDB
=== CONT  TestAccGlueConnection_description
=== CONT  TestAccGlueConnection_kafka
=== CONT  TestAccGlueConnection_tags
=== CONT  TestAccGlueConnection_bigQuery
=== CONT  TestAccGlueConnection_disappears
=== CONT  TestAccGlueConnection_snowflake
=== CONT  TestAccGlueConnection_azureCosmos
=== CONT  TestAccGlueConnection_openSearch
=== CONT  TestAccGlueConnection_azureSql
--- PASS: TestAccGlueConnection_disappears (23.74s)
--- PASS: TestAccGlueConnection_basic (27.32s)
--- PASS: TestAccGlueConnection_mongoDB (28.18s)
--- PASS: TestAccGlueConnection_kafka (28.81s)
--- PASS: TestAccGlueConnection_snowflake (28.84s)
--- PASS: TestAccGlueConnection_azureCosmos (29.83s)
--- PASS: TestAccGlueConnection_azureSQL (30.09s)
--- PASS: TestAccGlueConnection_openSearch (30.13s)
--- PASS: TestAccGlueConnection_bigQuery (30.13s)
--- PASS: TestAccGlueConnection_network (34.80s)
--- PASS: TestAccGlueConnection_description (38.89s)
--- PASS: TestAccGlueConnection_matchCriteria (47.44s)
--- PASS: TestAccGlueConnection_tags (47.58s)
--- PASS: TestAccGlueConnection_physicalConnectionRequirements (1352.11s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/glue       1352.343s

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this pull request by adding a 👍 reaction to the original post to help the community and maintainers prioritize this pull request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

For Submitters

• Review the contribution guide relating to the type of change you are making to ensure all of the necessary steps have been taken.
• For new resources and data sources, use skaff to generate scaffolding with comments detailing common expectations.
• Whether or not the branch has been rebased will not impact prioritization, but doing so is always a welcome surprise.

[fshskadz]

Hey @acwwat - Two things here.

a) You have conflicts, its in the documentation (I hand patched today)
b) It looks like there is still some validation broken:

│ Error: creating Glue Connection (Reporting - dev): InvalidInputException: Connection properties input is malformed: Unrecognized token 'foo': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false') │ at [Source: (String)"foo"; line: 1, column: 4] │ { │ RespMetadata: { │ StatusCode: 400, │ RequestID: "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaa" │ }, │ Message_: "Connection properties input is malformed: Unrecognized token 'foo': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')\n at [Source: (String)\"foo\"; line: 1, column: 4]" │ } │ │ with module.glue_dev.aws_glue_connection.reporting, │ on ../../../modules/glue/connection.tf line 20, in resource "aws_glue_connection" "reporting": │ 20: resource "aws_glue_connection" "reporting" { │

This doesn't work with any input. It works 100% fine when doing a JDBC connection, so I am guessing maybe something else needs to be added to the validation for connection properties. I tried looking though the code, but got dizzy :). Any help would be VERY VERY much appreciated.

[fshskadz]

Yes, conflicts are there, but I missed the Spark Properties thing, going too fast. Seems to be good now. Thanks!