resource/aws_api_gateway_rest_api: Additional OpenAPI specification acceptance testing and fix various attributes after import

[bflad]

Community Note

• Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Closes #5364
Closes #7161
Closes #9722
Closes #10766
Supersedes #12432
Reference: #13841
Closes #14290
Closes #14660

Release note for CHANGELOG:

* resource/aws_api_gateway_rest_api: Ensure `api_key_source`, `binary_media_types`, `description`, `minimum_compression_size`, `name`, and `policy` configuration values are correctly applied as an override after OpenAPI specification import (`body` argument)
* resource/aws_api_gateway_rest_api: Allow `api_key_source`, `binary_media_types`, and `description` arguments to be omitted from configuration with OpenAPI specification import (`body` argument)

The overall testing changes are:

• Ensuring the basic test covers all attributes
• Refactoring the basic test into per-attribute testing
• Adding per-attribute tests to cover OpenAPI specificiations (body argument) being set without Terraform configurations -- these should be allowed with Terraform showing a planned difference
• Adding per-attribute tests to cover OpenAPI specificiations (body argument) being set with Terraform configurations -- these should be allowed with the Terraform configuration value overriding the OpenAPI specification
• Removing extraneous API object TestCheckFunc (covered by ImportStateVerify testing)

It is worth mentioning that this does not cover the disable_execute_api_endpoint or tags attributes that can also be specified by OpenAPI since this change set is already very large. The minimum_compression_size attribute also needs an additional update to support OpenAPI-only configuration. Further updates can improve on this effort.

Before code updates, these new acceptance tests show how the Terraform configuration value would not be applied if an OpenAPI specification was imported:

=== CONT  TestAccAWSAPIGatewayRestApi_ApiKeySource_OverrideBody
    resource_aws_api_gateway_rest_api_test.go:428: Step 1/4 error: Check failed: 1 error occurred:
        	* Check 2/2 error: aws_api_gateway_rest_api.test: Attribute 'api_key_source' expected "AUTHORIZER", got "HEADER"

--- FAIL: TestAccAWSAPIGatewayRestApi_ApiKeySource_OverrideBody (8.82s)

=== CONT  TestAccAWSAPIGatewayRestApi_BinaryMediaTypes_OverrideBody
    resource_aws_api_gateway_rest_api_test.go:464: Step 1/4 error: Check failed: 1 error occurred:
        	* Check 3/3 error: aws_api_gateway_rest_api.test: Attribute 'binary_media_types.0' expected "application/octet-stream", got "image/jpeg"

=== CONT  TestAccAWSAPIGatewayRestApi_Description_OverrideBody
    resource_aws_api_gateway_rest_api_test.go:527: Step 1/4 error: Check failed: 1 error occurred:
        	* Check 2/2 error: aws_api_gateway_rest_api.test: Attribute 'description' expected "tfdescription1", got "oasdescription1"

--- FAIL: TestAccAWSAPIGatewayRestApi_Description_OverrideBody (9.60s)

=== CONT  TestAccAWSAPIGatewayRestApi_MinimumCompressionSize_OverrideBody
    resource_aws_api_gateway_rest_api_test.go:688: Step 1/4 error: Check failed: 1 error occurred:
        	* Check 2/2 error: aws_api_gateway_rest_api.test: Attribute 'minimum_compression_size' expected "1", got "5242880"

--- FAIL: TestAccAWSAPIGatewayRestApi_MinimumCompressionSize_OverrideBody (8.41s)

=== CONT  TestAccAWSAPIGatewayRestApi_Name_OverrideBody
    resource_aws_api_gateway_rest_api_test.go:528: Step 1/4 error: Check failed: 1 error occurred:
        	* Check 2/2 error: aws_api_gateway_rest_api.test: Attribute 'name' expected "tf-acc-test-4252368909257291928", got "title1"

--- FAIL: TestAccAWSAPIGatewayRestApi_Name_OverrideBody (8.57s)

=== CONT  TestAccAWSAPIGatewayRestApi_Policy_OverrideBody
    resource_aws_api_gateway_rest_api_test.go:593: Step 1/4 error: Check failed: 1 error occurred:
        	* Check 4/4 error: aws_api_gateway_rest_api.test: Attribute 'policy' didn't match "\"Allow\"", got ""

--- FAIL: TestAccAWSAPIGatewayRestApi_Policy_OverrideBody (9.37s)

Before code updates, these acceptance tests show how the Terraform resource would report an unexpected difference for missing configurations that were imported by the OpenAPI specification:

=== CONT  TestAccAWSAPIGatewayRestApi_ApiKeySource_SetByBody
    resource_aws_api_gateway_rest_api_test.go:471: Step 1/2 error: After applying this test step, the plan was not empty.
        stdout:

        An execution plan has been generated and is shown below.
        Resource actions are indicated with the following symbols:
          ~ update in-place

        Terraform will perform the following actions:

          # aws_api_gateway_rest_api.test will be updated in-place
          ~ resource "aws_api_gateway_rest_api" "test" {
              ~ api_key_source               = "AUTHORIZER" -> "HEADER"
                id                           = "5ja4mnzxta"
                name                         = "tf-acc-test-4415455482847955650"
                # (8 unchanged attributes hidden)

                # (1 unchanged block hidden)
            }

        Plan: 0 to add, 1 to change, 0 to destroy.
--- FAIL: TestAccAWSAPIGatewayRestApi_ApiKeySource_SetByBody (20.65s)

=== CONT  TestAccAWSAPIGatewayRestApi_BinaryMediaTypes_SetByBody
    resource_aws_api_gateway_rest_api_test.go:510: Step 1/2 error: After applying this test step, the plan was not empty.
        stdout:

        An execution plan has been generated and is shown below.
        Resource actions are indicated with the following symbols:
          ~ update in-place

        Terraform will perform the following actions:

          # aws_api_gateway_rest_api.test will be updated in-place
          ~ resource "aws_api_gateway_rest_api" "test" {
              ~ binary_media_types           = [
                  - "application/octet-stream",
                ]
                id                           = "7we4bv4s8b"
                name                         = "tf-acc-test-2053199682951305540"
                # (8 unchanged attributes hidden)

                # (1 unchanged block hidden)
            }

        Plan: 0 to add, 1 to change, 0 to destroy.

=== CONT  TestAccAWSAPIGatewayRestApi_Description_SetByBody
    resource_aws_api_gateway_rest_api_test.go:570: Step 1/2 error: After applying this test step, the plan was not empty.
        stdout:

        An execution plan has been generated and is shown below.
        Resource actions are indicated with the following symbols:
          ~ update in-place

        Terraform will perform the following actions:

          # aws_api_gateway_rest_api.test will be updated in-place
          ~ resource "aws_api_gateway_rest_api" "test" {
              - description                  = "oasdescription1" -> null
                id                           = "3k0fykhp76"
                name                         = "tf-acc-test-2107985362088533117"
                # (8 unchanged attributes hidden)

                # (1 unchanged block hidden)
            }

        Plan: 0 to add, 1 to change, 0 to destroy.
--- FAIL: TestAccAWSAPIGatewayRestApi_Description_SetByBody (10.02s)

=== CONT  TestAccAWSAPIGatewayRestApi_MinimumCompressionSize_SetByBody
    resource_aws_api_gateway_rest_api_test.go:731: Step 1/2 error: After applying this test step, the plan was not empty.
        stdout:

        An execution plan has been generated and is shown below.
        Resource actions are indicated with the following symbols:
          ~ update in-place

        Terraform will perform the following actions:

          # aws_api_gateway_rest_api.test will be updated in-place
          ~ resource "aws_api_gateway_rest_api" "test" {
                id                           = "bcmvzz0jfi"
              ~ minimum_compression_size     = 1048576 -> -1
                name                         = "tf-acc-test-2006611344091675720"
                # (7 unchanged attributes hidden)

                # (1 unchanged block hidden)
            }

        Plan: 0 to add, 1 to change, 0 to destroy.
--- FAIL: TestAccAWSAPIGatewayRestApi_MinimumCompressionSize_SetByBody (10.99s)

Additionally these new acceptance tests show how the Terraform resource already respected missing configurations that were imported by the OpenAPI specification:

--- PASS: TestAccAWSAPIGatewayRestApi_Policy_SetByBody (15.03s)

Output from acceptance testing:

--- PASS: TestAccAWSAPIGatewayRestApi_ApiKeySource (28.57s)
--- PASS: TestAccAWSAPIGatewayRestApi_ApiKeySource_OverrideBody (52.53s)
--- PASS: TestAccAWSAPIGatewayRestApi_ApiKeySource_SetByBody (25.48s)
--- PASS: TestAccAWSAPIGatewayRestApi_basic (23.16s)
--- PASS: TestAccAWSAPIGatewayRestApi_BinaryMediaTypes (80.33s)
--- PASS: TestAccAWSAPIGatewayRestApi_BinaryMediaTypes_OverrideBody (34.45s)
--- PASS: TestAccAWSAPIGatewayRestApi_BinaryMediaTypes_SetByBody (24.16s)
--- PASS: TestAccAWSAPIGatewayRestApi_Body (26.69s)
--- PASS: TestAccAWSAPIGatewayRestApi_Description (765.29s)
--- PASS: TestAccAWSAPIGatewayRestApi_Description_OverrideBody (32.87s)
--- PASS: TestAccAWSAPIGatewayRestApi_Description_SetByBody (51.31s)
--- PASS: TestAccAWSAPIGatewayRestApi_DisableExecuteApiEndpoint (30.21s)
--- PASS: TestAccAWSAPIGatewayRestApi_disappears (38.64s)
--- PASS: TestAccAWSAPIGatewayRestApi_EndpointConfiguration (58.23s)
--- PASS: TestAccAWSAPIGatewayRestApi_EndpointConfiguration_Private (15.02s)
--- PASS: TestAccAWSAPIGatewayRestApi_EndpointConfiguration_VPCEndpoint (305.78s)
--- PASS: TestAccAWSAPIGatewayRestApi_MinimumCompressionSize (42.89s)
--- PASS: TestAccAWSAPIGatewayRestApi_MinimumCompressionSize_OverrideBody (35.97s)
--- PASS: TestAccAWSAPIGatewayRestApi_MinimumCompressionSize_SetByBody (106.39s)
--- PASS: TestAccAWSAPIGatewayRestApi_Name_OverrideBody (86.16s)
--- PASS: TestAccAWSAPIGatewayRestApi_Parameters (39.90s)
--- PASS: TestAccAWSAPIGatewayRestApi_Policy (683.47s)
--- PASS: TestAccAWSAPIGatewayRestApi_Policy_OverrideBody (905.68s)
--- PASS: TestAccAWSAPIGatewayRestApi_Policy_SetByBody (28.12s)
--- PASS: TestAccAWSAPIGatewayRestApi_tags (32.94s)

[YakDriver]

Looks great! 🎉 Just a few minor nits, comments, questions.

GovCloud acceptance:

--- PASS: TestAccAWSAPIGatewayRestApi_EndpointConfiguration_Private (25.54s)
--- PASS: TestAccAWSAPIGatewayRestApi_EndpointConfiguration_VPCEndpoint (339.84s)
--- SKIP: TestAccAWSAPIGatewayRestApi_ApiKeySource (1.28s)
--- SKIP: TestAccAWSAPIGatewayRestApi_ApiKeySource_OverrideBody (1.28s)
--- SKIP: TestAccAWSAPIGatewayRestApi_ApiKeySource_SetByBody (0.00s)
--- SKIP: TestAccAWSAPIGatewayRestApi_basic (1.28s)
--- SKIP: TestAccAWSAPIGatewayRestApi_BinaryMediaTypes (0.00s)
--- SKIP: TestAccAWSAPIGatewayRestApi_BinaryMediaTypes_OverrideBody (1.28s)
--- SKIP: TestAccAWSAPIGatewayRestApi_BinaryMediaTypes_SetByBody (1.28s)
--- SKIP: TestAccAWSAPIGatewayRestApi_Body (1.28s)
--- SKIP: TestAccAWSAPIGatewayRestApi_Description (1.28s)
--- SKIP: TestAccAWSAPIGatewayRestApi_Description_OverrideBody (1.28s)
--- SKIP: TestAccAWSAPIGatewayRestApi_Description_SetByBody (1.28s)
--- SKIP: TestAccAWSAPIGatewayRestApi_DisableExecuteApiEndpoint (1.28s)
--- SKIP: TestAccAWSAPIGatewayRestApi_disappears (0.00s)
--- SKIP: TestAccAWSAPIGatewayRestApi_EndpointConfiguration (30.88s)
--- SKIP: TestAccAWSAPIGatewayRestApi_MinimumCompressionSize (1.28s)
--- SKIP: TestAccAWSAPIGatewayRestApi_MinimumCompressionSize_OverrideBody (1.28s)
--- SKIP: TestAccAWSAPIGatewayRestApi_MinimumCompressionSize_SetByBody (1.28s)
--- SKIP: TestAccAWSAPIGatewayRestApi_Name_OverrideBody (1.28s)
--- SKIP: TestAccAWSAPIGatewayRestApi_Parameters (1.28s)
--- SKIP: TestAccAWSAPIGatewayRestApi_Policy (1.28s)
--- SKIP: TestAccAWSAPIGatewayRestApi_Policy_OverrideBody (1.28s)
--- SKIP: TestAccAWSAPIGatewayRestApi_Policy_SetByBody (1.28s)
--- SKIP: TestAccAWSAPIGatewayRestApi_tags (0.00s)

Commercial acceptance:

--- PASS: TestAccAWSAPIGatewayRestApi_ApiKeySource (124.81s)
--- PASS: TestAccAWSAPIGatewayRestApi_ApiKeySource_OverrideBody (239.20s)
--- PASS: TestAccAWSAPIGatewayRestApi_ApiKeySource_SetByBody (724.71s)
--- PASS: TestAccAWSAPIGatewayRestApi_basic (114.21s)
--- PASS: TestAccAWSAPIGatewayRestApi_BinaryMediaTypes (475.54s)
--- PASS: TestAccAWSAPIGatewayRestApi_BinaryMediaTypes_OverrideBody (211.09s)
--- PASS: TestAccAWSAPIGatewayRestApi_BinaryMediaTypes_SetByBody (18.22s)
--- PASS: TestAccAWSAPIGatewayRestApi_Body (441.77s)
--- PASS: TestAccAWSAPIGatewayRestApi_Description (82.63s)
--- PASS: TestAccAWSAPIGatewayRestApi_Description_OverrideBody (552.04s)
--- PASS: TestAccAWSAPIGatewayRestApi_Description_SetByBody (904.68s)
--- PASS: TestAccAWSAPIGatewayRestApi_DisableExecuteApiEndpoint (114.88s)
--- PASS: TestAccAWSAPIGatewayRestApi_disappears (628.27s)
--- PASS: TestAccAWSAPIGatewayRestApi_EndpointConfiguration (228.60s)
--- PASS: TestAccAWSAPIGatewayRestApi_EndpointConfiguration_Private (247.87s)
--- PASS: TestAccAWSAPIGatewayRestApi_EndpointConfiguration_VPCEndpoint (305.36s)
--- PASS: TestAccAWSAPIGatewayRestApi_MinimumCompressionSize (376.50s)
--- PASS: TestAccAWSAPIGatewayRestApi_MinimumCompressionSize_OverrideBody (49.50s)
--- PASS: TestAccAWSAPIGatewayRestApi_MinimumCompressionSize_SetByBody (514.46s)
--- PASS: TestAccAWSAPIGatewayRestApi_Name_OverrideBody (176.46s)
--- PASS: TestAccAWSAPIGatewayRestApi_Parameters (652.60s)
--- PASS: TestAccAWSAPIGatewayRestApi_Policy (616.84s)
--- PASS: TestAccAWSAPIGatewayRestApi_Policy_OverrideBody (357.06s)
--- PASS: TestAccAWSAPIGatewayRestApi_Policy_SetByBody (410.33s)
--- PASS: TestAccAWSAPIGatewayRestApi_tags (707.86s)

[YakDriver]

Nice! 👍

[YakDriver]

Nit: I love one liners but conflicted on these. Readability and all. Not sure if there's even an alternative.

[bflad]

We could switch to just doing a simple regular expression check for something meaningful in the policy (e.g. aws:SourceIp or whatnot). There's a few other places across the provider that do this full JSON text testing, so might be worth creating a proposal if interested.

[YakDriver]

Perhaps efforts are underway already, but it would be handy if GetOk() were modified for lists/sets or a new method were added (maybe GetOkSest()? GetOkLet()? GetOkCollection()? I'm guessing there are many instances in the code base we don't have the len(v.([]interface{})) > 0 check.

[bflad]

Much of the existing Terraform Plugin SDK v2 should be considered frozen, so it is unlikely that the existing behavior would be modified since there can be valid use cases for d.GetOk() with no collection elements today. A new receiver method might be warranted in this case though, but I think we would need to do some analysis on how often we should be using it in this provider and others, since it is not necessary for every case.

The real problem in this space is that the semantics of each API makes it difficult to recommend the length check or not (let alone perform static analysis). Even within the AWS provider there is cases where zero element lists should be passed during updates, etc.

[ghost]

This has been released in version 3.25.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!