import error: aws_vpc_endpoint_route_table_association: InvalidVpcEndpointId.Malformed: Invalid Id: '' (expecting 'vpce-...;

[ghost]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.11.14

• provider.aws v2.22.0

Affected Resource(s)

aws_vpc_endpoint_route_table_association

Terraform Configuration Files

Only configuration used:

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 1.46.0"
...
}

Debug Output

Please see attachments.
when_vpce_id_used.txt
when_rt_associ_used.txt

Expected Behavior

Should process terraform import [of reasource aws_vpc_endpoint_route_table_association] without error.

Actual Behavior

Hey!

Trying to add our manually created VPC into Terraform*, every import goes well, till we are trying to import manually created aws_vpc_endpoint_route_table_association.public_s3

*adding our manually created VPC to TF by using module:

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 1.46.0"
...
}

When we are importing 'aws_vpc_endpoint_route_table_association', unfortunately we get following error:

Error: module.vpc.aws_vpc_endpoint_route_table_association.public_s3 (import id: rtbassoc-999b9cf): 1 error occurred:
	* import module.vpc.aws_vpc_endpoint_route_table_association.public_s3 result: rtbassoc-999b9cf: aws_vpc_endpoint_route_table_association.public_s3: InvalidVpcEndpointId.Malformed: Invalid Id: '' (expecting 'vpce-...; the Id may only contain lowercase alphanumeric characters and a single dash')
	status code: 400, request id: 2ff61888-01cb-4476-b834-f389f5697133

The same happens when we try to change import id into required vpce:

Error: module.vpc.aws_vpc_endpoint_route_table_association.public_s3 (import id: vpce-999ca99d): 1 error occurred:
	* import module.vpc.aws_vpc_endpoint_route_table_association.public_s3 result: vpce-999ca99d: aws_vpc_endpoint_route_table_association.public_s3: InvalidVpcEndpointId.Malformed: Invalid Id: '' (expecting 'vpce-...; the Id may only contain lowercase alphanumeric characters and a single dash')
	status code: 400, request id: 9ed351f6-4085-4bf8-a0da-968536c64259

In the attachment you can find the log from debugging (TRACE).

When provide terraform plan, the output for this resource looks like this:

   + module.vpc.aws_vpc_endpoint_route_table_association.public_s3
       id:                      <computed>
       route_table_id:          "rtb-e0a97999"
       vpc_endpoint_id:        "vpce-999ca99d"

Do you think it might be bug? Any idea/help, please? Thanks a lot!

Steps to Reproduce

0. Manually create VPC
1. Then add aws vpc module [see 'Terraform Configuration Files' part above]
2. Run terraform plan and import of all required (new) resources.
3. When touching the aws_vpc_endpoint_route_table_association you will come to the reported issue.

References

Posted also here: terraform-aws-modules/terraform-aws-vpc#312

[diogomatsubara]

I managed to reproduce this issue with plain terraform (without the extra aws modules) with the following steps:

1. Create a file vpc.tf

variable region {
  type    = "string"
  default = "eu-west-1"
}

resource "aws_vpc" "test-vpc" {
  cidr_block = "10.0.0.0/16"
}

resource "aws_route_table" "test-rt" {
  vpc_id = "${aws_vpc.test-vpc.id}"
}

resource "aws_route_table" "test-rt-2" {
  vpc_id = "${aws_vpc.test-vpc.id}"
}

resource "aws_vpc_endpoint" "s3" {
  vpc_id       = "${aws_vpc.test-vpc.id}"
  service_name = "com.amazonaws.${var.region}.s3"
}

resource "aws_vpc_endpoint_route_table_association" "internal-subnet-s3" {
  route_table_id  = "${aws_route_table.test-rt-2.id}"
  vpc_endpoint_id = "${aws_vpc_endpoint.s3.id}"
}

2. Deploy that to aws with terraform apply
3. Remove the association from the state: terraform state rm -state terraform.tfstate aws_vpc_endpoint_route_table_association.internal-subnet-s3
4. Try to import the association back into the state (change vpc association id accordingly): terraform import -state terraform.tfstate aws_vpc_endpoint_route_table_association.internal-subnet-s3 a-vpce-08363f49778ed4ed2675153282
5. Get the following error:

aws_vpc_endpoint_route_table_association.internal-subnet-s3: Importing from ID "a-vpce-08363f49778ed4ed2675153282"...
aws_vpc_endpoint_route_table_association.internal-subnet-s3: Import complete!
  Imported aws_vpc_endpoint_route_table_association (ID: a-vpce-08363f49778ed4ed2675153282)
aws_vpc_endpoint_route_table_association.internal-subnet-s3: Refreshing state... (ID: a-vpce-08363f49778ed4ed2675153282)

Error: aws_vpc_endpoint_route_table_association.internal-subnet-s3 (import id: a-vpce-08363f49778ed4ed2675153282): 1 error(s) occurred:

* import aws_vpc_endpoint_route_table_association.internal-subnet-s3 result: a-vpce-08363f49778ed4ed2675153282: aws_vpc_endpoint_route_table_association.internal-subnet-s3: InvalidVpcEndpointId.Malformed: Invalid Id: '' (expecting 'vpce-...; the Id may only contain lowercase alphanumeric characters and a single dash')
	status code: 400, request id: cffa5cee-d81a-4c15-893e-03fd74db5fdd

So it seems the creation code works correctly, but the import code is failing in some id validation. Notice in the trace logs how the vpc endpoint id passed to the DescribeVpcEndpoints api call gets an "" id.

aws_vpc_endpoint_route_table_association.internal-subnet-s3: Refreshing state... (ID: a-vpce-08363f49778ed4ed2675153282)
2019-09-06T10:14:13.658+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4: 2019/09/06 10:14:13 [DEBUG] [aws-sdk-go] DEBUG: Request ec2/DescribeVpcEndpoints Details:
2019-09-06T10:14:13.658+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4: ---[ REQUEST POST-SIGN ]-----------------------------
2019-09-06T10:14:13.658+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4: POST / HTTP/1.1
2019-09-06T10:14:13.658+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4: Host: ec2.eu-west-1.amazonaws.com
2019-09-06T10:14:13.658+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4: User-Agent: aws-sdk-go/1.23.15 (go1.12.6; darwin; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.12.7
2019-09-06T10:14:13.658+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4: Content-Length: 63
2019-09-06T10:14:13.658+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4: Authorization: [redacted]
2019-09-06T10:14:13.658+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4: Content-Type: application/x-www-form-urlencoded; charset=utf-8
2019-09-06T10:14:13.658+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4: X-Amz-Date: 20190906T081413Z
2019-09-06T10:14:13.658+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4: Accept-Encoding: gzip
2019-09-06T10:14:13.658+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4:
2019-09-06T10:14:13.658+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4: Action=DescribeVpcEndpoints&Version=2016-11-15&VpcEndpointId.1=
2019-09-06T10:14:13.658+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4: -----------------------------------------------------
2019-09-06T10:14:13.988+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4: 2019/09/06 10:14:13 [DEBUG] [aws-sdk-go] DEBUG: Response ec2/DescribeVpcEndpoints Details:
2019-09-06T10:14:13.988+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4: ---[ RESPONSE ]--------------------------------------
2019-09-06T10:14:13.988+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4: HTTP/1.1 400 Bad Request
2019-09-06T10:14:13.988+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4: Connection: close
2019-09-06T10:14:13.988+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4: Transfer-Encoding: chunked
2019-09-06T10:14:13.988+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4: Date: Fri, 06 Sep 2019 08:14:13 GMT
2019-09-06T10:14:13.988+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4: Server: AmazonEC2
2019-09-06T10:14:13.988+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4:
2019-09-06T10:14:13.988+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4:
2019-09-06T10:14:13.988+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4: -----------------------------------------------------
2019-09-06T10:14:13.988+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4: 2019/09/06 10:14:13 [DEBUG] [aws-sdk-go] <?xml version="1.0" encoding="UTF-8"?>
2019-09-06T10:14:13.988+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4: <Response><Errors><Error><Code>InvalidVpcEndpointId.Malformed</Code><Message>Invalid Id: '' (expecting 'vpce-...; the Id may only contain lowercase alphanumeric characters and a single dash')</Message></Error></Errors><RequestID>f3e7c80c-68ec-433b-9a09-b107a675c08e</RequestID></Response>
2019-09-06T10:14:13.988+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4: 2019/09/06 10:14:13 [DEBUG] [aws-sdk-go] DEBUG: Validate Response ec2/DescribeVpcEndpoints failed, attempt 0/25, error InvalidVpcEndpointId.Malformed: Invalid Id: '' (expecting 'vpce-...; the Id may only contain lowercase alphanumeric characters and a single dash')
2019-09-06T10:14:13.988+0200 [DEBUG] plugin.terraform-provider-aws_v2.27.0_x4: 	status code: 400, request id: f3e7c80c-68ec-433b-9a09-b107a675c08e

[diogomatsubara]

@bflad let us know if you need more information.

[diogomatsubara]

I found a workaround for this issue which will allow us to move forward before upstream can fix the issue.

Once the vpc endpoint is imported into the state, you can manually edit the bbl-terraform.tfstate file and add the vpc_endpoint_route_table_association there. To generate the proper ID, use the following go code (extracted from Terraform core):

https://paste.debian.net/1099848/

Save it to a file crc32.go and build it with go build -o crc32 crc32.go

To generate the ID, ./crc32.go <vpc-endpoint-id> <route-table-id>

Now, just below the vpc_endpoint block in the terraform.tfstate file, add another block for the vpc_endpoint_route_table_association

"aws_vpc_endpoint_route_table_association.internal-subnet-s3": {
                    "type": "aws_vpc_endpoint_route_table_association",
                    "depends_on": [
                        "aws_route_table.internal_route_table",
                        "aws_vpc_endpoint.s3"
                    ],
                    "primary": {
                        "id": "<id-generated-by-crc32.go>",
                        "attributes": {
                            "id": "<id-generated-by-crc32.go>",
                            "route_table_id": "<route-table-id>",
                            "vpc_endpoint_id": "<vpc_endpoint_id>"
                        },
                        "meta": {},
                        "tainted": false
                    },
                    "deposed": [],
                    "provider": "provider.aws"
                },

Save this state file and replace the existing one. Make sure to make a backup of the old state.

Once the terraform state is in the S3 bucket, run terraform plan again to make sure terraform will refresh the state of the vpc_endpoint_route_table_association.

If it shows up in the list of refreshed resources and it doesn’t show up in the list of new resources to be added, then it should be safe to apply.

[ewbankkit]

Yes, I can reproduce with a modified acceptance test:

$ make testacc TEST=./aws/ TESTARGS='-run=TestAccAWSVpcEndpointRouteTableAssociation_'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws/ -v -count 1 -parallel 20 -run=TestAccAWSVpcEndpointRouteTableAssociation_ -timeout 120m
=== RUN   TestAccAWSVpcEndpointRouteTableAssociation_basic
=== PAUSE TestAccAWSVpcEndpointRouteTableAssociation_basic
=== CONT  TestAccAWSVpcEndpointRouteTableAssociation_basic
--- FAIL: TestAccAWSVpcEndpointRouteTableAssociation_basic (50.45s)
    testing.go:569: Step 1 error: InvalidVpcEndpointId.Malformed: Invalid Id: '' (expecting 'vpce-...; the Id may only contain lowercase alphanumeric characters and a single dash')
        	status code: 400, request id: d5b4b5ac-2e19-4338-b188-d8cd0aa9dd3a
FAIL
FAIL	github.com/terraform-providers/terraform-provider-aws/aws	50.566s
FAIL
make: *** [testacc] Error 1

This resource was added with potential support for terraform import (Resource.Importer implemented) but no documentation.
I suggest documenting (and implementing) that the VPC Endpoint ID and Route Table ID must be passed when importing the resource.

[bflad]

The fix for this has been merged and will release with version 2.32.0 of the Terraform AWS Provider, shortly. Thanks to @ewbankkit for the implementation. 👍

[ghost]

This has been released in version 2.32.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!