Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: (logging) fatal error: concurrent map writes (Error: Plugin did not respond) #29236

Closed
jeremychauvet opened this issue Feb 3, 2023 · 11 comments · Fixed by #29269
Closed
Assignees
Labels
bug Addresses a defect in current functionality. crash Results from or addresses a Terraform crash or kernel panic. provider Pertains to the provider itself, rather than any interaction with AWS. regression Pertains to a degraded workflow resulting from an upstream patch or internal enhancement.

Comments

@jeremychauvet
Copy link
Contributor

jeremychauvet commented Feb 3, 2023

Previously titled: "[Bug]: The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ApplyResourceChange call."

Changing the name because at this stage I believe it is misleading since any (?) error may cause this crash.

See #29153

Terraform Core Version

v1.3.7

AWS Provider Version

v4.53.0

Affected Resource(s)

All ressources

Expected Behavior

Create, update or delete resources

Actual Behavior

Crash randomly

Relevant Error/Panic Output Snippet

[...]
module.xxxxx.aws_security_group.lambdas: Still destroying... [id=sg-xxxxx, 10s elapsed]
╷
│ Error: Plugin did not respond
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ApplyResourceChange call. The plugin logs may
│ contain more details.
╵
╷
│ Error: Plugin did not respond
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ApplyResourceChange call. The plugin logs may
│ contain more details.
╵
╷
│ Error: Plugin did not respond
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ApplyResourceChange call. The plugin logs may
│ contain more details.
╵
Stack trace from the terraform-provider-aws_v4.53.0_x5 plugin:
fatal error: concurrent map writes
goroutine 1730 [running]:
github.com/hashicorp/terraform-plugin-log/tflog.SetField.func1(...)
	github.com/hashicorp/terraform-plugin-log@v0.7.0/internal/logging/options.go:212
github.com/hashicorp/terraform-plugin-log/tflog.SetField({0xe25bd00, 0xc001d5f890}, {0xce4900e, 0xb}, {0xa619820?, 0xc00089eb90})
	github.com/hashicorp/terraform-plugin-log@v0.7.0/tflog/provider.go:18 +0x111
github.com/hashicorp/aws-sdk-go-base/v2/awsv1shim/v2.setAWSFields({0xe25bd00?, 0xc003bc12f0?}, 0xc005cbaa00)
	github.com/hashicorp/aws-sdk-go-base/v2/awsv1shim/v2@v2.0.0-beta.23/logger.go:40 +0x9c
github.com/hashicorp/aws-sdk-go-base/v2/awsv1shim/v2.logRequest(0xc005cbaa00)
	github.com/hashicorp/aws-sdk-go-base/v2/awsv1shim/v2@v2.0.0-beta.23/logger.go:69 +0x5f
github.com/aws/aws-sdk-go/aws/request.(*HandlerList).Run(0xc005cbac28, 0xc005cbaa00)
	github.com/aws/aws-sdk-go@v1.44.191/aws/request/handlers.go:267 +0x9a
github.com/aws/aws-sdk-go/aws/request.(*Request).sendRequest(0xc005cbaa00)
	github.com/aws/aws-sdk-go@v1.44.191/aws/request/request.go:602 +0xbf
github.com/aws/aws-sdk-go/aws/request.(*Request).Send(0xc005cbaa00)
	github.com/aws/aws-sdk-go@v1.44.191/aws/request/request.go:555 +0x156
github.com/aws/aws-sdk-go/aws/request.(*Pagination).Next(0xc00501bbf0)
	github.com/aws/aws-sdk-go@v1.44.191/aws/request/request_pagination.go:107 +0xb3
github.com/aws/aws-sdk-go/service/ec2.(*EC2).DescribeNetworkInterfacesPagesWithContext(0xc00423cc40, {0xe25bd00?, 0xc003bc12f0}, 0xc00333e050, 0xc00501bd20, {0x0, 0x0, 0x0})
	github.com/aws/aws-sdk-go@v1.44.191/service/ec2/api.go:26518 +0x19d
github.com/hashicorp/terraform-provider-aws/internal/service/ec2.FindNetworkInterfaces({0xe25bd00?, 0xc003bc12f0?}, 0x48?, 0xc00333e050)
	github.com/hashicorp/terraform-provider-aws/internal/service/ec2/find.go:1526 +0x8b
github.com/hashicorp/terraform-provider-aws/internal/service/ec2.FindNetworkInterface({0xe25bd00?, 0xc003bc12f0?}, 0xc00315b780?, 0xc00333e050)
	github.com/hashicorp/terraform-provider-aws/internal/service/ec2/find.go:1506 +0x2c
github.com/hashicorp/terraform-provider-aws/internal/service/ec2.FindNetworkInterfaceByID({0xe25bd00, 0xc003bc12f0}, 0xc002c92e28?, {0xc006346510, 0x15})
	github.com/hashicorp/terraform-provider-aws/internal/service/ec2/find.go:1559 +0x10c
github.com/hashicorp/terraform-provider-aws/internal/service/ec2.StatusNetworkInterfaceStatus.func1()
	github.com/hashicorp/terraform-provider-aws/internal/service/ec2/status.go:1123 +0x31
github.com/hashicorp/terraform-plugin-sdk/v2/helper/resource.(*StateChangeConf).WaitForStateContext.func1()
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/resource/state.go:110 +0x1ff
created by github.com/hashicorp/terraform-plugin-sdk/v2/helper/resource.(*StateChangeConf).WaitForStateContext
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/resource/state.go:83 +0x1d8
goroutine 1 [select, 1 minutes]:
github.com/hashicorp/go-plugin.Serve(0xc004ad04e0)
	github.com/hashicorp/go-plugin@v1.4.8/server.go:471 +0x14f7
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.Serve({0xcf5e0a9, 0x23}, 0xc0044c5760, {0x0, 0x0, 0x0})
	github.com/hashicorp/terraform-plugin-go@v0.14.3/tfprotov5/tf5server/server.go:312 +0xba5
main.main()
	github.com/hashicorp/terraform-provider-aws/main.go:32 +0x18d
goroutine 321 [select, 1 minutes]:
github.com/hashicorp/go-plugin.(*gRPCBrokerServer).Recv(0xc0000c7fa0?)
	github.com/hashicorp/go-plugin@v1.4.8/grpc_broker.go:121 +0x67
github.com/hashicorp/go-plugin.(*GRPCBroker).Run(0xc004b72140)
	github.com/hashicorp/go-plugin@v1.4.8/grpc_broker.go:411 +0x44
created by github.com/hashicorp/go-plugin.(*GRPCServer).Init
	github.com/hashicorp/go-plugin@v1.4.8/grpc_server.go:85 +0x46a
goroutine 322 [IO wait, 1 minutes]:
internal/poll.runtime_pollWait(0x7f1cfababed0, 0x72)
	runtime/netpoll.go:305 +0x89
internal/poll.(*pollDesc).wait(0xc004ad09c0?, 0xc000a3e000?, 0x1)
	internal/poll/fd_poll_runtime.go:84 +0x32
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc004ad09c0, {0xc000a3e000, 0x1000, 0x1000})
	internal/poll/fd_unix.go:167 +0x25a
os.(*File).read(...)
	os/file_posix.go:31
os.(*File).Read(0xc004a3a820, {0xc000a3e000?, 0x400?, 0xa6ff020?})
	os/file.go:118 +0x5e
bufio.(*Reader).Read(0xc0000c0f40, {0xc000a40000, 0x400, 0x0?})
	bufio/bufio.go:237 +0x1bb
github.com/hashicorp/go-plugin.copyChan({0xe284db0, 0xc0045f9170}, 0x0?, {0xe19eae0?, 0xc004a3a820?})
	github.com/hashicorp/go-plugin@v1.4.8/grpc_stdio.go:181 +0x1f6
created by github.com/hashicorp/go-plugin.newGRPCStdioServer
	github.com/hashicorp/go-plugin@v1.4.8/grpc_stdio.go:37 +0xf5
goroutine 323 [IO wait, 1 minutes]:
internal/poll.runtime_pollWait(0x7f1cfababcf0, 0x72)
	runtime/netpoll.go:305 +0x89
internal/poll.(*pollDesc).wait(0xc004ad0a80?, 0xc00134a000?, 0x1)
	internal/poll/fd_poll_runtime.go:84 +0x32
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc004ad0a80, {0xc00134a000, 0x1000, 0x1000})
	internal/poll/fd_unix.go:167 +0x25a
os.(*File).read(...)
	os/file_posix.go:31
os.(*File).Read(0xc004a3a830, {0xc00134a000?, 0x400?, 0xa6ff020?})
	os/file.go:118 +0x5e
bufio.(*Reader).Read(0xc0000c1740, {0xc00134c000, 0x400, 0x0?})
	bufio/bufio.go:237 +0x1bb
github.com/hashicorp/go-plugin.copyChan({0xe284db0, 0xc0045f9170}, 0x0?, {0xe19eae0?, 0xc004a3a830?})
	github.com/hashicorp/go-plugin@v1.4.8/grpc_stdio.go:181 +0x1f6
created by github.com/hashicorp/go-plugin.newGRPCStdioServer
	github.com/hashicorp/go-plugin@v1.4.8/grpc_stdio.go:38 +0x185
goroutine 337 [syscall, 1 minutes]:
os/signal.signal_recv()
	runtime/sigqueue.go:152 +0x2f
os/signal.loop()
	os/signal/signal_unix.go:23 +0x19
created by os/signal.Notify.func1.1
	os/signal/signal.go:151 +0x2a
goroutine 338 [chan receive, 1 minutes]:
github.com/hashicorp/go-plugin.Serve.func3()
	github.com/hashicorp/go-plugin@v1.4.8/server.go:436 +0x75
Error: The terraform-provider-aws_v4.53.0_x5 plugin crashed!
This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.
Cleaning up project directory and file based variables
00:01
ERROR: Job failed: exit code 1

Terraform Configuration Files

N/A

Steps to Reproduce

  1. Update a Terraform stack to v4.53.0 of the provider:
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "4.53.0"
    }
  }

  required_version = ">= 1.2"
}
  1. Terraform plan / apply / destroy

Debug Output

No response

Panic Output

Stack trace from the terraform-provider-aws_v4.53.0_x5 plugin:
fatal error: concurrent map writes
goroutine 1730 [running]:
github.com/hashicorp/terraform-plugin-log/tflog.SetField.func1(...)
	github.com/hashicorp/terraform-plugin-log@v0.7.0/internal/logging/options.go:212
github.com/hashicorp/terraform-plugin-log/tflog.SetField({0xe25bd00, 0xc001d5f890}, {0xce4900e, 0xb}, {0xa619820?, 0xc00089eb90})
	github.com/hashicorp/terraform-plugin-log@v0.7.0/tflog/provider.go:18 +0x111
github.com/hashicorp/aws-sdk-go-base/v2/awsv1shim/v2.setAWSFields({0xe25bd00?, 0xc003bc12f0?}, 0xc005cbaa00)
	github.com/hashicorp/aws-sdk-go-base/v2/awsv1shim/v2@v2.0.0-beta.23/logger.go:40 +0x9c
github.com/hashicorp/aws-sdk-go-base/v2/awsv1shim/v2.logRequest(0xc005cbaa00)
	github.com/hashicorp/aws-sdk-go-base/v2/awsv1shim/v2@v2.0.0-beta.23/logger.go:69 +0x5f
github.com/aws/aws-sdk-go/aws/request.(*HandlerList).Run(0xc005cbac28, 0xc005cbaa00)
	github.com/aws/aws-sdk-go@v1.44.191/aws/request/handlers.go:267 +0x9a
github.com/aws/aws-sdk-go/aws/request.(*Request).sendRequest(0xc005cbaa00)
	github.com/aws/aws-sdk-go@v1.44.191/aws/request/request.go:[602](https://gitlab.eulerhermes.com/deployment/datahub/business-events/eventgen-grade-record/-/jobs/12258660#L602) +0xbf
github.com/aws/aws-sdk-go/aws/request.(*Request).Send(0xc005cbaa00)
	github.com/aws/aws-sdk-go@v1.44.191/aws/request/request.go:555 +0x156
github.com/aws/aws-sdk-go/aws/request.(*Pagination).Next(0xc00501bbf0)
	github.com/aws/aws-sdk-go@v1.44.191/aws/request/request_pagination.go:107 +0xb3
github.com/aws/aws-sdk-go/service/ec2.(*EC2).DescribeNetworkInterfacesPagesWithContext(0xc00423cc40, {0xe25bd00?, 0xc003bc12f0}, 0xc00333e050, 0xc00501bd20, {0x0, 0x0, 0x0})
	github.com/aws/aws-sdk-go@v1.44.191/service/ec2/api.go:26518 +0x19d
github.com/hashicorp/terraform-provider-aws/internal/service/ec2.FindNetworkInterfaces({0xe25bd00?, 0xc003bc12f0?}, 0x48?, 0xc00333e050)
	github.com/hashicorp/terraform-provider-aws/internal/service/ec2/find.go:1526 +0x8b
github.com/hashicorp/terraform-provider-aws/internal/service/ec2.FindNetworkInterface({0xe25bd00?, 0xc003bc12f0?}, 0xc00315b780?, 0xc00333e050)
	github.com/hashicorp/terraform-provider-aws/internal/service/ec2/find.go:1506 +0x2c
github.com/hashicorp/terraform-provider-aws/internal/service/ec2.FindNetworkInterfaceByID({0xe25bd00, 0xc003bc12f0}, 0xc002c92e28?, {0xc006346510, 0x15})
	github.com/hashicorp/terraform-provider-aws/internal/service/ec2/find.go:1559 +0x10c
github.com/hashicorp/terraform-provider-aws/internal/service/ec2.StatusNetworkInterfaceStatus.func1()
	github.com/hashicorp/terraform-provider-aws/internal/service/ec2/status.go:1123 +0x31
github.com/hashicorp/terraform-plugin-sdk/v2/helper/resource.(*StateChangeConf).WaitForStateContext.func1()
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/resource/state.go:110 +0x1ff
created by github.com/hashicorp/terraform-plugin-sdk/v2/helper/resource.(*StateChangeConf).WaitForStateContext
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/resource/state.go:83 +0x1d8
goroutine 1 [select, 1 minutes]:
github.com/hashicorp/go-plugin.Serve(0xc004ad04e0)
	github.com/hashicorp/go-plugin@v1.4.8/server.go:471 +0x14f7
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.Serve({0xcf5e0a9, 0x23}, 0xc0044c5760, {0x0, 0x0, 0x0})
	github.com/hashicorp/terraform-plugin-go@v0.14.3/tfprotov5/tf5server/server.go:312 +0xba5
main.main()
	github.com/hashicorp/terraform-provider-aws/main.go:32 +0x18d
goroutine 321 [select, 1 minutes]:
github.com/hashicorp/go-plugin.(*gRPCBrokerServer).Recv(0xc0000c7fa0?)
	github.com/hashicorp/go-plugin@v1.4.8/grpc_broker.go:121 +0x67
github.com/hashicorp/go-plugin.(*GRPCBroker).Run(0xc004b72140)
	github.com/hashicorp/go-plugin@v1.4.8/grpc_broker.go:411 +0x44
created by github.com/hashicorp/go-plugin.(*GRPCServer).Init
	github.com/hashicorp/go-plugin@v1.4.8/grpc_server.go:85 +0x46a
goroutine 322 [IO wait, 1 minutes]:
internal/poll.runtime_pollWait(0x7f1cfababed0, 0x72)
	runtime/netpoll.go:305 +0x89
internal/poll.(*pollDesc).wait(0xc004ad09c0?, 0xc000a3e000?, 0x1)
	internal/poll/fd_poll_runtime.go:84 +0x32
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc004ad09c0, {0xc000a3e000, 0x1000, 0x1000})
	internal/poll/fd_unix.go:167 +0x25a
os.(*File).read(...)
	os/file_posix.go:31
os.(*File).Read(0xc004a3a820, {0xc000a3e000?, 0x400?, 0xa6ff020?})
	os/file.go:118 +0x5e
bufio.(*Reader).Read(0xc0000c0f40, {0xc000a40000, 0x400, 0x0?})
	bufio/bufio.go:237 +0x1bb
github.com/hashicorp/go-plugin.copyChan({0xe284db0, 0xc0045f9170}, 0x0?, {0xe19eae0?, 0xc004a3a820?})
	github.com/hashicorp/go-plugin@v1.4.8/grpc_stdio.go:181 +0x1f6
created by github.com/hashicorp/go-plugin.newGRPCStdioServer
	github.com/hashicorp/go-plugin@v1.4.8/grpc_stdio.go:37 +0xf5
goroutine 323 [IO wait, 1 minutes]:
internal/poll.runtime_pollWait(0x7f1cfababcf0, 0x72)
	runtime/netpoll.go:305 +0x89
internal/poll.(*pollDesc).wait(0xc004ad0a80?, 0xc00134a000?, 0x1)
	internal/poll/fd_poll_runtime.go:84 +0x32
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc004ad0a80, {0xc00134a000, 0x1000, 0x1000})
	internal/poll/fd_unix.go:167 +0x25a
os.(*File).read(...)
	os/file_posix.go:31
os.(*File).Read(0xc004a3a830, {0xc00134a000?, 0x400?, 0xa6ff020?})
	os/file.go:118 +0x5e
bufio.(*Reader).Read(0xc0000c1740, {0xc00134c000, 0x400, 0x0?})
	bufio/bufio.go:237 +0x1bb
github.com/hashicorp/go-plugin.copyChan({0xe284db0, 0xc0045f9170}, 0x0?, {0xe19eae0?, 0xc004a3a830?})
	github.com/hashicorp/go-plugin@v1.4.8/grpc_stdio.go:181 +0x1f6
created by github.com/hashicorp/go-plugin.newGRPCStdioServer
	github.com/hashicorp/go-plugin@v1.4.8/grpc_stdio.go:38 +0x185
goroutine 337 [syscall, 1 minutes]:
os/signal.signal_recv()
	runtime/sigqueue.go:152 +0x2f
os/signal.loop()
	os/signal/signal_unix.go:23 +0x19
created by os/signal.Notify.func1.1
	os/signal/signal.go:151 +0x2a
goroutine 338 [chan receive, 1 minutes]:
github.com/hashicorp/go-plugin.Serve.func3()
	github.com/hashicorp/go-plugin@v1.4.8/server.go:436 +0x75
Error: The terraform-provider-aws_v4.53.0_x5 plugin crashed!
This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

Important Factoids

Reverting in v4.52.0 is a workaround ✅

References

Would you like to implement a fix?

None

@jeremychauvet jeremychauvet added bug Addresses a defect in current functionality. needs-triage Waiting for first response or review from a maintainer. labels Feb 3, 2023
@github-actions
Copy link

github-actions bot commented Feb 3, 2023

Community Note

Voting for Prioritization

  • Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
  • Please see our prioritization guide for information on how we prioritize.
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

  • If you are interested in working on this issue, please leave a comment.
  • If this would be your first contribution, please review the contribution guide.

@ewbankkit ewbankkit added crash Results from or addresses a Terraform crash or kernel panic. provider Pertains to the provider itself, rather than any interaction with AWS. and removed needs-triage Waiting for first response or review from a maintainer. labels Feb 3, 2023
@ewbankkit
Copy link
Contributor

Similar: #29153.

@ewbankkit ewbankkit added upstream Addresses functionality related to the cloud provider. upstream-terraform Addresses functionality related to the Terraform core binary. and removed upstream Addresses functionality related to the cloud provider. upstream-terraform Addresses functionality related to the Terraform core binary. labels Feb 3, 2023
@ewbankkit
Copy link
Contributor

terraform-plugin-log issue: hashicorp/terraform-plugin-log#126.

@gdavison gdavison self-assigned this Feb 6, 2023
@YakDriver
Copy link
Member

YakDriver commented Feb 6, 2023

I suspect that any* error the AWS Provider returns will cause Terraform/SDK to crash in a similar way. This only occurs as of v4.53.0. The only change in v4.53.0 was #29223 (logging). (* Random golang errors do not seem to cause this. Perhaps only AWS errors?)

I can reproduce the issue in 100% 90% of situations where the plugin returns an error. Whether you get fatal error: out of memory or fatal error: concurrent map writes seems to depend largely on whether the goroutines trip up first or memory runs out first. These may not be useful clues.

In case it is helpful, here is a complete (~4100 line) trace: https://gist.github.com/YakDriver/66b508dab3342da50b2da75f4b17a73a

@YakDriver YakDriver added the regression Pertains to a degraded workflow resulting from an upstream patch or internal enhancement. label Feb 6, 2023
@YakDriver YakDriver changed the title [Bug]: The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ApplyResourceChange call. [Bug]: (logging) fatal error: concurrent map writes Feb 6, 2023
@YakDriver YakDriver changed the title [Bug]: (logging) fatal error: concurrent map writes [Bug]: (logging) fatal error: concurrent map writes Feb 6, 2023
@YakDriver YakDriver changed the title [Bug]: (logging) fatal error: concurrent map writes [Bug]: (logging) fatal error: concurrent map writes (Error: Plugin did not respond) Feb 6, 2023
@ascopes
Copy link

ascopes commented Feb 8, 2023

Can confirm the same issue, and it is when destroying security groups on Terraform 0.12.31, AWS provider 4.53.0.

Looks like the tf-plugin-log lib had a new release in the past hour to v0.8.0, which includes a bugfix that "tflog+tflogsdk: Prevented data race conditions when using SetField and other option functions (hashicorp/terraform-plugin-log#132)"

Given SetField is the first attribute in the stacktraces, could this be related?

Edit: this has just failed on another module (we have several dozen of these) and the place it has failed at is again on destroying a security group. Doesn't seem to be failing on anything else for me.

Edit 2: can also confirm 4.52.0 does not have this issue.

@ascopes
Copy link

ascopes commented Feb 8, 2023

Based on @bflad's comment at hashicorp/terraform-plugin-log#126 (comment), assuming this is indeed related to that fix, in which case bumping that version from 0.7.0 to 0.8.0 will assumably fix this?

@chrisba11
Copy link

I am also running into this. It's happening primarily when we are running a destroy workflow to clean up feature stack resources. The error occurs while waiting for our RDS cluster to be destroyed. If I re-run the workflow job again right away, it continues to error. Since the RDS database takes about 10-15 minutes to destroy, I've found that if I come back and re-run the workflow after I know the database is destroyed, the error goes away.

@bradhandy
Copy link

bradhandy commented Feb 9, 2023

I've experienced a similar situation with destroying Lambdas and their associated security groups. Normally, the Terraform run will wait for the NICs attached to the Lambdas to be destroyed then destroy the security group. The plugin now fails with the concurrent map writes error when attempting to destroy the security group.

I reverted back to the version I used prior to upgrading which was 4.31.0. It's good to know 4.52.0 is a viable option to get around this issue.

@Agony1023
Copy link

Just ran into the same issue today while doing a tf destroy.
Error: The terraform-provider-aws_v4.53.0_x5 plugin crashed!

Added a terraform.tf to the project:

    aws = {
      source  = "hashicorp/aws"
      version = ">= 4.52.0, != 4.53.0"
    }

Skipping 4.53.0 for now.

@YakDriver
Copy link
Member

This required an upstream fix (hashicorp/terraform-plugin-log#132) which will be part of v4.54.0. We are very sorry for the inconvenience! (Which also greatly impacted us internally 😞 .) Fixed with merged new version of terraform-plugin-log in #29269.

@github-actions
Copy link

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 12, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Addresses a defect in current functionality. crash Results from or addresses a Terraform crash or kernel panic. provider Pertains to the provider itself, rather than any interaction with AWS. regression Pertains to a degraded workflow resulting from an upstream patch or internal enhancement.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants