Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Terraform segfaults during apply #29707

Closed
jbg opened this issue Oct 6, 2021 · 4 comments
Closed

Terraform segfaults during apply #29707

jbg opened this issue Oct 6, 2021 · 4 comments
Assignees
Labels
bug core crash waiting for reproduction unable to reproduce issue without further information waiting-response An issue/pull request is waiting for a response from the community

Comments

@jbg
Copy link

jbg commented Oct 6, 2021

Terraform Version

Terraform v1.0.8
on linux_arm64
+ provider registry.terraform.io/chilicat/pkcs12 v0.0.7
+ provider registry.terraform.io/cyrilgdn/postgresql v1.14.0
+ provider registry.terraform.io/gavinbunney/kubectl v1.11.3
+ provider registry.terraform.io/grafana/grafana v1.14.0
+ provider registry.terraform.io/hashicorp/aws v3.61.0
+ provider registry.terraform.io/hashicorp/google v3.87.0
+ provider registry.terraform.io/hashicorp/http v2.1.0
+ provider registry.terraform.io/hashicorp/kubernetes v2.5.0
+ provider registry.terraform.io/hashicorp/random v3.1.0
+ provider registry.terraform.io/hashicorp/tls v3.1.0
+ provider registry.terraform.io/integrations/github v4.16.0
+ provider registry.terraform.io/linode/linode v1.21.0
+ provider registry.terraform.io/vancluever/acme v2.5.3

Crash Output

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0xb8 pc=0xaaaae9b17d18]

goroutine 25658 [running]:
github.com/hashicorp/terraform/internal/plans.(*ResourceInstanceChange).Simplify(0x0, 0x1)
	/home/buildozer/aports/community/terraform/src/src/github.com/hashicorp/terraform/internal/plans/changes.go:236 +0x28
github.com/hashicorp/terraform/internal/terraform.reducePlan({{}, {{}, 0x4d, {0x400173fde0, 0xb}, {0x400173fdf0, 0x4}}, {0xaaaaec63b618, 0x401389af90}}, 0x0, ...)
	/home/buildozer/aports/community/terraform/src/src/github.com/hashicorp/terraform/internal/terraform/reduce_plan.go:23 +0x30
github.com/hashicorp/terraform/internal/terraform.(*NodeDestroyResourceInstance).managedResourceExecute(0x400cde3e48, {0xaaaaec6b6040, 0x401d9abb20})
	/home/buildozer/aports/community/terraform/src/src/github.com/hashicorp/terraform/internal/terraform/node_resource_destroy.go:173 +0x2a8
github.com/hashicorp/terraform/internal/terraform.(*NodeDestroyResourceInstance).Execute(0x400cde3e48, {0xaaaaec6b6040, 0x401d9abb20}, 0x1)
	/home/buildozer/aports/community/terraform/src/src/github.com/hashicorp/terraform/internal/terraform/node_resource_destroy.go:140 +0x94
github.com/hashicorp/terraform/internal/terraform.(*ContextGraphWalker).Execute(0x401443dad0, {0xaaaaec6b6040, 0x401d9abb20}, {0xffff5a747b40, 0x400cde3e48})
	/home/buildozer/aports/community/terraform/src/src/github.com/hashicorp/terraform/internal/terraform/graph_walk_context.go:129 +0x9c
github.com/hashicorp/terraform/internal/terraform.(*Graph).walk.func1({0xaaaaec5ace20, 0x400cde3e48})
	/home/buildozer/aports/community/terraform/src/src/github.com/hashicorp/terraform/internal/terraform/graph.go:59 +0x224
github.com/hashicorp/terraform/internal/dag.(*Walker).walkVertex(0x4009190960, {0xaaaaec5ace20, 0x400cde3e48}, 0x401c3dd700)
	/home/buildozer/aports/community/terraform/src/src/github.com/hashicorp/terraform/internal/dag/walk.go:381 +0x318
created by github.com/hashicorp/terraform/internal/dag.(*Walker).Update
	/home/buildozer/aports/community/terraform/src/src/github.com/hashicorp/terraform/internal/dag/walk.go:304 +0xe34

Steps to Reproduce

This crash has been happening to us intermittently, a few times a day in our CI, during the last few 1.0.x versions. I occasionally reproduce it on my development laptop (darwin_arm64) too.

Running force-unlock (because it leaves the state locked when it crashes) and then apply again with no changes always works.

There does not seem to be any consistency about which provider it's applying changes using when it crashes.

The stacktrace is very similar to the one reported in #26346.

@jbg jbg added bug new new issue not yet triaged labels Oct 6, 2021
@jbardin jbardin added crash and removed new new issue not yet triaged labels Oct 6, 2021
@jbardin jbardin self-assigned this Oct 6, 2021
@jbardin jbardin added the core label Oct 6, 2021
@jbardin
Copy link
Member

jbardin commented Oct 6, 2021

Thanks @jbg,

I can see the logic error leading to this panic, so the fix is relatively straightforward; however I would really like to understand how the situation arises in the first place.

Any chance you could share a trace log from the crash? Using TF_LOG_CORE=trace will skip all the provider logs, and reduce the amount of data that may need to be redacted if any.

Thanks!

@jbardin jbardin added waiting for reproduction unable to reproduce issue without further information waiting-response An issue/pull request is waiting for a response from the community labels Oct 12, 2021
@jbardin
Copy link
Member

jbardin commented Oct 28, 2021

Since we have not heard back, I'm going to close this issue. If you do have any more info to help diagnose the root cause, feel free to reply here.

Thanks!

@jbardin jbardin closed this as completed Oct 28, 2021
@jbg
Copy link
Author

jbg commented Oct 29, 2021

The issue hasn't recurred recently, and it doesn't appear to have stopped at the same time as we updated from TF 1.0.8 -> 1.0.9, so it might be that a provider was triggering the issue and has since been updated. I'll update here including trace logs if it happens again.

@github-actions
Copy link

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 29, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug core crash waiting for reproduction unable to reproduce issue without further information waiting-response An issue/pull request is waiting for a response from the community
Projects
None yet
Development

No branches or pull requests

2 participants