[Bug] Plugin crashes with GRPCProvider error #1215
Description
Somehow our terraform state has gotten into a place where the elasticstack plugin crashes on any plan, apply, or even refresh. We've tried downgrading all the way to 0.9 and back up again to the latest, and we still get the same results. We were running terraform 1.1.7, and tried upgrading to 1.12.1, and this does not appear to have any effect.
We're seeing this issue from our local environments on OSX, as well as our build environments in Linux.
Elasticsearch version is 8.17.3.
Here are the logs from refreshing the state, and then showing where things turn south:
elasticstack_elasticsearch_script.ncctrllatency_controller_latency_v8_tmpl: Refreshing state... [id=2s5YNcSZQ2OAYbUhM9YJkQ/ncctrllatency_controller_latency_v8_tmpl]
elasticstack_elasticsearch_script.ncentityevent_all_events_tmpl: Refreshing state... [id=2s5YNcSZQ2OAYbUhM9YJkQ/ncentityevent_all_events_tmpl]
elasticstack_elasticsearch_script.ncentityevent_dashboard_events_tmpl: Refreshing state... [id=2s5YNcSZQ2OAYbUhM9YJkQ/ncentityevent_dashboard_events_tmpl]
elasticstack_elasticsearch_script.nf_component_usage_tmpl: Refreshing state... [id=2s5YNcSZQ2OAYbUhM9YJkQ/nf_component_usage_tmpl]
2025-07-24T08:43:33.850-0400 [WARN] provider.terraform-provider-elasticstack_v0.11.17: Script "nf_component_usage_tmpl" not found, removing from state: tf_rpc=ReadResource tf_mux_provider=tf5to6server.v5tov6Server tf_resource_type=elasticstack_elasticsearch_script @caller=github.com/elastic/terraform-provider-elasticstack/internal/elasticsearch/cluster/script.go:82 @module=elasticstack tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_req_id=02762957-2c5f-6d0b-79d9-b9f1ff71cbc0 timestamp=2025-07-24T08:43:33.850-0400
2025-07-24T08:43:33.855-0400 [ERROR] provider: plugin process exited: plugin=.terraform/providers/registry.terraform.io/elastic/elasticstack/0.11.17/darwin_amd64/terraform-provider-elasticstack_v0.11.17 id=26812 error="exit status 2"
2025-07-24T08:43:33.855-0400 [ERROR] plugin6.(*GRPCProvider).ReadResource: error="rpc error: code = Canceled desc = context canceled"
2025-07-24T08:43:33.855-0400 [ERROR] vertex "elasticstack_elasticsearch_script.nf_component_usage_tmpl" error: Request cancelled
2025-07-24T08:43:33.855-0400 [ERROR] vertex "elasticstack_elasticsearch_script.nf_component_usage_tmpl (expand)" error: Request cancelled
2025-07-24T08:43:33.855-0400 [ERROR] plugin6.(*GRPCProvider).UpgradeResourceState: error="rpc error: code = Canceled desc = context canceled"
Lots of repeat errors from here....
Lots of "request was cancelled" errors. Trimmed down for brevity.
╷
│ Error: Request cancelled
│
│ The plugin6.(*GRPCProvider).ReadResource request was cancelled.
╵
╷
│ Error: Request cancelled
│
│ The plugin6.(*GRPCProvider).UpgradeResourceState request was cancelled.
╵
Stack trace from the terraform-provider-elasticstack_v0.11.17 plugin:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x8d521b6]
goroutine 301 [running]:
github.com/elastic/terraform-provider-elasticstack/internal/elasticsearch/cluster.resourceScriptRead({0x9510350, 0xc00023c770}, 0xc000001830, {0x948c120?, 0xc00063c180?})
github.com/elastic/terraform-provider-elasticstack/internal/elasticsearch/cluster/script.go:92 +0x316
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).read(0xc000373a00, {0x95102a8, 0xc000790e70}, 0xc000001830, {0x948c120, 0xc00063c180})
github.com/hashicorp/terraform-plugin-sdk/v2@v2.37.0/helper/schema/resource.go:866 +0x119
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).RefreshWithoutUpgrade(0xc000373a00, {0x95102a8, 0xc000790e70}, 0xc0000c8820, {0x948c120, 0xc00063c180})
github.com/hashicorp/terraform-plugin-sdk/v2@v2.37.0/helper/schema/resource.go:1162 +0x52a
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ReadResource(0xc000173740, {0x95102a8?, 0xc000790d50?}, 0xc0005da190)
github.com/hashicorp/terraform-plugin-sdk/v2@v2.37.0/helper/schema/grpc_provider.go:908 +0xb6d
github.com/hashicorp/terraform-plugin-mux/tf5to6server.v5tov6Server.ReadResource({{0x95213d8?, 0xc000173740?}}, {0x95102a8, 0xc000790d50}, 0x0?)
github.com/hashicorp/terraform-plugin-mux@v0.20.0/tf5to6server/tf5to6server.go:187 +0x57
github.com/hashicorp/terraform-plugin-mux/tf6muxserver.(*muxServer).ReadResource(0xc000308900, {0x95102a8?, 0xc000790a80?}, 0xc0005da140)
github.com/hashicorp/terraform-plugin-mux@v0.20.0/tf6muxserver/mux_server_ReadResource.go:35 +0x190
github.com/hashicorp/terraform-plugin-go/tfprotov6/tf6server.(*server).ReadResource(0xc0002c77c0, {0x95102a8?, 0xc0007901e0?}, 0xc00023c070)
github.com/hashicorp/terraform-plugin-go@v0.28.0/tfprotov6/tf6server/server.go:860 +0x2f3
github.com/hashicorp/terraform-plugin-go/tfprotov6/internal/tfplugin6._Provider_ReadResource_Handler({0x94b8d00, 0xc0002c77c0}, {0x95102a8, 0xc0007901e0}, 0xc0005b4380, 0x0)
github.com/hashicorp/terraform-plugin-go@v0.28.0/tfprotov6/internal/tfplugin6/tfplugin6_grpc.pb.go:651 +0x1a9
google.golang.org/grpc.(*Server).processUnaryRPC(0xc000298a00, {0x95102a8, 0xc000790150}, 0xc000134180, 0xc000450a80, 0xa2eef38, 0x0)
google.golang.org/grpc@v1.72.1/server.go:1405 +0x1036
google.golang.org/grpc.(*Server).handleStream(0xc000298a00, {0x95110b0, 0xc000380680}, 0xc000134180)
google.golang.org/grpc@v1.72.1/server.go:1815 +0xb88
google.golang.org/grpc.(*Server).serveStreams.func2.1()
google.golang.org/grpc@v1.72.1/server.go:1035 +0x7f
created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 42
google.golang.org/grpc@v1.72.1/server.go:1046 +0x11d
Error: The terraform-provider-elasticstack_v0.11.17 plugin crashed!
This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.
Let me know what additional information I can provide.