Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix segv in sagemaker app resource #30786

Merged
merged 4 commits into from
Apr 18, 2023
Merged

Conversation

dod38fr
Copy link

@dod38fr dod38fr commented Apr 18, 2023

Description

This PR fixes a segv occuring with sagemaker when:

  • sagemakers apps are created by terraform
  • all apps are deleted with AWS console or CLI

At this stage, terraform plan fails with the following message:

2023-04-17T09:49:04.048+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5: HTTP Response Received: aws.operation=ListApps http.duration=113 @caller=github.com/hashicorp/aws-sdk-go-base/v2/awsv1shim/v2@v2.0.0-beta.26/logger.go:138 http.response.header.content_type=application/x-amz-json-1.1 http.status_code=200 tf_provider_addr=registry.terraform.io/hashicorp/aws tf_resource_type=aws_sagemaker_app @module=aws aws.sdk=aws-sdk-go aws.service=SageMaker http.response.header.date="Mon, 17 Apr 2023 07:49:03 GMT" tf_rpc=ReadResource http.response_content_length=11 tf_mux_provider=*schema.GRPCProviderServer tf_req_id=e79234a1-0bf3-d710-c3b0-b9891faacd4c aws.region=eu-west-1 http.response.body={"Apps":[]} http.response.header.x_amzn_requestid=6b73448b-5aa7-4d13-a2f3-13d74458a4b8 timestamp=2023-04-17T09:49:04.047+0200
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5: panic: runtime error: invalid memory address or nil pointer dereference
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5: [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0xa0718ba]
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5: 
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5: goroutine 1803 [running]:
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5: github.com/hashicorp/terraform-provider-aws/internal/service/sagemaker.FindAppByName({0xeabbd40, 0xc0046db380}, 0x203001?, {0xc005bd752d, 0xe}, {0xc005bd753c?, 0xc0046db350?}, {0xd69ad04, 0xd}, {0xc005bd7554, ...})
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5:        github.com/hashicorp/terraform-provider-aws/internal/service/sagemaker/find.go:336 +0x1da
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5: github.com/hashicorp/terraform-provider-aws/internal/service/sagemaker.resourceAppRead({0xeabbd40, 0xc0046db380}, 0xc004a8b780, {0xd635200?, 0xc0004aa800?})
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5:        github.com/hashicorp/terraform-provider-aws/internal/service/sagemaker/app.go:167 +0x390
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5: github.com/hashicorp/terraform-provider-aws/internal/provider.interceptedHandler[...].func1(0x0?, {0xd635200?, 0xc0004aa800?})
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5:        github.com/hashicorp/terraform-provider-aws/internal/provider/intercept.go:96 +0x175
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5: github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).read(0xeabbd40?, {0xeabbd40?, 0xc0046f0ae0?}, 0xd?, {0xd635200?, 0xc0004aa800?})
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5:        github.com/hashicorp/terraform-plugin-sdk/v2@v2.26.1/helper/schema/resource.go:719 +0x87
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5: github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).RefreshWithoutUpgrade(0xc0018acee0, {0xeabbd40, 0xc0046f0ae0}, 0xc003091930, {0xd635200, 0xc0004aa800})
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5:        github.com/hashicorp/terraform-plugin-sdk/v2@v2.26.1/helper/schema/resource.go:1015 +0x585
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5: github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ReadResource(0xc001a95308, {0xeabbd40?, 0xc0046f0840?}, 0xc004659480)
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5:        github.com/hashicorp/terraform-plugin-sdk/v2@v2.26.1/helper/schema/grpc_provider.go:613 +0x4a5
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5: github.com/hashicorp/terraform-plugin-mux/tf5muxserver.muxServer.ReadResource({0xc0030810e0, 0xc003081140, {0xc003676560, 0x2, 0x2}, {0x0, 0x0, 0x0}, {0x0, 0x0, ...}, ...}, ...)
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5:        github.com/hashicorp/terraform-plugin-mux@v0.9.0/tf5muxserver/mux_server_ReadResource.go:26 +0x102
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5: github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ReadResource(0xc002ce1e00, {0xeabbd40?, 0xc0046cec60?}, 0xc005fe0ae0)
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5:        github.com/hashicorp/terraform-plugin-go@v0.14.3/tfprotov5/tf5server/server.go:748 +0x4b1
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5: github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadResource_Handler({0xd32b6e0?, 0xc002ce1e00}, {0xeabbd40, 0xc0046cec60}, 0xc001cdf8f0, 0x0)
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5:        github.com/hashicorp/terraform-plugin-go@v0.14.3/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:349 +0x170
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5: google.golang.org/grpc.(*Server).processUnaryRPC(0xc003e64000, {0xeacbc40, 0xc0034cc000}, 0xc005a7dd40, 0xc0030d35c0, 0x154e0130, 0x0)
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5:        google.golang.org/grpc@v1.53.0/server.go:1336 +0xd23
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5: google.golang.org/grpc.(*Server).handleStream(0xc003e64000, {0xeacbc40, 0xc0034cc000}, 0xc005a7dd40, 0x0)
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5:        google.golang.org/grpc@v1.53.0/server.go:1704 +0xa2f
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5: google.golang.org/grpc.(*Server).serveStreams.func1.2()
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5:        google.golang.org/grpc@v1.53.0/server.go:965 +0x98
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5: created by google.golang.org/grpc.(*Server).serveStreams.func1
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5:        google.golang.org/grpc@v1.53.0/server.go:963 +0x28a
2023-04-17T09:49:04.061+0200 [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/hashicorp/aws/4.63.0/linux_amd64/terraform-provider-aws_v4.63.0_x5 pid=8029 error="exit status 2"
2023-04-17T09:49:04.062+0200 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF"

With this patch, terraform plan works as expected.

Relations

None

Output from Acceptance Testing

Sorry, I cannot run the acceptance tests in my client's environment.

This PR fixes a segv occuring with sagemaker when:
- sagemakers apps are created by terraform
- all apps are deleted with AWS console or CLI

At this stage, terraform plan fails with the following message:
```
2023-04-17T09:49:04.048+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5: HTTP Response Received: aws.operation=ListApps http.duration=113 @caller=github.com/hashicorp/aws-sdk-go-base/v2/awsv1shim/v2@v2.0.0-beta.26/logger.go:138 http.response.header.content_type=application/x-amz-json-1.1 http.status_code=200 tf_provider_addr=registry.terraform.io/hashicorp/aws tf_resource_type=aws_sagemaker_app @module=aws aws.sdk=aws-sdk-go aws.service=SageMaker http.response.header.date="Mon, 17 Apr 2023 07:49:03 GMT" tf_rpc=ReadResource http.response_content_length=11 tf_mux_provider=*schema.GRPCProviderServer tf_req_id=e79234a1-0bf3-d710-c3b0-b9891faacd4c aws.region=eu-west-1 http.response.body={"Apps":[]} http.response.header.x_amzn_requestid=6b73448b-5aa7-4d13-a2f3-13d74458a4b8 timestamp=2023-04-17T09:49:04.047+0200
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5: panic: runtime error: invalid memory address or nil pointer dereference
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5: [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0xa0718ba]
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5:
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5: goroutine 1803 [running]:
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5: github.com/hashicorp/terraform-provider-aws/internal/service/sagemaker.FindAppByName({0xeabbd40, 0xc0046db380}, 0x203001?, {0xc005bd752d, 0xe}, {0xc005bd753c?, 0xc0046db350?}, {0xd69ad04, 0xd}, {0xc005bd7554, ...})
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5:        github.com/hashicorp/terraform-provider-aws/internal/service/sagemaker/find.go:336 +0x1da
2023-04-17T09:49:04.052+0200 [DEBUG] provider.terraform-provider-aws_v4.63.0_x5: github.com/hashicorp/terraform-provider-aws/internal/service/sagemaker.resourceAppRead({0xeabbd40, 0xc0046db380}, 0xc004a8b780, {0xd635200?, 0xc0004aa800?})
```

With this patch, terraform plan works as expected.
@github-actions
Copy link

Community Note

Voting for Prioritization

  • Please vote on this pull request by adding a 👍 reaction to the original post to help the community and maintainers prioritize this pull request.
  • Please see our prioritization guide for information on how we prioritize.
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

For Submitters

  • Review the contribution guide relating to the type of change you are making to ensure all of the necessary steps have been taken.
  • For new resources and data sources, use skaff to generate scaffolding with comments detailing common expectations.
  • Whether or not the branch has been rebased will not impact prioritization, but doing so is always a welcome surprise.

@github-actions github-actions bot added size/XS Managed by automation to categorize the size of a PR. service/sagemaker Issues and PRs that pertain to the sagemaker service. needs-triage Waiting for first response or review from a maintainer. labels Apr 18, 2023
@dod38fr dod38fr changed the title fix segv in sagemaker app fix segv in sagemaker app resource Apr 18, 2023
@ewbankkit ewbankkit added crash Results from or addresses a Terraform crash or kernel panic. and removed needs-triage Waiting for first response or review from a maintainer. labels Apr 18, 2023
@github-actions github-actions bot added size/M Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. and removed size/XS Managed by automation to categorize the size of a PR. labels Apr 18, 2023
Copy link
Contributor

@ewbankkit ewbankkit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀.

% make testacc TESTARGS='-run=TestAccSageMaker_serial/^App$$' PKG=sagemaker
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/sagemaker/... -v -count 1 -parallel 20  -run=TestAccSageMaker_serial/^App$ -timeout 180m
=== RUN   TestAccSageMaker_serial
=== PAUSE TestAccSageMaker_serial
=== CONT  TestAccSageMaker_serial
=== RUN   TestAccSageMaker_serial/App
=== RUN   TestAccSageMaker_serial/App/tags
=== RUN   TestAccSageMaker_serial/App/resourceSpec
=== RUN   TestAccSageMaker_serial/App/resourceSpecLifecycle
=== RUN   TestAccSageMaker_serial/App/space
=== RUN   TestAccSageMaker_serial/App/basic
=== RUN   TestAccSageMaker_serial/App/disappears
--- PASS: TestAccSageMaker_serial (3091.82s)
    --- PASS: TestAccSageMaker_serial/App (3091.82s)
        --- PASS: TestAccSageMaker_serial/App/tags (593.80s)
        --- PASS: TestAccSageMaker_serial/App/resourceSpec (535.62s)
        --- PASS: TestAccSageMaker_serial/App/resourceSpecLifecycle (543.39s)
        --- PASS: TestAccSageMaker_serial/App/space (478.35s)
        --- PASS: TestAccSageMaker_serial/App/basic (417.97s)
        --- PASS: TestAccSageMaker_serial/App/disappears (522.68s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/sagemaker	3101.374s

@ewbankkit
Copy link
Contributor

@dod38fr Thanks for the contribution 🎉 👏.

@ewbankkit ewbankkit merged commit 5d24bb4 into hashicorp:main Apr 18, 2023
38 checks passed
@github-actions github-actions bot added this to the v4.64.0 milestone Apr 18, 2023
@dod38fr
Copy link
Author

dod38fr commented Apr 18, 2023

That was fast ! Thanks :-)

@github-actions
Copy link

This functionality has been released in v4.64.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

@github-actions
Copy link

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 22, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
crash Results from or addresses a Terraform crash or kernel panic. service/sagemaker Issues and PRs that pertain to the sagemaker service. size/M Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants