AWS SDK V2 Upgrade #4939

jeevanragula · 2023-09-02T17:51:16Z

In the Keda project, we are currently using below AWS Go SDK version.

github.com/aws/aws-sdk-go v1.44.287

But AWS released a V2 version long back which has few advantages over V1.
This PR is to update the code to use AWS SDK V2.

Checklist

When introducing a new scaler, I agree with the scaling governance policy
I have verified that my change is according to the deprecations & breaking changes policy
Tests have been added
Changelog has been updated and is aligned with our changelog requirements
A PR is opened to update our Helm chart (repo) (if applicable, ie. when deployment manifests are modified)
A PR is opened to update the documentation on (repo) (if applicable)
Commits are signed with Developer Certificate of Origin (DCO - learn more)

Fixes #4905

Relates to #4905

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

github-actions · 2023-09-02T17:51:28Z

Thank you for your contribution! 🙏 We will review your PR as soon as possible.

🏖️ Over the summer, the response time will be longer than usual due to maintainers taking time off so please bear with us.

While you are waiting, make sure to:

Add an entry in our changelog in alphabetical order and link related issue
Update the documentation, if needed
Add unit & e2e tests for your changes
GitHub checks are passing
Is the DCO check failing? Here is how you can fix DCO issues

Learn more about:

Our contribution guide

semgrep-app · 2023-09-02T17:52:11Z

Semgrep found 6 context-todo findings:

pkg/scalers/aws_sqs_queue_scaler.go: L232
pkg/scalers/aws_kinesis_stream_scaler.go: L177
pkg/scalers/aws_dynamodb_streams_scaler.go: L203
pkg/scalers/aws_dynamodb_scaler.go: L232
pkg/scalers/aws_common.go: L34
pkg/scalers/aws_cloudwatch_scaler.go: L370

Consider to use well-defined context

_{Ignore this finding from context-todo.}

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

JorTurFer · 2023-09-02T18:32:57Z

/run-e2e aws
Update: You can check the progress here

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

CHANGELOG.md

pkg/metricsservice/api/metrics.pb.go

pkg/metricsservice/api/metrics_grpc.pb.go

pkg/scalers/aws_cloudwatch_scaler.go

pkg/scalers/aws_common.go

pkg/scalers/aws_kinesis_stream_scaler.go

pkg/scalers/aws_sqs_queue_scaler.go

pkg/scalers/externalscaler/externalscaler.pb.go

tests/scalers/aws/aws_cloudwatch/aws_cloudwatch_test.go

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

JorTurFer · 2023-09-03T18:55:07Z

pkg/scalers/aws_dynamodb_scaler.go

+	return attributeValues, nil
+}
+
+func attributeValueFromInterface(value interface{}) (types.AttributeValue, error) {


Does the SDK have something like this internally?

@JorTurFer Found this package. But few test cases are failing when using that.

"github.com/aws/aws-sdk-go-v2/feature/dynamodb/attributevalue"

Do they fail because the behavior has changed?

TestParseDynamoMetadata test case was failing.
May be I am not using this "attributevalue" package correctly. But with my code, all the test cases are working fine.
If any one suggest easy way to replace this method, happy to do that.

JorTurFer · 2023-09-03T18:58:38Z

/run-e2e aws
Update: You can check the progress here

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

zroubalik · 2023-09-05T14:27:09Z

/run-e2e aws
Update: You can check the progress here

JorTurFer · 2023-09-05T19:45:39Z

/run-e2e aws
Update: You can check the progress here

tests/scalers/aws/aws_dynamodb_streams_pod_identity/aws_dynamodb_streams_pod_identity_test.go

…db_streams_pod_identity_test.go Co-authored-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es> Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

semgrep-app · 2023-09-06T20:01:43Z

Semgrep found 6 context-todo findings:

pkg/scalers/aws_sqs_queue_scaler.go: L241
pkg/scalers/aws_kinesis_stream_scaler.go: L177
pkg/scalers/aws_dynamodb_streams_scaler.go: L203
pkg/scalers/aws_dynamodb_scaler.go: L232
pkg/scalers/aws_common.go: L35
pkg/scalers/aws_cloudwatch_scaler.go: L371

Consider to use well-defined context

_{Ignore this finding from context-todo.}

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

…#4922) Co-authored-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es>

…ring (kedacore#4584) * fix RabbitMQ Scaler, allow subpaths along with vhost in connection string (kedacore#4584) Signed-off-by: Roman Bielyi <romanbielyi.amor@gmail.com> * Fix CHANGELOG.md, remove redundant/wrong pieces that were made during rebase, this fix is aimed to resolve failed workflow's Static Check (validate-changelog) test Signed-off-by: Roman Bielyi <romanbielyi.amor@gmail.com> --------- Signed-off-by: Roman Bielyi <romanbielyi.amor@gmail.com>

Signed-off-by: Jorge Turrado <jorge.turrado@scrm.lidl> Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

jeevanragula · 2023-09-07T07:47:56Z

@JorTurFer I have cancelled it and raised a fresh review.
#4953

jeevanragula added 3 commits September 2, 2023 23:13

Updating AWS SDK to V2

6181021

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

Updating AWS SDK to V2

1b316db

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

Updating Changelog

bc5927e

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

jeevanragula requested a review from a team as a code owner September 2, 2023 17:51

jeevanragula added 3 commits September 2, 2023 23:24

Merge branch 'main' into aws_version_update

a95aec5

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

update go mod file and vendor modules text

01f25f3

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

committing vendor folder

dab2106

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

jeevanragula added 2 commits September 3, 2023 00:10

fixing pre-commit issues

c998166

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

fixing pre-commit issues

4bca057

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

JorTurFer reviewed Sep 2, 2023

View reviewed changes

jeevanragula added 2 commits September 3, 2023 01:37

passing context to aws methods

18c5059

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

revert protoc changes

a9a8e39

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

JorTurFer reviewed Sep 3, 2023

View reviewed changes

jeevanragula added 2 commits September 4, 2023 23:51

Merge branch 'main' into aws_version_update

f6bf9de

adding aws endpoint to client

ab94c06

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

Merge branch 'main' into aws_version_update

179aa98

JorTurFer reviewed Sep 5, 2023

View reviewed changes

tests/scalers/aws/aws_dynamodb_streams_pod_identity/aws_dynamodb_streams_pod_identity_test.go Show resolved Hide resolved

jeevanragula and others added 4 commits September 7, 2023 00:38

Update tests/scalers/aws/aws_dynamodb_streams_pod_identity/aws_dynamo…

2fd0008

…db_streams_pod_identity_test.go Co-authored-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es> Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

Updating AWS SDK to V2

96597bc

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

fixing pre-commit issues

f9bd71b

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

go mod update

59e5bf4

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

jeevanragula added 2 commits September 7, 2023 01:36

Updating AWS SDK to V2

9ba65f0

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

resolving conflicts and revert protoc version

4a21f1a

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

Wolfe1 and others added 7 commits September 7, 2023 02:13

Selenium Grid | Fix latest browser version use case scaling (kedacore…

e0c8e0d

…#4922) Co-authored-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es>

chore: bump deps (kedacore#4942)

782fa54

Signed-off-by: Jorge Turrado <jorge.turrado@scrm.lidl> Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

updating AWS SDK to V2

9fbb58c

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

resolving conflicts and revert protoc version

223068c

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

resolve conflicts

6c2d055

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

resolve conflicts

f2557d1

Signed-off-by: Jeevan Reddy Ragula <jeevanragula@gmail.com>

jeevanragula closed this Sep 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AWS SDK V2 Upgrade #4939

AWS SDK V2 Upgrade #4939

jeevanragula commented Sep 2, 2023 •

edited

github-actions bot commented Sep 2, 2023

semgrep-app bot commented Sep 2, 2023

JorTurFer commented Sep 2, 2023 •

edited by github-actions bot

JorTurFer Sep 3, 2023

jeevanragula Sep 4, 2023

JorTurFer Sep 5, 2023

jeevanragula Sep 6, 2023

JorTurFer commented Sep 3, 2023 •

edited by github-actions bot

zroubalik commented Sep 5, 2023 •

edited by github-actions bot

JorTurFer commented Sep 5, 2023 •

edited by github-actions bot

semgrep-app bot commented Sep 6, 2023

jeevanragula commented Sep 7, 2023

AWS SDK V2 Upgrade #4939

AWS SDK V2 Upgrade #4939

Conversation

jeevanragula commented Sep 2, 2023 • edited

Checklist

github-actions bot commented Sep 2, 2023

semgrep-app bot commented Sep 2, 2023

JorTurFer commented Sep 2, 2023 • edited by github-actions bot

JorTurFer Sep 3, 2023

Choose a reason for hiding this comment

jeevanragula Sep 4, 2023

Choose a reason for hiding this comment

JorTurFer Sep 5, 2023

Choose a reason for hiding this comment

jeevanragula Sep 6, 2023

Choose a reason for hiding this comment

JorTurFer commented Sep 3, 2023 • edited by github-actions bot

zroubalik commented Sep 5, 2023 • edited by github-actions bot

JorTurFer commented Sep 5, 2023 • edited by github-actions bot

semgrep-app bot commented Sep 6, 2023

jeevanragula commented Sep 7, 2023

jeevanragula commented Sep 2, 2023 •

edited

JorTurFer commented Sep 2, 2023 •

edited by github-actions bot

JorTurFer commented Sep 3, 2023 •

edited by github-actions bot

zroubalik commented Sep 5, 2023 •

edited by github-actions bot

JorTurFer commented Sep 5, 2023 •

edited by github-actions bot