Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SLS-2492] Upgrade aws sdk v2 #113

Merged
merged 8 commits into from
Nov 8, 2022
Merged

[SLS-2492] Upgrade aws sdk v2 #113

merged 8 commits into from
Nov 8, 2022

Conversation

DylanLovesCoffee
Copy link
Contributor

@DylanLovesCoffee DylanLovesCoffee commented Aug 31, 2022

What does this PR do?

Migrates existing KMS decryption logic to use aws-sdk-go-v2

Motivation

Upgrade/migrate to aws-sdk-go-v2

Testing Guidelines

Manual testing with KMS encrypted keys

Additional Notes

note: the unit test was removed due to difficulty in mocking the new kms Client with the new sdk version, since previously a KMS interface was available in v1. Ideally we'd want to add back the unit test and potentially have an integration test covering this feature too.

Types of changes

  • Bug fix
  • New feature
  • Breaking change
  • Misc (docs, refactoring, dependency upgrade, etc.)

Checklist

  • This PR's description is comprehensive
  • This PR contains breaking changes that are documented in the description
  • This PR introduces new APIs or parameters that are documented and unlikely to change in the foreseeable future
  • This PR impacts documentation, and it has been updated (or a ticket has been logged)
  • This PR's changes are covered by the automated tests
  • This PR collects user input/sensitive content into Datadog

@DylanLovesCoffee DylanLovesCoffee marked this pull request as ready for review October 27, 2022 14:55
@DylanLovesCoffee DylanLovesCoffee requested a review from a team as a code owner October 27, 2022 14:55
@codecov-commenter
Copy link

codecov-commenter commented Oct 27, 2022

Codecov Report

Merging #113 (ba49bca) into main (1cdcde5) will increase coverage by 0.01%.
The diff coverage is 88.88%.

@@            Coverage Diff             @@
##             main     #113      +/-   ##
==========================================
+ Coverage   83.45%   83.47%   +0.01%     
==========================================
  Files          13       13              
  Lines         846      847       +1     
==========================================
+ Hits          706      707       +1     
  Misses        111      111              
  Partials       29       29              
Impacted Files Coverage Δ
internal/metrics/kms_decrypter.go 70.00% <88.88%> (+1.03%) ⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Contributor

@purple4reina purple4reina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good as is, but left comments on ideas for improvement.

},
}
response, err = kmsClient.Decrypt(params)
response, err = kmsClient.Decrypt(context.TODO(), params)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is kinda a nit pick, but I think we wanna use context.Background(). We can then also use the same context for both of these kms client calls. It won't change any of the functionality, so really this is just a nit, but it would allow us to do cool things like add timeouts in the future.

client := mockKMSClientNoEncryptionContext{}
result, _ := decryptKMS(client, mockEncryptedAPIKeyBase64)
assert.Equal(t, expectedDecryptedAPIKey, result)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, I think these tests can still work, and would actually be a bit more simple than they currently are.

I think (haven't tested this) we can mock out the HTTPClient to just send us the response we want to test. Something like this (note I have not tested this at all):

type mockHTTPClient string

func (m mockHTTPClient) Do(*http.Request) (*http.Response, err) {
  return &http.Response{
    StatusCode: 200,
    Body: ioutil.NopCloser(bytes.NewBufferString(m)),
  }
}

func TestWhatever(t *testing.T) {
  client := kms.New(kms.Options{HTTPClient: mockHTTPClient("hello world")})
  result, _ decryptKMS(client, mockEncryptedAPIKeyBase64)
  assert.Equal(t, expectedDecryptedAPIKey, result)
}

Copy link
Contributor

@IvanTopolcic IvanTopolcic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Comment on lines +28 to +32
kmsClient *kms.Client
}

clientDecrypter interface {
Decrypt(context.Context, *kms.DecryptInput, ...func(*kms.Options)) (*kms.DecryptOutput, error)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does AWS not provide the KMSAPI interface anymore?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like not with v2 😭

Copy link
Contributor

@purple4reina purple4reina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

@DylanLovesCoffee DylanLovesCoffee merged commit b23e66a into main Nov 8, 2022
@DylanLovesCoffee DylanLovesCoffee deleted the dylan/upgrade-sdk branch November 8, 2022 16:51
peterdeme pushed a commit to spacelift-io/datadog-lambda-go that referenced this pull request Nov 15, 2023
peterdeme added a commit to spacelift-io/datadog-lambda-go that referenced this pull request Dec 4, 2023
* Create codeql-analysis.yml (DataDog#100)

* Create codeql-analysis.yml

* Update codeql-analysis.yml

* Update run_integration_tests.sh

* Do not show error messages even if neither DD_API_KEY nor DD_KMS_API_KEY is set when Lambda Extension is running (DataDog#102)

* Bump version to 1.4.0

* Bump go + fasthttp + lint (DataDog#104)

* Consolidate serverless configurations into one place (DataDog#105)

* Update README.md

* Update README.md

* Bump dd-trace-go to latest version to address some vulnerabilities (DataDog#109)

* Bump dd-trace-go to latest version to address some vulnaribilities
* update go.sum with `go mod tidy`

* Bump version to 1.6.0

* bump codeql (DataDog#112)

* Bump dd-trace-go to v1.41 (DataDog#115)

* Bump version to 1.7.0

* [SLS-2330] Add support for universal instrumentation with the extension (DataDog#116)

add option to use universal instrumentation

* [EEP-444] include error in failed metric send log (DataDog#118)

Co-authored-by: Corey Griffin <CoreyGriffin@users.noreply.github.com>

* [SLS-2492] Upgrade aws sdk v2 (DataDog#113)

upgrade sdk

* Bump version to 1.8.0

* Use new account in integration tests (DataDog#119)

* set the architecture explicitely (DataDog#122)

* mask init runtime logs (DataDog#123)

* Update libs (DataDog#121)

* bump go 1.18 (DataDog#125)

* Retry sending trace payloads on failure. (DataDog#128)

* Bump version to 1.9.0

* Update DD Trace to  v1.51.0(DataDog#133)

* Bump go version to 1.20 (DataDog#140)

Bump go version to 1.20

* Upgrade version of dd-trace-go to v1.54.1 (DataDog#141)

* Bump version to 1.10.0

* Propagate trace context from SQS events (DataDog#142)

* Default parent id to be trace id if not found elsewhere.

* Look for trace context in context object as well as headers.

* Apply trace context before starting the function execution span.

* Update signature in tests.

* Add spanid of execution span to context.

* Do not ignore priority "-128".

* Test that default parent id set to trace id.

* Test span id added to context.

* Test uses trace context from context object.

* Bump version to 1.11.0

* feat: automate AppSec enablement setup (e.g: `AWS_LAMBDA_RUNTIME_API`) (DataDog#143)

* feat: honor AWS_LAMBDA_EXEC_WRAPPER when AWS Lambda does not

In order to simplify onboarding & make it more uniform across languages,
inspect the value of the `AWS_LAMBDA_EXEC_WRAPPER` environment variable
and apply select environment variable changes it perofrms upon
decorating a handler.

This is necessary/useful because that environment variable is not
honored by custom runtimes (`provided`, `provided.al2`) as well as the
`go1.x` runtime (which is a glorified provided runtime). The datadog
Lambda wrapper starts a proxy to inject ASM functionality directly on
the Lambda runtime API instead of having to manually instrument each and
every lambda handler/application, and modifies `AWS_LAMBDA_RUNTIME_API`
to instruct Lambda language runtime client libraries to go through it
instead of directly interacting with the Lambda control plane.

APPSEC-11534

* pivot to a different, cheaper strategy

* typo fix

* PR feedback

* minor fixups

* add warning in go1.x runtime if lambda.norpc build tag was not enabled

* Bump version to 1.12.0

* Re-add configs after upstream rebase

* Bump packages

* Remove deprecated `io/ioutil` calls

---------

Co-authored-by: Tian Chu <tian.chu@datadoghq.com>
Co-authored-by: Soshi Katsuta <skatsuta@users.noreply.github.com>
Co-authored-by: Maxime David <maxime.david@datadoghq.com>
Co-authored-by: kimi <47579703+kimi-p@users.noreply.github.com>
Co-authored-by: Kimi Wu <kimi.wu@datadoghq.com>
Co-authored-by: Dylan Yang <dylan.yang@datadoghq.com>
Co-authored-by: Corey Griffin <15809365+CoreyGriffin@users.noreply.github.com>
Co-authored-by: Corey Griffin <CoreyGriffin@users.noreply.github.com>
Co-authored-by: Marcin Rabenda <xrn.design@gmail.com>
Co-authored-by: Rey Abolofia <purple4reina@gmail.com>
Co-authored-by: Rey Abolofia <rey.abolofia@datadoghq.com>
Co-authored-by: Andrew Rodriguez <49878080+zARODz11z@users.noreply.github.com>
Co-authored-by: Ivan Topolcic <IvanTopolcic@users.noreply.github.com>
Co-authored-by: Romain Marcadier <romain.muller@telecomnancy.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants