Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Port Shared Config Improvements & Credential Providers #488

Merged
merged 12 commits into from
Mar 2, 2020
Merged

Port Shared Config Improvements & Credential Providers #488

merged 12 commits into from
Mar 2, 2020

Conversation

skmcgrail
Copy link
Member

@skmcgrail skmcgrail commented Feb 7, 2020 •
edited
Loading

Criteria for Success:

  • Shared Config Improvements from V1
  • Process Credential Provider
  • Web Identity Token Provider
  • Credential Chaining and Loading from Environment and Shared Config
  • Separate Credential Provider Configurable Options, Provide Functional Arguments
  • Support Configuring Credential Providers created in the Credential Chain via functional arguments

@skmcgrail skmcgrail added the pr/work-in-progress This PR is a draft and needs further work. label Feb 7, 2020
This was linked to issues Feb 7, 2020
Port Web Identity Credential Provider to V2 #338
Closed
Missing support for Web Identity Tokens #475
Closed
@skmcgrail skmcgrail changed the title Port Shared Config Improvements & Credential Providers [Draft] Port Shared Config Improvements & Credential Providers Feb 7, 2020
@hasheddan hasheddan mentioned this pull request Feb 11, 2020
Merged
4 tasks
@skmcgrail skmcgrail changed the title [Draft] Port Shared Config Improvements & Credential Providers Port Shared Config Improvements & Credential Providers Feb 12, 2020
@skmcgrail skmcgrail added pr/needs-review This PR needs a review from a Member. and removed pr/work-in-progress This PR is a draft and needs further work. labels Feb 12, 2020
jasdel
jasdel suggested changes Feb 25, 2020
View reviewed changes
aws/external/resolve_credentials.go Outdated Show resolved Hide resolved
aws/external/resolve_credentials.go Show resolved Hide resolved
aws/external/shared_config.go Outdated Show resolved Hide resolved
aws/processcreds/provider.go Outdated Show resolved Hide resolved
aws/stscreds/provider.go Outdated Show resolved Hide resolved
aws/stscreds/web_identity_provider.go Outdated Show resolved Hide resolved
aws/external/resolve_credentials.go Outdated Show resolved Hide resolved
@skmcgrail skmcgrail merged commit 9fc62ee into aws:master Mar 2, 2020
@hasheddan hasheddan mentioned this pull request Mar 3, 2020
Closed
@skmcgrail skmcgrail mentioned this pull request Mar 5, 2020
Closed
This was referenced Mar 5, 2020
Closed
Closed
skmcgrail added a commit to skmcgrail/aws-sdk-go-v2 that referenced this pull request Mar 17, 2020
@skmcgrail
Release v0.20.0 (2020-03-17)
3fb24bf 
Breaking Change
---
* Update SDK retry behavior
  * Significant updates were made the SDK's retry behavior. The SDK will now retry all connections error. In addition, to changing what errors are retried the SDK's retry behavior not distinguish the difference between throttling errors, and regular retryable errors. All errors will be retried with the same backoff jitter delay scaling.
  * The SDK will attempt an operation request 3 times by default. This is one less than the previous initial request with 3 retires.
  * New helper functions in the new `aws/retry` package allow wrapping a `Retrier` with custom behavior, overriding the base `Retrier`, (e.g. `AddWithErrorCodes`, and `AddWithMaxAttempts`)
* Update SDK error handling
  * Updates the SDK's handling of errors to take advantage of Go 1.13's new `errors.As`, `Is`, and `Unwrap`. The SDK's errors were updated to satisfy the `Unwrap` interface, returning the underlying error.
  * With this update, you can now more easily access the SDK's layered errors, and meaningful state such as, `Timeout`, `Temporary`, and other states added to the SDK such as `CanceledError`.
* Bump SDK minimum supported version from Go 1.12 to Go 1.13
  * The SDK's minimum supported version is bumped to take advantage of Go 1.13's updated `errors` package.

Services
---
* Synced the V2 SDK with latest AWS service API definitions.

SDK Features
---
* `aws`: Add Support for additional credential providers and credential configuration chaining ([aws#488](aws#488))
  * `aws/processcreds`: Adds Support for the Process Credential Provider
    * Fixes [aws#249](aws#249)
  * `aws/stscreds`: Adds Support for the Web Identity Credential Provider
    * Fixes [aws#475](aws#475)
    * Fixes [aws#338](aws#338)
  * Adds Support for `credential_source`
    * Fixes [aws#274](aws#274)
* `aws/awserr`: Adds support for Go 1.13's `errors.Unwrap` ([aws#487](aws#487))
* `aws`: Updates SDK retry behavior ([aws#487](aws#487))
  * `aws/retry`: New package defining logic to determine if a request should be retried, and backoff.
  * `aws/ratelimit`: New package defining rate limit logic such as token bucket used by the `retry.Standard` retrier.

SDK Enhancements
---
* `aws`: Add grouping of concurrent refresh of credentials ([aws#503](aws#503)
  * Concurrent calls to `Retrieve` are now grouped in order to prevent numerous synchronous calls to refresh the credentials. Replacing the mutex with a singleflight reduces the overall amount of time request signatures need to wait while retrieving credentials. This is improvement becomes pronounced when many requests are made concurrently.
* `service/s3/s3manager`: Improve memory allocation behavior by replacing sync.Pool with custom pool implementation
  * Improves memory allocations that occur when the provided `io.Reader` to upload does not satisfy both the `io.ReaderAt` and `io.ReadSeeker` interfaces.

SDK Bugs
---
* `service/s3/s3manager`: Fix resource leaks when the following occurred:
  * Failed CreateMultipartUpload call
  * Failed UploadPart call
@skmcgrail skmcgrail mentioned this pull request Mar 17, 2020
Merged
skmcgrail added a commit to skmcgrail/aws-sdk-go-v2 that referenced this pull request Mar 17, 2020
@skmcgrail
Release v0.20.0 (2020-03-17)
dd25caf 
Breaking Change
---
* Update SDK retry behavior
  * Significant updates were made the SDK's retry behavior. The SDK will now retry all connections error. In addition, to changing what errors are retried the SDK's retry behavior not distinguish the difference between throttling errors, and regular retryable errors. All errors will be retried with the same backoff jitter delay scaling.
  * The SDK will attempt an operation request 3 times by default. This is one less than the previous initial request with 3 retires.
  * New helper functions in the new `aws/retry` package allow wrapping a `Retrier` with custom behavior, overriding the base `Retrier`, (e.g. `AddWithErrorCodes`, and `AddWithMaxAttempts`)
* Update SDK error handling
  * Updates the SDK's handling of errors to take advantage of Go 1.13's new `errors.As`, `Is`, and `Unwrap`. The SDK's errors were updated to satisfy the `Unwrap` interface, returning the underlying error.
  * With this update, you can now more easily access the SDK's layered errors, and meaningful state such as, `Timeout`, `Temporary`, and other states added to the SDK such as `CanceledError`.
* Bump SDK minimum supported version from Go 1.12 to Go 1.13
  * The SDK's minimum supported version is bumped to take advantage of Go 1.13's updated `errors` package.

Services
---
* Synced the V2 SDK with latest AWS service API definitions.

SDK Features
---
* `aws`: Add Support for additional credential providers and credential configuration chaining ([aws#488](aws#488))
  * `aws/processcreds`: Adds Support for the Process Credential Provider
    * Fixes [aws#249](aws#249)
  * `aws/stscreds`: Adds Support for the Web Identity Credential Provider
    * Fixes [aws#475](aws#475)
    * Fixes [aws#338](aws#338)
  * Adds Support for `credential_source`
    * Fixes [aws#274](aws#274)
* `aws/awserr`: Adds support for Go 1.13's `errors.Unwrap` ([aws#487](aws#487))
* `aws`: Updates SDK retry behavior ([aws#487](aws#487))
  * `aws/retry`: New package defining logic to determine if a request should be retried, and backoff.
  * `aws/ratelimit`: New package defining rate limit logic such as token bucket used by the `retry.Standard` retrier.

SDK Enhancements
---
* `aws`: Add grouping of concurrent refresh of credentials ([aws#503](aws#503))
  * Concurrent calls to `Retrieve` are now grouped in order to prevent numerous synchronous calls to refresh the credentials. Replacing the mutex with a singleflight reduces the overall amount of time request signatures need to wait while retrieving credentials. This is improvement becomes pronounced when many requests are made concurrently.
* `service/s3/s3manager`: Improve memory allocation behavior by replacing sync.Pool with custom pool implementation
  * Improves memory allocations that occur when the provided `io.Reader` to upload does not satisfy both the `io.ReaderAt` and `io.ReadSeeker` interfaces.

SDK Bugs
---
* `service/s3/s3manager`: Fix resource leaks when the following occurred:
  * Failed CreateMultipartUpload call
  * Failed UploadPart call
skmcgrail added a commit that referenced this pull request Mar 17, 2020
@skmcgrail
Release v0.20.0 (2020-03-17) (#507)
2930532 
Breaking Change
---
* Update SDK retry behavior
  * Significant updates were made the SDK's retry behavior. The SDK will now retry all connections error. In addition, to changing what errors are retried the SDK's retry behavior not distinguish the difference between throttling errors, and regular retryable errors. All errors will be retried with the same backoff jitter delay scaling.
  * The SDK will attempt an operation request 3 times by default. This is one less than the previous initial request with 3 retires.
  * New helper functions in the new `aws/retry` package allow wrapping a `Retrier` with custom behavior, overriding the base `Retrier`, (e.g. `AddWithErrorCodes`, and `AddWithMaxAttempts`)
* Update SDK error handling
  * Updates the SDK's handling of errors to take advantage of Go 1.13's new `errors.As`, `Is`, and `Unwrap`. The SDK's errors were updated to satisfy the `Unwrap` interface, returning the underlying error.
  * With this update, you can now more easily access the SDK's layered errors, and meaningful state such as, `Timeout`, `Temporary`, and other states added to the SDK such as `CanceledError`.
* Bump SDK minimum supported version from Go 1.12 to Go 1.13
  * The SDK's minimum supported version is bumped to take advantage of Go 1.13's updated `errors` package.

Services
---
* Synced the V2 SDK with latest AWS service API definitions.

SDK Features
---
* `aws`: Add Support for additional credential providers and credential configuration chaining ([#488](#488))
  * `aws/processcreds`: Adds Support for the Process Credential Provider
    * Fixes [#249](#249)
  * `aws/stscreds`: Adds Support for the Web Identity Credential Provider
    * Fixes [#475](#475)
    * Fixes [#338](#338)
  * Adds Support for `credential_source`
    * Fixes [#274](#274)
* `aws/awserr`: Adds support for Go 1.13's `errors.Unwrap` ([#487](#487))
* `aws`: Updates SDK retry behavior ([#487](#487))
  * `aws/retry`: New package defining logic to determine if a request should be retried, and backoff.
  * `aws/ratelimit`: New package defining rate limit logic such as token bucket used by the `retry.Standard` retrier.

SDK Enhancements
---
* `aws`: Add grouping of concurrent refresh of credentials ([#503](#503))
  * Concurrent calls to `Retrieve` are now grouped in order to prevent numerous synchronous calls to refresh the credentials. Replacing the mutex with a singleflight reduces the overall amount of time request signatures need to wait while retrieving credentials. This is improvement becomes pronounced when many requests are made concurrently.
* `service/s3/s3manager`: Improve memory allocation behavior by replacing sync.Pool with custom pool implementation
  * Improves memory allocations that occur when the provided `io.Reader` to upload does not satisfy both the `io.ReaderAt` and `io.ReadSeeker` interfaces.

SDK Bugs
---
* `service/s3/s3manager`: Fix resource leaks when the following occurred:
  * Failed CreateMultipartUpload call
  * Failed UploadPart call
@skmcgrail skmcgrail deleted the credentials/providers branch April 16, 2020 17:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jasdel jasdel jasdel approved these changes

@skotambkar skotambkar Awaiting requested review from skotambkar

Assignees
No one assigned
Labels
pr/needs-review This PR needs a review from a Member.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Missing support for Web Identity Tokens Port Web Identity Credential Provider to V2
2 participants
@skmcgrail @jasdel