Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aws: Add context on credential providers, EC2Metadata client operations #461

Merged
merged 7 commits into from
Jan 8, 2020

Conversation

skotambkar
Copy link
Contributor

This PR makes a breaking change to the aws sdk for go v2, by adding context as a required parameter to each operation on EC2Metadata client. We also modified the Retrieve method on Credential Provider to take in a user provided context.

SDK operations that relied on credential providers have been updated to accommodate this change. Tests have also been modified.

@skotambkar skotambkar marked this pull request as ready for review December 19, 2019 21:30
@skotambkar skotambkar self-assigned this Dec 20, 2019
@skotambkar skotambkar added the pr/work-in-progress This PR is a draft and needs further work. label Dec 20, 2019
@skotambkar skotambkar added pr/needs-review This PR needs a review from a Member. and removed pr/work-in-progress This PR is a draft and needs further work. labels Dec 20, 2019
Copy link
Contributor

@jasdel jasdel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just a couple comments. Only notable one is the external config loading for EC2 Metadata region.

aws/credentials_test.go Show resolved Hide resolved
aws/static_provider.go Outdated Show resolved Hide resolved
aws/external/provider.go Outdated Show resolved Hide resolved
@jasdel
Copy link
Contributor

jasdel commented Dec 31, 2019

Are #461 and #462 duplicates of each other? They seem very similar.

@jasdel jasdel removed the pr/needs-review This PR needs a review from a Member. label Dec 31, 2019
@skotambkar
Copy link
Contributor Author

#462 is based of #461

@skotambkar skotambkar force-pushed the add/ctxToCredProvider branch 3 times, most recently from d7187c5 to d97f6a6 Compare January 3, 2020 23:02
@skotambkar skotambkar added the pr/needs-review This PR needs a review from a Member. label Jan 5, 2020
service/rds/rdsutils/connect.go Outdated Show resolved Hide resolved
@skotambkar skotambkar added pr/work-in-progress This PR is a draft and needs further work. and removed pr/needs-review This PR needs a review from a Member. labels Jan 6, 2020
@skotambkar skotambkar force-pushed the add/ctxToCredProvider branch 3 times, most recently from 3f57866 to 0f126b6 Compare January 6, 2020 22:59
@skotambkar skotambkar added pr/needs-review This PR needs a review from a Member. and removed pr/work-in-progress This PR is a draft and needs further work. labels Jan 6, 2020
@skotambkar skotambkar requested a review from jasdel January 6, 2020 23:39
@skotambkar skotambkar added pr/work-in-progress This PR is a draft and needs further work. and removed pr/needs-review This PR needs a review from a Member. labels Jan 8, 2020
@skotambkar skotambkar changed the title aws: Add context on credential providers, EC2Metadata client operations aws: Add context on credential providers, EC2Metadata client o… Jan 8, 2020
@skotambkar skotambkar merged commit 1e71955 into aws:master Jan 8, 2020
@skotambkar skotambkar deleted the add/ctxToCredProvider branch January 8, 2020 23:19
@skotambkar skotambkar changed the title aws: Add context on credential providers, EC2Metadata client o… aws: Add context on credential providers, EC2Metadata client operations Jan 8, 2020
skmcgrail added a commit to skmcgrail/aws-sdk-go-v2 that referenced this pull request Jan 30, 2020
Breaking Change
---
* `service`: Add generated service for wafregional and dynamodbstreams aws#463
  * Updates the wafregional and dynamodbstreams API clients to include all API operations, and types that were previously shared between waf and dynamodb API clients respectively. This update ensures that all API clients include all operations and types needed for that client, and shares no types with another client package.
  * To migrate your applications to use the updated wafregional and dynamodbstreams you'll need to update the package the impacted type is imported from to match the client the type is being used with.
* `aws`: Context has been added to EC2Metadata operations.([aws#461](aws#461))
  * Also updates utilities that directly or indirectly depend on EC2Metadata client. Signer utilities, credential providers now take in context.
* `private/model`: Add utility for validating shape names for structs and enums for the service packages ([aws#471](aws#471))
  * Fixes bug which allowed service package structs, enums to start with non alphabetic character
  * Fixes the incorrect enum types in mediapackage service package, changing enum types __AdTriggersElement, __PeriodTriggersElement to AdTriggersElement, PeriodTriggersElement respectively.
* `aws`: Client, Metadata, and Request structures have been refactored to simplify the usage of resolved endpoints ([aws#473](aws#473))
  * `aws.Client.Endpoint` struct member has been removed, and `aws.Request.Endpoint` struct member has been added of type `aws.Endpoint`
  * `aws.Client.Region` structure member has been removed

Services
---
* Synced the V2 SDK with latest AWS service API definitions.

SDK Features
---
* `aws`: `PartitionID` has been added to `aws.Endpoint` structure, and is used by the endpoint resolver to indicate which AWS partition an endpoint was resolved for ([aws#473](aws#473))
* `aws/endpoints`: Updated resolvers to populate `PartitionID` for a resolved `aws.Endpoint` ([aws#473](aws#473))
* `service/s3`: Add support for Access Point resources
  * Adds support for using Access Point resource with Amazon S3 API operation calls. The Access Point resource are identified by an Amazon Resource Name (ARN).
  * To make operation calls to an S3 Access Point instead of a S3 Bucket, provide the Access Point ARN string as the value of the Bucket parameter. You can create an Access Point for your bucket with the Amazon S3 Control API. The Access Point ARN can be obtained from the S3 Control API. You should avoid building the ARN directly.

SDK Enhancements
---
* `internal/sdkio`: Adds RingBuffer data structure to the sdk [aws#417](aws#417)
  * Adds an implementation of RingBuffer data structure which acts as a revolving buffer of a predefined length. The RingBuffer implements io.ReadWriter interface.
  * Adds unit tests to test the behavior of the ring buffer.
* `aws/ec2metadata`: Adds support for EC2Metadata client to use secure tokens provided by the IMDS ([aws#453](aws#453))
  * Modifies EC2Metadata client to use request context within its operations ([aws#462](aws#462))
  * Reduces the default dialer timeout and response header timeout to help reduce latency for known issues with EC2Metadata client running inside a container
  * Modifies and adds tests to verify the behavior of the EC2Metadata client.
* `service/dynamodb/dynamodbattribute`: Adds clarifying docs on dynamodbattribute.UnixTime ([aws#464](aws#464))
* `example/service/sts/assumeRole`: added sts assume role example ([aws#224](aws#224))
  * Fixes [aws#157](aws#157) by adding an example for Amazon STS assume role to retrieve credentials.

SDK Bugs
---
* `service/dynamodb/dynamodbattribute`: Fixes a panic when decoding into a map with a key string type alias. ([aws#465](aws/aws-sdk-go#465))
  * Fixes [aws#410](aws#410),  by adding support for keys that are string aliases.
skmcgrail added a commit to skmcgrail/aws-sdk-go-v2 that referenced this pull request Jan 30, 2020
Breaking Change
---
* `service`: Add generated service for wafregional and dynamodbstreams aws#463
  * Updates the wafregional and dynamodbstreams API clients to include all API operations, and types that were previously shared between waf and dynamodb API clients respectively. This update ensures that all API clients include all operations and types needed for that client, and shares no types with another client package.
  * To migrate your applications to use the updated wafregional and dynamodbstreams you'll need to update the package the impacted type is imported from to match the client the type is being used with.
* `aws`: Context has been added to EC2Metadata operations.([aws#461](aws#461))
  * Also updates utilities that directly or indirectly depend on EC2Metadata client. Signer utilities, credential providers now take in context.
* `private/model`: Add utility for validating shape names for structs and enums for the service packages ([aws#471](aws#471))
  * Fixes bug which allowed service package structs, enums to start with non alphabetic character
  * Fixes the incorrect enum types in mediapackage service package, changing enum types __AdTriggersElement, __PeriodTriggersElement to AdTriggersElement, PeriodTriggersElement respectively.
* `aws`: Client, Metadata, and Request structures have been refactored to simplify the usage of resolved endpoints ([aws#473](aws#473))
  * `aws.Client.Endpoint` struct member has been removed, and `aws.Request.Endpoint` struct member has been added of type `aws.Endpoint`
  * `aws.Client.Region` structure member has been removed

Services
---
* Synced the V2 SDK with latest AWS service API definitions.

SDK Features
---
* `aws`: `PartitionID` has been added to `aws.Endpoint` structure, and is used by the endpoint resolver to indicate which AWS partition an endpoint was resolved for ([aws#473](aws#473))
* `aws/endpoints`: Updated resolvers to populate `PartitionID` for a resolved `aws.Endpoint` ([aws#473](aws#473))
* `service/s3`: Add support for Access Point resources
  * Adds support for using Access Point resource with Amazon S3 API operation calls. The Access Point resource are identified by an Amazon Resource Name (ARN).
  * To make operation calls to an S3 Access Point instead of a S3 Bucket, provide the Access Point ARN string as the value of the Bucket parameter. You can create an Access Point for your bucket with the Amazon S3 Control API. The Access Point ARN can be obtained from the S3 Control API. You should avoid building the ARN directly.

SDK Enhancements
---
* `internal/sdkio`: Adds RingBuffer data structure to the sdk [aws#417](aws#417)
  * Adds an implementation of RingBuffer data structure which acts as a revolving buffer of a predefined length. The RingBuffer implements io.ReadWriter interface.
  * Adds unit tests to test the behavior of the ring buffer.
* `aws/ec2metadata`: Adds support for EC2Metadata client to use secure tokens provided by the IMDS ([aws#453](aws#453))
  * Modifies EC2Metadata client to use request context within its operations ([aws#462](aws#462))
  * Reduces the default dialer timeout and response header timeout to help reduce latency for known issues with EC2Metadata client running inside a container
  * Modifies and adds tests to verify the behavior of the EC2Metadata client.
* `service/dynamodb/dynamodbattribute`: Adds clarifying docs on dynamodbattribute.UnixTime ([aws#464](aws#464))
* `example/service/sts/assumeRole`: added sts assume role example ([aws#224](aws#224))
  * Fixes [aws#157](aws#157) by adding an example for Amazon STS assume role to retrieve credentials.

SDK Bugs
---
* `service/dynamodb/dynamodbattribute`: Fixes a panic when decoding into a map with a key string type alias. ([aws#465](aws/aws-sdk-go#465))
  * Fixes [aws#410](aws#410),  by adding support for keys that are string aliases.
skmcgrail added a commit that referenced this pull request Jan 30, 2020
Breaking Change
---
* `service`: Add generated service for wafregional and dynamodbstreams #463
  * Updates the wafregional and dynamodbstreams API clients to include all API operations, and types that were previously shared between waf and dynamodb API clients respectively. This update ensures that all API clients include all operations and types needed for that client, and shares no types with another client package.
  * To migrate your applications to use the updated wafregional and dynamodbstreams you'll need to update the package the impacted type is imported from to match the client the type is being used with.
* `aws`: Context has been added to EC2Metadata operations.([#461](#461))
  * Also updates utilities that directly or indirectly depend on EC2Metadata client. Signer utilities, credential providers now take in context.
* `private/model`: Add utility for validating shape names for structs and enums for the service packages ([#471](#471))
  * Fixes bug which allowed service package structs, enums to start with non alphabetic character
  * Fixes the incorrect enum types in mediapackage service package, changing enum types __AdTriggersElement, __PeriodTriggersElement to AdTriggersElement, PeriodTriggersElement respectively.
* `aws`: Client, Metadata, and Request structures have been refactored to simplify the usage of resolved endpoints ([#473](#473))
  * `aws.Client.Endpoint` struct member has been removed, and `aws.Request.Endpoint` struct member has been added of type `aws.Endpoint`
  * `aws.Client.Region` structure member has been removed

Services
---
* Synced the V2 SDK with latest AWS service API definitions.

SDK Features
---
* `aws`: `PartitionID` has been added to `aws.Endpoint` structure, and is used by the endpoint resolver to indicate which AWS partition an endpoint was resolved for ([#473](#473))
* `aws/endpoints`: Updated resolvers to populate `PartitionID` for a resolved `aws.Endpoint` ([#473](#473))
* `service/s3`: Add support for Access Point resources
  * Adds support for using Access Point resource with Amazon S3 API operation calls. The Access Point resource are identified by an Amazon Resource Name (ARN).
  * To make operation calls to an S3 Access Point instead of a S3 Bucket, provide the Access Point ARN string as the value of the Bucket parameter. You can create an Access Point for your bucket with the Amazon S3 Control API. The Access Point ARN can be obtained from the S3 Control API. You should avoid building the ARN directly.

SDK Enhancements
---
* `internal/sdkio`: Adds RingBuffer data structure to the sdk [#417](#417)
  * Adds an implementation of RingBuffer data structure which acts as a revolving buffer of a predefined length. The RingBuffer implements io.ReadWriter interface.
  * Adds unit tests to test the behavior of the ring buffer.
* `aws/ec2metadata`: Adds support for EC2Metadata client to use secure tokens provided by the IMDS ([#453](#453))
  * Modifies EC2Metadata client to use request context within its operations ([#462](#462))
  * Reduces the default dialer timeout and response header timeout to help reduce latency for known issues with EC2Metadata client running inside a container
  * Modifies and adds tests to verify the behavior of the EC2Metadata client.
* `service/dynamodb/dynamodbattribute`: Adds clarifying docs on dynamodbattribute.UnixTime ([#464](#464))
* `example/service/sts/assumeRole`: added sts assume role example ([#224](#224))
  * Fixes [#157](#157) by adding an example for Amazon STS assume role to retrieve credentials.

SDK Bugs
---
* `service/dynamodb/dynamodbattribute`: Fixes a panic when decoding into a map with a key string type alias. ([#465](#465))
  * Fixes [#410](#410),  by adding support for keys that are string aliases.
skotambkar added a commit to skotambkar/aws-sdk-go-v2 that referenced this pull request Jan 31, 2020
…s#461)

This PR makes a breaking change to the aws sdk for go v2, by adding context as a required parameter to each operation on EC2Metadata client. We also modified the Retrieve method on Credential Provider and signer utility to take in a user provided context.

SDK operations that relied on credential providers have been updated to accommodate this change. Tests and examples have also been modified.
skotambkar pushed a commit to skotambkar/aws-sdk-go-v2 that referenced this pull request Jan 31, 2020
Breaking Change
---
* `service`: Add generated service for wafregional and dynamodbstreams aws#463
  * Updates the wafregional and dynamodbstreams API clients to include all API operations, and types that were previously shared between waf and dynamodb API clients respectively. This update ensures that all API clients include all operations and types needed for that client, and shares no types with another client package.
  * To migrate your applications to use the updated wafregional and dynamodbstreams you'll need to update the package the impacted type is imported from to match the client the type is being used with.
* `aws`: Context has been added to EC2Metadata operations.([aws#461](aws#461))
  * Also updates utilities that directly or indirectly depend on EC2Metadata client. Signer utilities, credential providers now take in context.
* `private/model`: Add utility for validating shape names for structs and enums for the service packages ([aws#471](aws#471))
  * Fixes bug which allowed service package structs, enums to start with non alphabetic character
  * Fixes the incorrect enum types in mediapackage service package, changing enum types __AdTriggersElement, __PeriodTriggersElement to AdTriggersElement, PeriodTriggersElement respectively.
* `aws`: Client, Metadata, and Request structures have been refactored to simplify the usage of resolved endpoints ([aws#473](aws#473))
  * `aws.Client.Endpoint` struct member has been removed, and `aws.Request.Endpoint` struct member has been added of type `aws.Endpoint`
  * `aws.Client.Region` structure member has been removed

Services
---
* Synced the V2 SDK with latest AWS service API definitions.

SDK Features
---
* `aws`: `PartitionID` has been added to `aws.Endpoint` structure, and is used by the endpoint resolver to indicate which AWS partition an endpoint was resolved for ([aws#473](aws#473))
* `aws/endpoints`: Updated resolvers to populate `PartitionID` for a resolved `aws.Endpoint` ([aws#473](aws#473))
* `service/s3`: Add support for Access Point resources
  * Adds support for using Access Point resource with Amazon S3 API operation calls. The Access Point resource are identified by an Amazon Resource Name (ARN).
  * To make operation calls to an S3 Access Point instead of a S3 Bucket, provide the Access Point ARN string as the value of the Bucket parameter. You can create an Access Point for your bucket with the Amazon S3 Control API. The Access Point ARN can be obtained from the S3 Control API. You should avoid building the ARN directly.

SDK Enhancements
---
* `internal/sdkio`: Adds RingBuffer data structure to the sdk [aws#417](aws#417)
  * Adds an implementation of RingBuffer data structure which acts as a revolving buffer of a predefined length. The RingBuffer implements io.ReadWriter interface.
  * Adds unit tests to test the behavior of the ring buffer.
* `aws/ec2metadata`: Adds support for EC2Metadata client to use secure tokens provided by the IMDS ([aws#453](aws#453))
  * Modifies EC2Metadata client to use request context within its operations ([aws#462](aws#462))
  * Reduces the default dialer timeout and response header timeout to help reduce latency for known issues with EC2Metadata client running inside a container
  * Modifies and adds tests to verify the behavior of the EC2Metadata client.
* `service/dynamodb/dynamodbattribute`: Adds clarifying docs on dynamodbattribute.UnixTime ([aws#464](aws#464))
* `example/service/sts/assumeRole`: added sts assume role example ([aws#224](aws#224))
  * Fixes [aws#157](aws#157) by adding an example for Amazon STS assume role to retrieve credentials.

SDK Bugs
---
* `service/dynamodb/dynamodbattribute`: Fixes a panic when decoding into a map with a key string type alias. ([aws#465](aws#465))
  * Fixes [aws#410](aws#410),  by adding support for keys that are string aliases.
skotambkar added a commit that referenced this pull request Jan 31, 2020
* Add generated service for wafregional and dynamodbstreams (#463)

Updates the wafregional and dynamodbstreams API clients to include all API operations, and types that were previously shared between waf and dynamodb API clients respectively. This update ensures that all API clients include all operations and types needed for that client, and shares no types with another client package.

To migrate your applications to use the updated wafregional and dynamodbstreams you'll need to update the package the impacted type is imported from to match the client the type is being used with.

* `aws`: Add context on credential providers, EC2Metadata client o… (#461)

This PR makes a breaking change to the aws sdk for go v2, by adding context as a required parameter to each operation on EC2Metadata client. We also modified the Retrieve method on Credential Provider and signer utility to take in a user provided context.

SDK operations that relied on credential providers have been updated to accommodate this change. Tests and examples have also been modified.

* `private/model`: Add shape name validation for structs and enums (#471)

* Adds `validateShapeName` utility to validate that the shape names for structs and enums adhere to a defined format. 
* Fixes bug which allowed service package structs, enums to start with non alphabetic character 
* Fixes the incorrect enum types in mediapackage service package, changing enum types __AdTriggersElement, __PeriodTriggersElement to AdTriggersElement, PeriodTriggersElement respectively.
* Adds unit tests to test the validateShapeName utility.

* internal/sdkio: add RingBuffer data structure to the sdk (#417)

Adds a RingBuffer data structure. RingBuffer acts as a revolving buffer of a predefined length. It implements io.ReadWriter interface.

* aws/ec2metadata`: modifies IMDS client to use shorter request timeout

The PR modifies EC2Metadata client to use request context within its operations. Reduces the dialer timeout and response header timeout on the EC2Metadata client to help reduce latency for known issues with EC2Metadata client running inside a container.

* aws: Endpoints Usage Refactor for aws.Request and aws.Metadata (#473)

* Support for Loading Service Specific Configuration from External Sources During Client Creation (#474)

* service/s3: Adds Support for S3 AccessPoints (#476)

* Simplify Service Client Configuration Resolving (#478)

* Add timestamp support for SDK's generated API smoke tests (#477)

* Release v0.19.0 (2020-01-30) (#480)

Breaking Change
---
* `service`: Add generated service for wafregional and dynamodbstreams #463
  * Updates the wafregional and dynamodbstreams API clients to include all API operations, and types that were previously shared between waf and dynamodb API clients respectively. This update ensures that all API clients include all operations and types needed for that client, and shares no types with another client package.
  * To migrate your applications to use the updated wafregional and dynamodbstreams you'll need to update the package the impacted type is imported from to match the client the type is being used with.
* `aws`: Context has been added to EC2Metadata operations.([#461](#461))
  * Also updates utilities that directly or indirectly depend on EC2Metadata client. Signer utilities, credential providers now take in context.
* `private/model`: Add utility for validating shape names for structs and enums for the service packages ([#471](#471))
  * Fixes bug which allowed service package structs, enums to start with non alphabetic character
  * Fixes the incorrect enum types in mediapackage service package, changing enum types __AdTriggersElement, __PeriodTriggersElement to AdTriggersElement, PeriodTriggersElement respectively.
* `aws`: Client, Metadata, and Request structures have been refactored to simplify the usage of resolved endpoints ([#473](#473))
  * `aws.Client.Endpoint` struct member has been removed, and `aws.Request.Endpoint` struct member has been added of type `aws.Endpoint`
  * `aws.Client.Region` structure member has been removed

Services
---
* Synced the V2 SDK with latest AWS service API definitions.

SDK Features
---
* `aws`: `PartitionID` has been added to `aws.Endpoint` structure, and is used by the endpoint resolver to indicate which AWS partition an endpoint was resolved for ([#473](#473))
* `aws/endpoints`: Updated resolvers to populate `PartitionID` for a resolved `aws.Endpoint` ([#473](#473))
* `service/s3`: Add support for Access Point resources
  * Adds support for using Access Point resource with Amazon S3 API operation calls. The Access Point resource are identified by an Amazon Resource Name (ARN).
  * To make operation calls to an S3 Access Point instead of a S3 Bucket, provide the Access Point ARN string as the value of the Bucket parameter. You can create an Access Point for your bucket with the Amazon S3 Control API. The Access Point ARN can be obtained from the S3 Control API. You should avoid building the ARN directly.

SDK Enhancements
---
* `internal/sdkio`: Adds RingBuffer data structure to the sdk [#417](#417)
  * Adds an implementation of RingBuffer data structure which acts as a revolving buffer of a predefined length. The RingBuffer implements io.ReadWriter interface.
  * Adds unit tests to test the behavior of the ring buffer.
* `aws/ec2metadata`: Adds support for EC2Metadata client to use secure tokens provided by the IMDS ([#453](#453))
  * Modifies EC2Metadata client to use request context within its operations ([#462](#462))
  * Reduces the default dialer timeout and response header timeout to help reduce latency for known issues with EC2Metadata client running inside a container
  * Modifies and adds tests to verify the behavior of the EC2Metadata client.
* `service/dynamodb/dynamodbattribute`: Adds clarifying docs on dynamodbattribute.UnixTime ([#464](#464))
* `example/service/sts/assumeRole`: added sts assume role example ([#224](#224))
  * Fixes [#157](#157) by adding an example for Amazon STS assume role to retrieve credentials.

SDK Bugs
---
* `service/dynamodb/dynamodbattribute`: Fixes a panic when decoding into a map with a key string type alias. ([#465](#465))
  * Fixes [#410](#410),  by adding support for keys that are string aliases.

Co-authored-by: Sean McGrail <skmcgrail@gmail.com>
Co-authored-by: Jason Del Ponte <delpontej@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr/work-in-progress This PR is a draft and needs further work.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants