Skip to content

Update aws sdk to 1.10.69 and add throttle_retries repository setting #273

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Jun 16, 2016
Merged

Update aws sdk to 1.10.69 and add throttle_retries repository setting #273

merged 2 commits into from
Jun 16, 2016

Conversation

@dadoonet
Copy link
Contributor

@dadoonet dadoonet commented Apr 28, 2016

This PR brings 2 changes:

  • Upgrade to AWS SDK 1.10.69
  • Add cloud.aws.s3.throttle_retries setting
  • Moving from JSON.org to Jackson for request marshallers.
  • The Java SDK now supports retry throttling to limit the rate of retries during periods of reduced availability. This throttling behavior can be enabled via ClientConfiguration or via the system property "-Dcom.amazonaws.sdk.enableThrottledRetry".
  • Fixed String case conversion issues when running with non English locales.
  • AWS SDK for Java introduces a new dynamic endpoint system that can compute endpoints for services in new regions.
  • Introducing a new AWS region, ap-northeast-2.
  • Added a new metric, HttpSocketReadTime, that records socket read latency. You can enable this metric by adding enableHttpSocketReadMetric to the system property com.amazonaws.sdk.enableDefaultMetrics. For more information, see Enabling Metrics with the AWS SDK for Java.
  • New Client Execution timeout feature to set a limit spent across retries, backoffs, ummarshalling, etc. This new timeout can be specified at the client level or per request.
    Also included in this release is the ability to specify the existing HTTP Request timeout per request rather than just per client.
  • Added support for RequesterPays for all operations.
  • Ignore the 'Connection' header when generating S3 responses.
  • Allow users to generate an AmazonS3URI from a string without using URL encoding.
  • Fixed issue that prevented creating buckets when using a client configured for the s3-external-1 endpoint.
  • Amazon S3 bucket lifecycle configuration supports two new features: the removal of expired object delete markers and an action to abort incomplete multipart uploads.
  • Allow TransferManagerConfiguration to accept integer values for multipart upload threshold.
  • Copy the list of ETags before sorting Copy the list of ETags before sorting aws/aws-sdk-java#589.
  • Option to disable chunked encoding Option to disable chunked encoding aws/aws-sdk-java#586.
  • Adding retry on InternalErrors in CompleteMultipartUpload operation. Add Default Retry on InternalError during CompleteMultipartUpload aws/aws-sdk-java#538
  • Deprecated two APIs : AmazonS3#changeObjectStorageClass and AmazonS3#setObjectRedirectLocation.
  • Added support for the aws-exec-read canned ACL. Owner gets FULL_CONTROL. Amazon EC2 gets READ access to GET an Amazon Machine Image (AMI) bundle from Amazon S3.
  • Added support for referencing security groups in peered Virtual Private Clouds (VPCs). For more information see the service announcement at https://aws.amazon.com/about-aws/whats-new/2016/03/announcing-support-for-security-group-references-in-a-peered-vpc/ .
  • Fixed a bug in AWS SDK for Java - Amazon EC2 module that returns NPE for dry run requests.
  • Regenerated client with new implementation of code generator.
  • This feature enables support for DNS resolution of public hostnames to private IP addresses when queried over ClassicLink. Additionally, you can now access private hosted zones associated with your VPC from a linked EC2-Classic instance. ClassicLink DNS support makes it easier for EC2-Classic instances to communicate with VPC resources using public DNS hostnames.
  • You can now use Network Address Translation (NAT) Gateway, a highly available AWS managed service that makes it easy to connect to the Internet from instances within a private subnet in an AWS Virtual Private Cloud (VPC). Previously, you needed to launch a NAT instance to enable NAT for instances in a private subnet. Amazon VPC NAT Gateway is available in the US East (N. Virginia), US West (Oregon), US West (N. California), EU (Ireland), Asia Pacific (Tokyo), Asia Pacific (Singapore), and Asia Pacific (Sydney) regions. To learn more about Amazon VPC NAT, see New - Managed NAT (Network Address Translation) Gateway for AWS
  • A default read timeout is now applied when querying data from EC2 metadata service.

Defaults to true.
If anyone is having trouble with this option, you could disable it with cloud.aws.s3.throttle_retries: false in elasticsearch.yml file.

Backport of elastic/elasticsearch#17784 for 1.7 series

@dadoonet
Update aws sdk to 1.10.69 and add cloud.aws.s3.throttle_retries setting
33366f9 
This PR brings 2 changes:

* Upgrade to AWS SDK 1.10.69
* Add cloud.aws.s3.throttle_retries setting

* Moving from JSON.org to Jackson for request marshallers.
* The Java SDK now supports retry throttling to limit the rate of retries during periods of reduced availability. This throttling behavior can be enabled via ClientConfiguration or via the system property "-Dcom.amazonaws.sdk.enableThrottledRetry".
* Fixed String case conversion issues when running with non English locales.
* AWS SDK for Java introduces a new dynamic endpoint system that can compute endpoints for services in new regions.
* Introducing a new AWS region, ap-northeast-2.
* Added a new metric, HttpSocketReadTime, that records socket read latency. You can enable this metric by adding enableHttpSocketReadMetric to the system property com.amazonaws.sdk.enableDefaultMetrics. For more information, see [Enabling Metrics with the AWS SDK for Java](https://java.awsblog.com/post/Tx3C0RV4NRRBKTG/Enabling-Metrics-with-the-AWS-SDK-for-Java).
* New Client Execution timeout feature to set a limit spent across retries, backoffs, ummarshalling, etc. This new timeout can be specified at the client level or per request.
  Also included in this release is the ability to specify the existing HTTP Request timeout per request rather than just per client.

* Added support for RequesterPays for all operations.
* Ignore the 'Connection' header when generating S3 responses.
* Allow users to generate an AmazonS3URI from a string without using URL encoding.
* Fixed issue that prevented creating buckets when using a client configured for the s3-external-1 endpoint.
* Amazon S3 bucket lifecycle configuration supports two new features: the removal of expired object delete markers and an action to abort incomplete multipart uploads.
* Allow TransferManagerConfiguration to accept integer values for multipart upload threshold.
* Copy the list of ETags before sorting aws/aws-sdk-java#589.
* Option to disable chunked encoding aws/aws-sdk-java#586.
* Adding retry on InternalErrors in CompleteMultipartUpload operation. aws/aws-sdk-java#538
* Deprecated two APIs : AmazonS3#changeObjectStorageClass and AmazonS3#setObjectRedirectLocation.
* Added support for the aws-exec-read canned ACL. Owner gets FULL_CONTROL. Amazon EC2 gets READ access to GET an Amazon Machine Image (AMI) bundle from Amazon S3.

* Added support for referencing security groups in peered Virtual Private Clouds (VPCs). For more information see the service announcement at https://aws.amazon.com/about-aws/whats-new/2016/03/announcing-support-for-security-group-references-in-a-peered-vpc/ .
* Fixed a bug in AWS SDK for Java - Amazon EC2 module that returns NPE for dry run requests.
* Regenerated client with new implementation of code generator.
* This feature enables support for DNS resolution of public hostnames to private IP addresses when queried over ClassicLink. Additionally, you can now access private hosted zones associated with your VPC from a linked EC2-Classic instance. ClassicLink DNS support makes it easier for EC2-Classic instances to communicate with VPC resources using public DNS hostnames.
* You can now use Network Address Translation (NAT) Gateway, a highly available AWS managed service that makes it easy to connect to the Internet from instances within a private subnet in an AWS Virtual Private Cloud (VPC). Previously, you needed to launch a NAT instance to enable NAT for instances in a private subnet. Amazon VPC NAT Gateway is available in the US East (N. Virginia), US West (Oregon), US West (N. California), EU (Ireland), Asia Pacific (Tokyo), Asia Pacific (Singapore), and Asia Pacific (Sydney) regions. To learn more about Amazon VPC NAT, see [New - Managed NAT (Network Address Translation) Gateway for AWS](https://aws.amazon.com/blogs/aws/new-managed-nat-network-address-translation-gateway-for-aws/)
* A default read timeout is now applied when querying data from EC2 metadata service.

Defaults to `true`.
If anyone is having trouble with this option, you could disable it with `cloud.aws.s3.throttle_retries: false` in `elasticsearch.yml` file.

Backport of elastic/elasticsearch#17784 for 1.7 series
@dadoonet dadoonet self-assigned this Apr 28, 2016
@dadoonet dadoonet added this to the 2.7.2 milestone Apr 28, 2016
@dadoonet
Copy link
Contributor Author

dadoonet commented May 27, 2016

@tlrx Also updated here if you want to review.

Thanks!

@dadoonet dadoonet mentioned this pull request May 30, 2016
Closed
@tlrx
Copy link
Member

tlrx commented Jun 14, 2016

LGTM

@dadoonet dadoonet changed the title Update aws sdk to 1.10.69 and add cloud.aws.s3.throttle_retries setting Update aws sdk to 1.10.69 and add throttle_retries repository setting Jun 15, 2016
@dadoonet dadoonet merged commit dd85356 into elastic:es-1.7 Jun 16, 2016
@dadoonet dadoonet deleted the pr/17784-update-aws-sdk branch June 16, 2016 04:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@dadoonet dadoonet

Labels

2.7.2 new

Projects

None yet

Milestone

2.7.2

Development

Successfully merging this pull request may close these issues.

2 participants

@dadoonet @tlrx