Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adding x-amz-content-sha256 header for signed requests #339

Merged
merged 8 commits into from
Jan 24, 2023
Merged

Adding x-amz-content-sha256 header for signed requests #339

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
bfc1ac1
Adding X-Amz-Content-Sha256 header for signed requests
VachaShah Jan 21, 2023
81f1045
Adding CHANGELOG entry
VachaShah Jan 21, 2023
301e2e4
Adding documentation comment
VachaShah Jan 21, 2023
4d917fe
Adding tests
VachaShah Jan 23, 2023
e88486c
Addressing comments
VachaShah Jan 23, 2023
eed32d5
Addressing comments
VachaShah Jan 24, 2023
57492fc
Removing refresh policy for integ tests for Sigv4
VachaShah Jan 24, 2023
66c4eb1
Updating the developer guide
VachaShah Jan 24, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
- Fix version and build ([#254](https://github.com/opensearch-project/opensearch-java/pull/254))
- Update Gradle to 7.6 ([#309](https://github.com/opensearch-project/opensearch-java/pull/309))
- Prevent SPI calls at runtime ([#293](https://github.com/opensearch-project/opensearch-java/pull/293))
- Add support for OpenSearch Serverless ([#339](https://github.com/opensearch-project/opensearch-java/pull/339))

### Deprecated

Expand Down
6 changes: 4 additions & 2 deletions USER_GUIDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
- [Sample data](#sample-data)
- [IndexData class](#indexdata-class)
- [Create a client](#create-a-client)
- [Create a client using `RestClientTransport`](#create-a-client-using-restclienttransport)
- [Create a client using `ApacheHttpClient5Transport`](#create-a-client-using-apachehttpclient5transport)
- [Create an index](#create-an-index)
- [Index data](#index-data)
- [Search for the documents](#search-for-the-documents)
Expand Down Expand Up @@ -175,7 +177,7 @@ DeleteIndexResponse deleteIndexResponse = client.indices().delete(deleteIndexReq

## Amazon Managed OpenSearch

Use `AwsSdk2Transport` to make requests to Amazon Managed OpenSearch.
Use `AwsSdk2Transport` to make requests to Amazon Managed OpenSearch and OpenSearch Serverless.

```java
SdkHttpClient httpClient = ApacheHttpClient.builder().build();
Expand All @@ -184,7 +186,7 @@ OpenSearchClient client = new OpenSearchClient(
new AwsSdk2Transport(
httpClient,
"search-...us-west-2.es.amazonaws.com", // OpenSearch endpoint, without https://
"es" // signing service name
"es" // signing service name, use "aoss" for OpenSearch Serverless
Region.US_WEST_2, // signing service region
AwsSdk2TransportOptions.builder().build()
)
Expand Down
3 changes: 3 additions & 0 deletions java-client/src/main/java/org/opensearch/client/transport/aws/AwsSdk2Transport.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -363,6 +363,9 @@ private <RequestT> SdkHttpFullRequest prepareRequest(
}
req.putHeader("Content-Length", String.valueOf(body.getContentLength()));
req.contentStreamProvider(body::getInputStream);
// To add the "X-Amz-Content-Sha256" header, it needs to set as required.
// It is a required header for Amazon OpenSearch Serverless.
req.putHeader("x-amz-content-sha256", "required");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What a weird API! Took me a while to find https://github.com/aws/aws-sdk-java-v2/blob/0510a17ae41d601cf5f03e7af01e4519a6b3a744/core/auth/src/main/java/software/amazon/awssdk/auth/signer/internal/AbstractAws4Signer.java#L97 that handles this.

Is there a higher level method we can use like https://github.com/aws/aws-sdk-java-v2/blob/0510a17ae41d601cf5f03e7af01e4519a6b3a744/core/auth/src/main/java/software/amazon/awssdk/auth/signer/Aws4UnsignedPayloadSigner.java#L69 so we don't have to hard-code "required"?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah it took me a while to find too. We use Aws4Signer which does not have the sign method overriden like Aws4UnsignedPayloadSigner. According to documentation, the Aws4UnsignedPayloadSigner is similar to Aws4Signer but just adds UNSIGNED-PAYLOAD when protocol is HTTPS. Can we use Aws4UnsignedPayloadSigner?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it works we sure can I think.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried the Aws4UnsignedPayloadSigner but it does not work for Amazon OpenSearch Service since it signs the payload with UNSIGNED_PAYLOAD over https protocol. Works for Amazon OpenSearch Serverless. I can PR this change on their repo may be as a new signer class but for now looks like we might have to use the hard-coded "required".

}

boolean responseCompression = Optional.ofNullable(options)
Expand Down