Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Normalize URI paths in RestClient #14423

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Xtansia
Copy link
Contributor

@Xtansia Xtansia commented Jun 18, 2024

Description

There are a sprinkling of locations such as this line that construct a RestClient request with a non-absolute path (ie. no leading /). This is being passed verbatim to the HTTP request which results in a technically invalid request start-line of GET _render/template HTTP/1.1 which should be GET /_render/template HTTP/1.1. OpenSearch itself is lenient and happily serves this request, however in cases of stricter intermediaries such as proxies or load-balances as in Amazon OpenSearch Service, this request ends up denied with a 400 Bad Request.

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link
Contributor

❌ Gradle check result for 0d562b7: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for fa00762: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

fullPath = path;
}

String fullPath = buildUriPath(pathPrefix, path);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like we just need to return new URI("/").resolve(uriBuilder.build().toASCIIString()); to make sure URI is always resolved properly:

 new URI("/").resolve("/_render/template") ==> /_render/template
 new URI("/").resolve("_render/template") ==> /_render/template

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ended up tweaking the URIBuilder construction to use its appending rather than building up the full path manually which then takes care of prefixing the /.

@Xtansia Xtansia added the backport 2.x Backport to 2.x branch label Jun 18, 2024
Copy link
Contributor

❌ Gradle check result for c2349ff: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

fullPath = path;
URIBuilder uriBuilder = new URIBuilder();

if (pathPrefix != null && !pathPrefix.isEmpty() && !"/".equals(pathPrefix)) {
Copy link
Collaborator

@reta reta Jun 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure the equality checks are suitable here, fe passing // as path would be a miss, the previous code definitely was more robust towards such cases.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case I was skipping a single / due to the logic of URIBuilder.appendPath:

new URIBuilder().appendPath("/").appendPath("/") -> //
new URIBuilder().appendPath("/foo").appendPath("/bar") -> /foo/bar

I've combined this URIBuilder.appendPath approach with the previous trimSlashes implementation and some extra test cases to see how we feel about this level of normalization. Though this will strip duplicate / in leading or trailing position, it won't touch foo////bar. If we would want to normalize that, then this could be changed to instead split on / and skip empty segments.

@Xtansia Xtansia force-pushed the feat/restclient-normalize-paths branch from c2349ff to 0d33093 Compare June 18, 2024 23:29
Copy link
Contributor

❌ Gradle check result for 0d33093: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@Xtansia Xtansia force-pushed the feat/restclient-normalize-paths branch from 0d33093 to 2b8571b Compare June 18, 2024 23:36
Copy link
Contributor

❌ Gradle check result for 2b8571b: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@Xtansia Xtansia force-pushed the feat/restclient-normalize-paths branch from 2b8571b to a59b21e Compare June 18, 2024 23:48
Copy link
Contributor

❌ Gradle check result for a59b21e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@Xtansia
Copy link
Contributor Author

Xtansia commented Jun 19, 2024

There's two tests failing with these changes:

Would be great to hear your thoughts on these two cases @reta?

@reta
Copy link
Collaborator

reta commented Jun 19, 2024

There's two tests failing with these changes:

Thanks @Xtansia I think for main (3.0.0) we could change the tests to reflect the new normalization logic, but I would not recommend to backport this change to 2.x branch - it leads to the change in behavior (it is breaking).

@Xtansia Xtansia marked this pull request as draft June 19, 2024 23:01
@Xtansia Xtansia removed the backport 2.x Backport to 2.x branch label Jun 21, 2024
@opensearch-trigger-bot
Copy link
Contributor

This PR is stalled because it has been open for 30 days with no activity.

@opensearch-trigger-bot opensearch-trigger-bot bot added the stalled Issues that have stalled label Jul 22, 2024
@dblock
Copy link
Member

dblock commented Jul 22, 2024

@Xtansia Want to finish this?

@opensearch-trigger-bot opensearch-trigger-bot bot removed the stalled Issues that have stalled label Jul 23, 2024
@opensearch-trigger-bot
Copy link
Contributor

This PR is stalled because it has been open for 30 days with no activity.

@opensearch-trigger-bot opensearch-trigger-bot bot added the stalled Issues that have stalled label Aug 22, 2024
Signed-off-by: Thomas Farr <tsfarr@amazon.com>
@Xtansia Xtansia force-pushed the feat/restclient-normalize-paths branch from a59b21e to 8a6f900 Compare August 22, 2024 21:19
Copy link
Contributor

❌ Gradle check result for 8a6f900: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@opensearch-trigger-bot opensearch-trigger-bot bot removed the stalled Issues that have stalled label Aug 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants