fix: resolve failure in uploading files to a presigned AWS S3 upload url #1645
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Merge upstream
lucian-torje-siemens
requested review from
bscholtes1A and
jimmarino
as code owners
juliapampus
requested changes
Jul 11, 2022
jimmarino
reviewed
Jul 12, 2022
...va/org/eclipse/dataspaceconnector/dataplane/http/pipeline/NonChunkedTransferRequestBody.java
Outdated
Show resolved
Hide resolved
jimmarino
reviewed
Jul 12, 2022
.../src/main/java/org/eclipse/dataspaceconnector/dataplane/http/pipeline/HttpRequestParams.java
Outdated
Show resolved
Hide resolved
jimmarino
reviewed
Jul 12, 2022
.../src/main/java/org/eclipse/dataspaceconnector/dataplane/http/pipeline/HttpRequestParams.java
Outdated
Show resolved
Hide resolved
jimmarino
reviewed
Jul 12, 2022
...n/java/org/eclipse/dataspaceconnector/dataplane/http/pipeline/HttpRequestParamsSupplier.java
Outdated
Show resolved
Hide resolved
jimmarino
reviewed
Jul 12, 2022
...n/java/org/eclipse/dataspaceconnector/dataplane/http/pipeline/HttpRequestParamsSupplier.java
Outdated
Show resolved
Hide resolved
jimmarino
reviewed
Jul 12, 2022
...va/org/eclipse/dataspaceconnector/dataplane/http/pipeline/HttpSinkRequestParamsSupplier.java
Outdated
Show resolved
Hide resolved
jimmarino
reviewed
Jul 12, 2022
.../org/eclipse/dataspaceconnector/dataplane/http/pipeline/HttpSourceRequestParamsSupplier.java
Outdated
Show resolved
Hide resolved
jimmarino
reviewed
Jul 12, 2022
...va/org/eclipse/dataspaceconnector/dataplane/http/pipeline/NonChunkedTransferRequestBody.java
Outdated
Show resolved
Hide resolved
jimmarino
reviewed
Jul 12, 2022
...va/org/eclipse/dataspaceconnector/dataplane/http/pipeline/HttpRequestParamsSupplierTest.java
Outdated
Show resolved
Hide resolved
jimmarino
reviewed
Jul 12, 2022
spi/core-spi/src/main/java/org/eclipse/dataspaceconnector/spi/types/domain/HttpDataAddress.java
Outdated
Show resolved
Hide resolved
jimmarino
reviewed
Jul 12, 2022
spi/core-spi/src/main/java/org/eclipse/dataspaceconnector/spi/types/domain/HttpDataAddress.java
Outdated
Show resolved
Hide resolved
jimmarino
reviewed
Jul 12, 2022
.../src/main/java/org/eclipse/dataspaceconnector/dataplane/http/pipeline/HttpRequestParams.java
Outdated
Show resolved
Hide resolved
jimmarino
reviewed
Jul 12, 2022
...va/org/eclipse/dataspaceconnector/dataplane/http/pipeline/HttpSinkRequestParamsSupplier.java
Outdated
Show resolved
Hide resolved
jimmarino
reviewed
Jul 12, 2022
.../java/org/eclipse/dataspaceconnector/dataplane/http/pipeline/ChunkedTransferRequestBody.java
Outdated
Show resolved
Hide resolved
7 tasks
jimmarino
reviewed
Jul 15, 2022
jimmarino
reviewed
Jul 15, 2022
...va/org/eclipse/dataspaceconnector/dataplane/http/pipeline/NonChunkedTransferRequestBody.java
Outdated
Show resolved
Hide resolved
jimmarino
approved these changes
Jul 19, 2022
|
@lucian-torje-siemens can you fix the conflicts and we'll merge? |
juliapampus
approved these changes
Jul 20, 2022
|
@lucian-torje-siemens, if you can resolve the conflicts by COB today, we can include this in the release. Otherwise, we'll put it in the next one. |
juliapampus
changed the title
|
@lucian-torje-siemens Can you pls take a look why the pipeline is failing? |
|
@juliapampus @jimmarino I'm back and checking what caused the issue - first finding: faker library change in main |
jimmarino
approved these changes
Aug 4, 2022
|
Is there any update, colleagues? we are looking forward to finilizing this pull request.. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR changes/adds
It adds a new switch for http dataplane sink which can be used to choose between chunked (streamed) transfer and non chunked transfer (supported by AWS S3 presigned upload url).
Why it does that
AWS S3 upload on presigned url does not work when header Transfer-Encoding is present (added automatically by OkHttp).
see https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Transfer-Encoding
Note: HTTP/2 doesn't support HTTP 1.1's chunked transfer encoding mechanism, as it provides its own, more efficient, mechanisms for data streaming.
Further notes
Http request parameter changed
Linked Issue(s)
Closes #1643
Checklist
no-changelog)