Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sending data to an AWS S3 presigned upload url does not work #1643

Closed
lucian-torje-siemens opened this issue Jul 11, 2022 · 0 comments · Fixed by #1645
Closed

Sending data to an AWS S3 presigned upload url does not work #1643

lucian-torje-siemens opened this issue Jul 11, 2022 · 0 comments · Fixed by #1645
Labels
bug Something isn't working
Projects
Connector

Comments

@lucian-torje-siemens
Copy link
Contributor

lucian-torje-siemens commented Jul 11, 2022 •
edited
Loading

Bug Report

Describe the Bug

AWS complains with 501 Not Implemented and response body like:

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>NotImplemented</Code><Message>A header you provided implies functionality that is not implemented</Message><Header>Transfer-Encoding</Header><RequestId>qwerty</RequestId><HostId>qwerty</HostId></Error>

Expected Behavior

Response code 200 OK when using dataplane sink to push data to AWS S3

Observed Behavior

Failure to create the new file due to 501 http status

Steps to Reproduce

Steps to reproduce the behavior:

  1. Create asset with data destination baseUrl as an AWS S3 presigned upload url
  2. Trigger transfer process

Context Information

Using the latest EDC code from branch main

Detailed Description

The problem is within OkHttp implementation
see OkHttp Transfer Encoding search

image

Possible Implementation

A new property called chunked: true/false that would allow sending the data as a all or as stream (in chunks)
@lucian-torje-siemens lucian-torje-siemens added the bug Something isn't working label Jul 11, 2022
@github-actions github-actions bot added this to Open in Connector Jul 11, 2022
lucian-torje-siemens added a commit to mindsphere/DataSpaceConnector-Fork that referenced this issue Jul 11, 2022
@lucian-torje-siemens
bug: sending data to an AWS S3 presigned upload url does not work ecl…
618dd15 
…ipse-edc#1643
Merged
7 tasks
lucian-torje-siemens added a commit to mindsphere/DataSpaceConnector-Fork that referenced this issue Jul 15, 2022
@lucian-torje-siemens
bug: fix sample assets and add missing properties (#29)
1897798 
* bug: sending data to an AWS S3 presigned upload url does not work eclipse-edc#1643

* fix checkstyle

* bug: fix sample assets and add missing properties

* switch build env

* fix checkstyle

* documentation

* apply review from @jimmarino

* fix checkstyle

* fix test

* apply review - rename to transferInOneGo and reverse logic

* fix test

* use transfer in one go

* update postman

* adjust policy - add more tests

* fix checkstyle

* add logging (to be reverted)

* Revert "add logging (to be reverted)"

This reverts commit 870ecd0.
Connector automation moved this from Open to Done Aug 10, 2022
ndr-brt pushed a commit that referenced this issue Aug 10, 2022
@lucian-torje-siemens @andreic94
fix: resolve failure in uploading files to a presigned AWS S3 upload …
e323243 
…url (#1645)

* bug: sending data to an AWS S3 presigned upload url does not work #1643

* fix checkstyle

* documentation

* apply review from @jimmarino

* fix checkstyle

* fix test

* apply review - rename to transferInOneGo and reverse logic

* fix test

* apply review from @jimmarino - rename to "transfer in one go" to "nonChunkedTransfer"

* apply review

* fix changelog for proper milestone

* fix NonChunkedTransferRequestBodyTest.java - use the new faker import

Co-authored-by: Andrei-Laurentiu Coman <andrei.coman@siemens.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
No open projects
Connector
  
Done
Milestone
No milestone
1 participant
@lucian-torje-siemens