You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm trying to transfer multiple files between S3 buckets using the EDC connector.
Although I've specified a destination path in the 'objectPrefix' (or 'objectName'), the EDC Data Plane seems to ignore it and attempts to upload the files back to the source provider's directory instead. This results in a 403 Access Denied error because the consumer's IAM user does not have permission to write to the provider's folder.
I have confirmed that the consumer is correctly selected.
Below are the details of the transfer status and the error log.
Could anyone help me identify why the EDC connector is defaulting to the source path instead of using the specified destination prefix?
DEBUG 2026-01-08T16:27:55.626370629 Transferring from AmazonS3 to AmazonS3 for flow id: xxxx-xxxx-xxxx-xxxx-xxxx.
DEBUG 2026-01-08T16:27:55.626658982 [DataPlaneManagerImpl] DataFlow xxxx-xxxx-xxxx-xxxx-xxxx is now in state STARTED
SEVERE 2026-01-08T16:27:56.051116197 Failed to upload the develop/provider1/multidir/foldername/xxxx.json object: User: arn:aws:iam::00000000:user/xxxx-xxxx is not authorized to perform: s3:PutObject on resource: "arn:aws:s3:::00000000-xxxxxx-xxxxxx-bucket/develop/provider1/multidir/foldername/xxxx.json" because no identity-based policy allows the
s3:PutObject action (Service: S3, Status Code: 403, Request ID: XXXXXXXXXXXXXX, Extended Request ID: XXXXXXXXXXXXXX) (SDK Attempt Count: 1)
software.amazon.awssdk.services.s3.model.S3Exception: User: arn:aws:iam::00000000:user/xxxxxx-consumer1 is not authorized to perform: s3:PutObject on resource: "arn:aws:s3:::00000000-xxxxxx-xxxxxx-bucket/develop/provider1/multidir/foldername/dataset.json" because no identity-based policy allows the s3:PutObject action (Service: S3, Status Code: 403, Request ID:XXXXXXXXXXXXXX, Extended Request ID: XXXXXXXXXXXXXX) (SDK Attempt Count: 1)
at software.amazon.awssdk.services.s3.model.S3Exception$BuilderImpl.build(S3Exception.java:113)
at software.amazon.awssdk.services.s3.model.S3Exception$BuilderImpl.build(S3Exception.java:61)
at software.amazon.awssdk.core.internal.http.pipeline.stages.utils.RetryableStageHelper.retryPolicyDisallowedRetryException(RetryableStageHelper.java:168)
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:73)
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.ex
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
-
I'm trying to transfer multiple files between S3 buckets using the EDC connector.
Although I've specified a destination path in the 'objectPrefix' (or 'objectName'), the EDC Data Plane seems to ignore it and attempts to upload the files back to the source provider's directory instead. This results in a 403 Access Denied error because the consumer's IAM user does not have permission to write to the provider's folder.
I have confirmed that the consumer is correctly selected.
Below are the details of the transfer status and the error log.
Could anyone help me identify why the EDC connector is defaulting to the source path instead of using the specified destination prefix?
EDC v0.14.1
Transfer response log
provider connecter log
Beta Was this translation helpful? Give feedback.
All reactions