Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Amazon S3 custom endpoint results in invalid region, causing publishing to fail #1464

Open
Tinytimrob opened this issue Feb 25, 2017 · 2 comments

Comments

Projects
None yet
4 participants
@Tinytimrob
Copy link

commented Feb 25, 2017

Expected Behavior

When using the system property "org.gradle.s3.endpoint" to specify a custom endpoint for S3 publishing or retrieval (e.g. -Dorg.gradle.s3.endpoint=https://s3.example.com/) the default AWS region of US East 1 should be used, and/or it should be possible to specify the region of the custom endpoint.

Current Behavior

A made-up invalid region is used based on the provided hostname of the custom endpoint, e.g. for endpoint s3.example.com, the region sent to the server when publishing is "example"

Context

The above bug renders it impossible to publish an artifact to a custom endpoint if the server rejects signatures with an invalid region. For example, if trying to publish to a minio server, the following error occurs:

com.amazonaws.services.s3.model.AmazonS3Exception: Region does not match. (Service: Amazon S3; Status Code: 400; Error Code: InvalidRegion; Request ID: 3L137), S3 Extended Request ID: 3L137
        at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1305)
        at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:852)
        at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:630)
        at com.amazonaws.http.AmazonHttpClient.doExecute(AmazonHttpClient.java:405)
        at com.amazonaws.http.AmazonHttpClient.executeWithTimer(AmazonHttpClient.java:367)
        at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:318)
        at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3787)
        at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1137)
        at org.gradle.internal.resource.transport.aws.s3.S3Client.doGetS3Object(S3Client.java:194)

Workarounds

1 - I changed the configuration of the minio server to set the configured region to the invalid one that Gradle chose to use (e.g. 'example'). This allowed Gradle to connect and the publish worked. Unfortunately, this broke other software that expected the region to be valid (e.g. Cyberduck is unable to create new buckets or upload files to minio in this configuration, because it expects the region to be set to the default region of US East 1 on custom endpoints)

2 - I created a new custom endpoint with a URL that had the format "s3-us-east-1.example.com" and pointed Gradle there instead. This caused Gradle to determine the region name "us-east-1" from the hostname, so publishing succeeded.

Steps to Reproduce

Specify a custom endpoint using the system property and attempt to publish. e.g.:
gradlew publish --stacktrace -Dorg.gradle.s3.endpoint=https://s3.example.com/

Your Environment

Gradle 3.4

@Tinytimrob

This comment has been minimized.

Copy link
Author

commented Feb 25, 2017

Want to add to this, although it's not a bug: Having to pass the custom endpoint as a system property on the command line was not intuitive. I was expecting to be able to specify the property in gradle.properties, but that doesn't work.

@n0mer

This comment has been minimized.

Copy link

commented Feb 4, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.