Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3: Consider not deprecating path-style access #2364

Closed
mpdn opened this issue Jul 24, 2020 · 6 comments
Closed

S3: Consider not deprecating path-style access #2364

mpdn opened this issue Jul 24, 2020 · 6 comments
Labels
Milestone

Comments

@mpdn
Copy link
Contributor

mpdn commented Jul 24, 2020

While AWS is deprecating path-style access, many other providers of S3-like services are not. Path-style access is usually the easiest way to setup MinIO as it does not require fudging around with the DNS.

So while path-style access for AWS should be deprecated, path-style access for custom endpoints are still relevant.

For the same reason, it would be nice to have a way to disable the warning here:

"AWS S3 is going to retire path-style access (https://aws.amazon.com/blogs/aws/amazon-s3-path-deprecation-plan-the-rest-of-the-story/)"

I would be willing to put some work into this. The simplest way of achieving this would probably be something like un-deprecating the withPathStyleAccess function and remove the warning for custom endpoints.

@ennru ennru added the p:aws-s3 label Aug 3, 2020
@ennru
Copy link
Member

ennru commented Aug 3, 2020

You are right, for compatible storage solutions Alpakka should continue to allow path-style access in one way or the other.

Would it be possible to use the enpoint-url setting to achieve something without removing the deprecation?

# Custom endpoint url, used for alternate s3 implementations
# To enable virtual-host-style access with Alpakka S3 use the placeholder `{bucket}` in the URL
# eg. endpoint-url = "http://{bucket}.s3minio.alpakka:9000"
#
# endpoint-url = null

@KGausel
Copy link

KGausel commented Aug 17, 2020

@ennru I tried that, but maybe I configured it wrong. Do you have any input on how to make it work with MinIO using the endpoint-url template method you are suggesting?

I tried http://myminio.local/{bucket} and http://myminio.local/{bucket}/

@ennru
Copy link
Member

ennru commented Aug 29, 2020

Alpakka uses Minio with virtual-host style access. The trick is to configure a forward-proxy and a dynamic enpoint-url:

override def attributes(s3Settings: S3Settings): Attributes = {
S3Attributes.settings(
s3Settings
.withForwardProxy(
ForwardProxy.http("localhost", 9000)
)
)
}
it should "properly set the endpointUrl" in {
S3Settings().endpointUrl.value shouldEqual endpointUrlVirtualHostStyle
}
}
object MinioS3IntegrationSpec {
val accessKey = "TESTKEY"
val secret = "TESTSECRET"
val endpointUrlPathStyle = "http://localhost:9000"
val localMinioDomain = "s3minio.alpakka"
val endpointUrlVirtualHostStyle = s"http://{bucket}.$localMinioDomain:9000"

@KGausel
Copy link

KGausel commented Aug 31, 2020

This might not be an option for all users though and seems like an overly complicated way to stop Alpakka from spamming log warnings of path style access being deprecated.

@ennru
Copy link
Member

ennru commented Sep 3, 2020

@KGausel Good news, @laszlovandenhoek just convinced me that we can should [re-think the path-style access settings](#2193 (comment)

@ennru ennru added this to the 2.0.2 milestone Sep 7, 2020
@ennru
Copy link
Member

ennru commented Sep 7, 2020

With #2392 the deprecation shows only if you don't use an endpointUrl thus use AWS' services.
This cleaner solution resolved your request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants