-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Move to AWS S3 SDK #4289
Comments
+1, aws-java-sdk seems better than jets3t these days. Related: #4116 which would add support for s3a (aws-java-sdk backed S3) to Hadoop indexing. |
There are some in the community who use jets3t.properties, so such a change would either need a migration plan or a major version update |
My feeling is it's not worth supporting both at the same time, and instead we should do it in 0.11.0 and tell people that jets3t properties will no longer be respected (but whatever new aws-java-sdk configurator thing will). |
@gianm agreed. also it wasn't clear in my prior message, but I think moving to the AWS sdk is a great idea. |
Duplicate of #3886 |
Labelling |
@leventov is the solution for this one going to have to have a config that creates a https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/index.html?com/amazonaws/ClientConfiguration.html , or do you know of a way to have a |
@drcrallen I don't know, but creating a config via "Druid native" properties sounds like a good solution, for unification. |
Related #4382 |
I'm working on this. |
Cool! The endpoint configuration should be trivial with the new AWSClient-builder thingy. That just needs to be propagated trough-out the stack. |
I plan to move to using AWS S3 SDK instead of using JetS3T. I have noticed the following problems with JetS3T:
What do you think @gianm @leventov @cheddar ?
The text was updated successfully, but these errors were encountered: