Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make S3 Capability More Configurable #2418

Merged
merged 4 commits into from
Oct 11, 2017

Conversation

jamesmcclain
Copy link
Member

The val s3Client lines have been changed to def s3Client in both S3ValueReader and S3AttributeStore. Examples of how to override them have been included, as well.

Connects #2304

@echeipesh
Copy link
Contributor

echeipesh commented Oct 10, 2017

+💯 on adding the documentation example

@pomadchin
Copy link
Member

Does it break binary compatibility and do we care about it too much? (well nobody called changed things in a client code i believe) 💃

@echeipesh
Copy link
Contributor

echeipesh commented Oct 10, 2017

It shouldn't but loading it up to inspect it using the mima plugin atm

@pomadchin
Copy link
Member

Looks like it's clean.

s3 >  mimaReportBinaryIssues
[info] Resolving com.amazonaws#jmespath-java;1.11.92 ...
[info] geotrellis-s3: found 8 potential binary incompatibilities while checking against org.locationtech.geotrellis:geotrellis-s3_2.11:1.1.0 
[error]  * method apply(org.apache.spark.rdd.RDD,scala.Function2,geotrellis.spark.io.s3.S3GeoTiffRDD#Options,geotrellis.spark.io.RasterReader)org.apache.spark.rdd.RDD in object geotrellis.spark.io.s3.S3GeoTiffRDD does not have a correspondent in current version
[error]    filter with: ProblemFilters.exclude[DirectMissingMethodProblem]("geotrellis.spark.io.s3.S3GeoTiffRDD.apply")
[error]  * synthetic method geotrellis$spark$io$s3$S3RDDReader$_setter_$DefaultThreadCount_=(Int)Unit in trait geotrellis.spark.io.s3.S3RDDReader is present only in current version
[error]    filter with: ProblemFilters.exclude[ReversedMissingMethodProblem]("geotrellis.spark.io.s3.S3RDDReader.geotrellis$spark$io$s3$S3RDDReader$_setter_$DefaultThreadCount_=")
[error]  * method DefaultThreadCount()Int in trait geotrellis.spark.io.s3.S3RDDReader is present only in current version
[error]    filter with: ProblemFilters.exclude[ReversedMissingMethodProblem]("geotrellis.spark.io.s3.S3RDDReader.DefaultThreadCount")
[error]  * synthetic method update$default$7()Int in trait geotrellis.spark.io.s3.S3RDDWriter is present only in current version
[error]    filter with: ProblemFilters.exclude[ReversedMissingMethodProblem]("geotrellis.spark.io.s3.S3RDDWriter.update$default$7")
[error]  * method update(org.apache.spark.rdd.RDD,java.lang.String,scala.Function1,scala.Option,scala.Option,scala.Function1,Int,geotrellis.spark.io.avro.AvroRecordCodec,scala.reflect.ClassTag,geotrellis.spark.io.avro.AvroRecordCodec,scala.reflect.ClassTag)Unit in trait geotrellis.spark.io.s3.S3RDDWriter is present only in current version
[error]    filter with: ProblemFilters.exclude[ReversedMissingMethodProblem]("geotrellis.spark.io.s3.S3RDDWriter.update")
[error]  * synthetic method update$default$6()scala.Function1 in trait geotrellis.spark.io.s3.S3RDDWriter is present only in current version
[error]    filter with: ProblemFilters.exclude[ReversedMissingMethodProblem]("geotrellis.spark.io.s3.S3RDDWriter.update$default$6")
[error]  * synthetic method geotrellis$spark$io$s3$S3RDDWriter$_setter_$DefaultThreadCount_=(Int)Unit in trait geotrellis.spark.io.s3.S3RDDWriter is present only in current version
[error]    filter with: ProblemFilters.exclude[ReversedMissingMethodProblem]("geotrellis.spark.io.s3.S3RDDWriter.geotrellis$spark$io$s3$S3RDDWriter$_setter_$DefaultThreadCount_=")
[error]  * method DefaultThreadCount()Int in trait geotrellis.spark.io.s3.S3RDDWriter is present only in current version
[error]    filter with: ProblemFilters.exclude[ReversedMissingMethodProblem]("geotrellis.spark.io.s3.S3RDDWriter.DefaultThreadCount")
[trace] Stack trace suppressed: run last s3/*:mimaReportBinaryIssues for the full output.
[error] (s3/*:mimaReportBinaryIssues) geotrellis-s3: Binary compatibility check failed!
[error] Total time: 7 s, completed Oct 10, 2017 6:36:56 PM

@echeipesh echeipesh merged commit e5ddd78 into locationtech:master Oct 11, 2017
@jamesmcclain jamesmcclain deleted the fix/s3 branch October 11, 2017 15:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants