New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rasters write support to HDFS / S3 #2102

Merged
merged 7 commits into from Apr 3, 2017

Conversation

Projects
None yet
3 participants
@pomadchin
Member

pomadchin commented Mar 27, 2017

Hadoop based PNG / JPG / GeoTiff images write. Fixes #2080

  • tests

@pomadchin pomadchin changed the title from Rasters write support to HDFS / S3 to [WIP] Rasters write support to HDFS / S3 Mar 27, 2017

PNG / JPG / GeoTiff hadoop write methods
Signed-off-by: Grigory Pomadchin <gr.pomadchin@gmail.com>

@pomadchin pomadchin force-pushed the pomadchin:feature/generic-raster-writers branch from cde2e71 to a77e695 Mar 27, 2017

pomadchin added some commits Mar 28, 2017

add geotifff tests
Signed-off-by: Grigory Pomadchin <gr.pomadchin@gmail.com>
fix tests
Signed-off-by: Grigory Pomadchin <gr.pomadchin@gmail.com>

@pomadchin pomadchin force-pushed the pomadchin:feature/generic-raster-writers branch from b007766 to da40aa3 Mar 28, 2017

@pomadchin pomadchin changed the title from [WIP] Rasters write support to HDFS / S3 to Rasters write support to HDFS / S3 Mar 28, 2017

@pomadchin pomadchin removed the in progress label Mar 28, 2017

add geotiff reader into codebase
Signed-off-by: Grigory Pomadchin <gr.pomadchin@gmail.com>

@pomadchin pomadchin force-pushed the pomadchin:feature/generic-raster-writers branch from da40aa3 to 68b7eb0 Mar 28, 2017

@lossyrob lossyrob self-requested a review Mar 29, 2017

@pomadchin pomadchin added this to the 1.1 milestone Mar 30, 2017

add missing tests and Jpg / Png readers
Signed-off-by: Grigory Pomadchin <gr.pomadchin@gmail.com>

@pomadchin pomadchin force-pushed the pomadchin:feature/generic-raster-writers branch from 0d7ead8 to da84e38 Mar 31, 2017

@@ -59,7 +59,8 @@ package object spark
with stitch.Implicits
with summary.Implicits
with summary.polygonal.Implicits
with tiling.Implicits {
with tiling.Implicits
with spark.raster.io.Implicits {

This comment has been minimized.

@lossyrob

lossyrob Mar 31, 2017

Member

This breaks the pattern. This should be on the import geotrellis.spark.io._ package.

import org.apache.hadoop.fs.Path
import org.apache.spark.SparkContext
object GeoTiffHadoopReader {

This comment has been minimized.

@lossyrob

lossyrob Mar 31, 2017

Member

Should be in hadoop subproject

Refactored functions namings / packages, removed explicit gzip flag.
Signed-off-by: Grigory Pomadchin <gr.pomadchin@gmail.com>

@pomadchin pomadchin force-pushed the pomadchin:feature/generic-raster-writers branch from e0eef5d to db86ac5 Mar 31, 2017

refactor r / w hadoop methods
Signed-off-by: Grigory Pomadchin <gr.pomadchin@gmail.com>

@pomadchin pomadchin force-pushed the pomadchin:feature/generic-raster-writers branch from c7547ff to cf14df3 Apr 3, 2017

@echeipesh

This comment has been minimized.

Contributor

echeipesh commented Apr 3, 2017

This looks good and comment from @lossyrob are addressed.

@lossyrob lossyrob merged commit 490c6c2 into locationtech:master Apr 3, 2017

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
ip-validation
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment