You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When using GeoTiffRasterSource instance in multi-threaded context, either in spark multi-partition per executor or as part of async http4s application the following error is possible:
[info] java.lang.IllegalArgumentException
[info] at java.nio.Buffer.position(Buffer.java:244)
[info] at geotrellis.util.StreamingByteReader.ensureChunk(StreamingByteReader.scala:114)
[info] at geotrellis.util.StreamingByteReader.getBytes(StreamingByteReader.scala:118)
[info] at geotrellis.raster.io.geotiff.LazySegmentBytes.getBytes(LazySegmentBytes.scala:120)
[info] at geotrellis.raster.io.geotiff.LazySegmentBytes$$anonfun$readChunk$1.apply(LazySegmentBytes.scala:99)
[info] at geotrellis.raster.io.geotiff.LazySegmentBytes$$anonfun$readChunk$1.apply(LazySegmentBytes.scala:97)
[info] at scala.collection.immutable.List.map(List.scala:284)
[info] at geotrellis.raster.io.geotiff.LazySegmentBytes.readChunk(LazySegmentBytes.scala:97)
[info] at geotrellis.raster.io.geotiff.LazySegmentBytes$$anonfun$getSegments$1.apply(LazySegmentBytes.scala:115)
[info] at geotrellis.raster.io.geotiff.LazySegmentBytes$$anonfun$getSegments$1.apply(LazySegmentBytes.scala:115)
[info] at scala.collection.Iterator$$anon$12.nextCur(Iterator.scala:435)
[info] at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:441)
[info] at scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:462)
[info] at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
[info] at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:440)
[info] at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
[info] at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
[info] at geotrellis.contrib.vlm.geotiff.GeoTiffRasterSource.read(GeoTiffRasterSource.scala:58)
[info] at geotrellis.contrib.vlm.RasterSource$class.read(RasterSource.scala:163)
[info] at geotrellis.contrib.vlm.geotiff.GeoTiffRasterSource.read(GeoTiffRasterSource.scala:28)
This happens because underlying geotrellis.raster.io.geotiff.LazySegmentBytes nor GeoTiffTile which uses it is not thread safe. GeoTiffReader does not make promises about being thread safe, but RasterSource makes an implicit promise to be thread safe and thats a useful expectation.
For this issue I suggest:
Use synchronize on GeoTiff[MultibandTile] contained by GeoTiffRasterSource to make all read methods thread-safe
Update the documentation on RasterSource interface to state that thread-safe behavior is expected
The text was updated successfully, but these errors were encountered:
When using
GeoTiffRasterSource
instance in multi-threaded context, either in spark multi-partition per executor or as part of asynchttp4s
application the following error is possible:This happens because underlying
geotrellis.raster.io.geotiff.LazySegmentBytes
norGeoTiffTile
which uses it is not thread safe.GeoTiffReader
does not make promises about being thread safe, butRasterSource
makes an implicit promise to be thread safe and thats a useful expectation.For this issue I suggest:
synchronize
onGeoTiff[MultibandTile]
contained byGeoTiffRasterSource
to make allread
methods thread-safeRasterSource
interface to state that thread-safe behavior is expectedThe text was updated successfully, but these errors were encountered: