Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rasterio source #927

Closed
wants to merge 56 commits into from
Closed
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
4336a34
draft rasterio driver
12rambau Aug 11, 2022
07f085e
unify docstring
12rambau Aug 11, 2022
d5ee2e1
apply black formatting
12rambau Aug 11, 2022
831c6ce
flake8 checks
12rambau Aug 11, 2022
f185517
use camelCase instead of snakeCase
12rambau Aug 11, 2022
5d814b7
edit color functions
12rambau Aug 11, 2022
8f44b28
isort
12rambau Aug 11, 2022
c15d4d8
isort and black disagree
12rambau Aug 11, 2022
e223cf1
last check before draft PR
12rambau Aug 11, 2022
ff0f5c3
add the tests
12rambau Aug 11, 2022
d5bf1eb
Merge branch 'master' into rasterio
12rambau Aug 11, 2022
2fab721
add the alphatest
12rambau Aug 11, 2022
ea90ca9
add rasterio source to setup
12rambau Aug 11, 2022
21a3e0e
bug fixes
12rambau Aug 12, 2022
4145c4a
import Proj, Transformer and CRS directly
12rambau Aug 12, 2022
885f63e
use affine
12rambau Aug 12, 2022
28bf3fa
indexes is 1 indexed
12rambau Aug 12, 2022
d3f47d0
debug against tests step 1
12rambau Aug 12, 2022
7f89243
Merge branch 'master' into rasterio
banesullivan Nov 18, 2022
16af499
Formatting
banesullivan Nov 18, 2022
6160222
Make getInternalMetadata JSON serializable
banesullivan Nov 18, 2022
e3b0395
Hackily support srs in getBounds
banesullivan Nov 18, 2022
53295b4
Fix getTile when extent is larger than image
banesullivan Nov 18, 2022
892c514
Fix add_alpha issues
banesullivan Nov 18, 2022
3a8bcc8
Check metadata for None
banesullivan Nov 18, 2022
1e72b5c
Catch rio_cogeo import error
banesullivan Nov 18, 2022
4782121
format_ -> format
banesullivan Nov 18, 2022
2bbb823
Formatting
banesullivan Nov 18, 2022
e7d6b9d
tox -e format
banesullivan Nov 18, 2022
3422f68
Merge branch 'master' into rasterio
banesullivan Nov 21, 2022
5dd8f18
Autopep8 and cleanup
banesullivan Nov 21, 2022
ca287dc
Autopep8 tests
banesullivan Nov 21, 2022
9fac7df
Make GeoFileTileSource base class
banesullivan Nov 22, 2022
5c4f9e2
Merge branch 'master' into rasterio
banesullivan Nov 22, 2022
9435e77
Formatting
banesullivan Nov 22, 2022
39a87ae
Skip <python3.8 for rasterio tests
banesullivan Nov 22, 2022
21b1fb4
Clean up _rasterioParameters
banesullivan Nov 22, 2022
f75b4b3
Cherry pick @manthey's changes
banesullivan Nov 22, 2022
000ccf3
Bump rasterio to Python 3.8+
banesullivan Nov 22, 2022
e4b256a
Fix
banesullivan Nov 22, 2022
cfcb9f8
Merge branch 'master' into rasterio
banesullivan Jan 30, 2023
0405493
Merge branch 'master' into rasterio
banesullivan Feb 8, 2023
5703743
Merge branch 'master' into rasterio
banesullivan Feb 10, 2023
448b2f4
Fix projection issue
banesullivan Feb 10, 2023
873f76b
Fix odd issues
banesullivan Feb 10, 2023
718a573
Fix no projection tile issues
banesullivan Feb 10, 2023
d4b5f5e
Fix no projection getRegion issue
banesullivan Feb 10, 2023
535df6b
Setup.py metadata
banesullivan Feb 10, 2023
6ebaecf
getRegion working
banesullivan Feb 14, 2023
90c5a94
Remove _getRegionVrt
banesullivan Feb 14, 2023
aa32fcb
TODO
banesullivan Feb 15, 2023
8870e4f
Merge branch 'master' into rasterio
banesullivan Feb 15, 2023
634de24
Merge branch 'master' into rasterio
banesullivan Feb 15, 2023
e452ec3
Reproject with getRegion
banesullivan Feb 15, 2023
eca5321
Merge branch 'master' into rasterio
banesullivan Apr 11, 2023
5280a84
Merge branch 'master' into rasterio
banesullivan Apr 18, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
7 changes: 7 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,13 @@ coverage.xml
# pyenv
.python-version

# IPython
profile_default/
ipython_config.py

# Jupyter Notebook
.ipynb_checkpoints

# dotenv
.env

Expand Down
2 changes: 1 addition & 1 deletion large_image/tilesource/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
from .utilities import (_encodeImage, _encodeImageBinary, # noqa: F401
_gdalParameters, _imageToNumpy, _imageToPIL,
_letterboxImage, _makeSameChannelDepth, _vipsCast,
_vipsParameters, dictToEtree, etreeToDict,
_vipsParameters, dictToEtree, etreeToDict, _rasterioParameters,
getPaletteColors, histogramThreshold, nearPowerOfTwo)


Expand Down
83 changes: 83 additions & 0 deletions large_image/tilesource/utilities.py
Original file line number Diff line number Diff line change
Expand Up @@ -275,6 +275,43 @@ def _vipsCast(image, mustBe8Bit=False, originalScale=None):
image = ((image.cast(pyvips.BandFormat.DOUBLE) + offset) * multiplier).cast(target)
return image

def _rasterioParameters(defaultCompression=None, eightbit=None, **kwargs):
"""
return a dictionnary of creation option for the rasterio driver

:param defaultCompression: if not specified, use this value.
:param eightbit: True or False to indicate that the bit depth per sample is known. None for unknown.

Optional parameters that can be specified in kwargs:

:param tileSize: the horizontal and vertical tile size.
:param compression: one of 'jpeg', 'deflate' (zip), 'lzw', 'packbits', 'zstd', or 'none'.
:param quality: a jpeg quality passed to gdal. 0 is small, 100 is high quality. 90 or above is recommended.
:param level: compression level for zstd, 1-22 (default is 10).
:param predictor: one of 'none', 'horizontal', or 'float' used for lzw and deflate.

:returns: a dictionary of parameters.
"""

# some default option and parameters
options = {"blocksize":256, "compress": "lzw", "quality": 90}

# the name of the predictor need to be strings so we convert here from set values to actual
# required values (https://rasterio.readthedocs.io/en/latest/topics/image_options.html)
predictor = {"none": "NO", "horizontal": "STANDARD", "float": "FLOATING_POINT", "yes": "YES"}

if eightbit is not None:
options["predictor"] = "yes" if eightbit else "none"

# add the values from kwargs to the options. Remove anything that isnot set.
options.update({k: v for k, v in kwargs.items() if v not in (None, "")})

# add the remaining options
options.update(bigtiff="IF_SAFER")
"predictor" not in options or options.update(predictor=predictor[options["predictor"]])

return options


def _gdalParameters(defaultCompression=None, eightbit=None, **kwargs):
"""
Expand Down Expand Up @@ -319,7 +356,53 @@ def _gdalParameters(defaultCompression=None, eightbit=None, **kwargs):
if 'level' in options:
cmdopt += ['-co', 'LEVEL=%s' % options['level']]
return cmdopt

def _rasterioParameters(defaultCompression=None, eightbit=None, **kwargs):
"""
return a dictionnary of creation option for the rasterio driver

:param defaultCompression: if not specified, use this value.
:param eightbit: True or False to indicate that the bit depth per sample is
known. None for unknown.
Optional parameters that can be specified in kwargs:
:param tileSize: the horizontal and vertical tile size.
:param compression: one of 'jpeg', 'deflate' (zip), 'lzw', 'packbits',
'zstd', or 'none'.
:param quality: a jpeg quality passed to gdal. 0 is small, 100 is high
quality. 90 or above is recommended.
:param level: compression level for zstd, 1-22 (default is 10).
:param predictor: one of 'none', 'horizontal', or 'float' used for lzw and
deflate.
:returns: a dictionary of parameters.
"""

# some default option and parameters
options = {
"blocksize":256,
"compress": "lzw",
"quality": 90,
}

# the name of the predictor need to be strings so we convert here from set values to actual
# required values (https://rasterio.readthedocs.io/en/latest/topics/image_options.html)
predictor = {
'none': 'NO',
'horizontal': 'STANDARD',
'float': 'FLOATING_POINT',
'yes': 'YES',
}

if eightbit is not None:
options['predictor'] = 'yes' if eightbit else 'none'

# add the values from kwargs to the options. Remove anything that isnot set.
options.update({k: v for k, v in kwargs.items() if v not in (None, '')})

# add the remaining options
options.update(bigtiff="IF_SAFER")
"predictor" not in options or options.update(predictor=predictor[options["predictor"]])

return options

def _vipsParameters(forTiled=True, defaultCompression=None, **kwargs):
"""
Expand Down
1 change: 1 addition & 0 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ def prerelease_local_scheme(version):
'tiff': [f'large-image-source-tiff{limit_version}'],
'tifffile': [f'large-image-source-tifffile{limit_version}'],
'vips': [f'large-image-source-vips{limit_version}'],
'rasterio': [f'large-image-source-rasterio{limit_version}'],
}
if sys.version_info >= (3, 7):
sources.update({
Expand Down
2 changes: 1 addition & 1 deletion sources/gdal/large_image_source_gdal/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -483,7 +483,7 @@ def getBounds(self, srs=None):
"""
Returns bounds of the image.

:param srs: the projection for the bounds. None for the default.
:param srs: the projection for the bounds. None for the default 4326.
:returns: an object with the four corners and the projection that was
used. None if we don't know the original projection.
"""
Expand Down