Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add function to load raster tile maps using contextily #2125

Merged
merged 35 commits into from
Mar 2, 2023
Merged
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
6f7a4f0
Add function to load raster basemap tiles using contextily
weiji14 Sep 19, 2022
917604d
Format to remove triple dots
weiji14 Sep 19, 2022
66f66cc
Use correct Spherical Mercator coordinates
weiji14 Sep 19, 2022
e99ed4d
Fix coordinates for doctest
weiji14 Sep 19, 2022
3c437c0
Change ll parameter to lonlat
weiji14 Sep 19, 2022
c738375
Reduce number of local variables to fix pylint R0914
weiji14 Sep 19, 2022
30fd9a2
Add contextily to CI build matrix and include it as optional dependency
weiji14 Sep 19, 2022
f913222
Set default bounding box coordinates to be lonlat
weiji14 Sep 20, 2022
1270c3f
Merge branch 'main' into contextily/load_map_tiles
weiji14 Oct 12, 2022
0455bb4
Skip doctest when contextily is not installed
weiji14 Oct 12, 2022
8b5317f
Merge branch 'main' into contextily/load_map_tiles
weiji14 Nov 4, 2022
6142b40
Fix typo
weiji14 Nov 4, 2022
5400710
Add intersphinx link for rasterio
weiji14 Nov 4, 2022
8aa1c7a
Merge branch 'main' into contextily/load_map_tiles
weiji14 Dec 6, 2022
bf1c007
Merge branch 'main' into contextily/load_map_tiles
weiji14 Dec 28, 2022
26dd404
Merge branch 'main' into contextily/load_map_tiles
weiji14 Dec 28, 2022
2959e58
Document wait and max_retries parameters used in contextily.bounds2img
weiji14 Dec 28, 2022
241aec1
Merge branch 'main' into contextily/load_map_tiles
weiji14 Jan 7, 2023
eeda973
Reorder deps and add a fullstop
weiji14 Jan 7, 2023
cc86e7d
Merge branch 'main' into contextily/load_map_tiles
weiji14 Jan 24, 2023
67b17fc
Use PyGMT's convention for default values in docstrings
weiji14 Jan 24, 2023
5a446fb
Merge branch 'main' into contextily/load_map_tiles
weiji14 Feb 26, 2023
3dc8846
Rename load_map_tiles to load_tile_map
weiji14 Feb 26, 2023
5c61779
Lint to change coords dict() to {}
weiji14 Feb 26, 2023
d06e2c0
Add zoom parameter and remove kwargs
weiji14 Feb 26, 2023
6b2d5cd
Revert "Reduce number of local variables to fix pylint R0914"
weiji14 Feb 26, 2023
b1ad795
Add contextily to docs build CI requirements
weiji14 Feb 27, 2023
484b6c8
Add contextily to pygmt.show_versions() dependency list
weiji14 Feb 27, 2023
cf7ab34
Apply suggestions from code review
weiji14 Feb 27, 2023
8563ecf
Lint to reduce length of comment line
weiji14 Feb 27, 2023
aa2d5fb
Document the three possible source options thoroughly
weiji14 Mar 2, 2023
d74f878
Remove extra fullstops and fix typos
weiji14 Mar 2, 2023
9f14d31
Add more detail about the zoom level of detail
weiji14 Mar 2, 2023
b02b50c
Remove another fullstop
weiji14 Mar 2, 2023
10af781
Wrap integers in backticks and fix some typos
weiji14 Mar 2, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci_docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ jobs:
- name: Install dependencies
run: |
mamba install gmt=6.4.0 numpy pandas xarray netCDF4 packaging \
build ipython make myst-parser geopandas \
build ipython make myst-parser contextily geopandas \
sphinx sphinx-copybutton sphinx-design sphinx-gallery sphinx_rtd_theme

# Show installed pkg information for postmortem diagnostic
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci_tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ jobs:
optional-packages: ''
- python-version: '3.11'
numpy-version: '1.24'
optional-packages: 'geopandas ipython'
optional-packages: 'contextily geopandas ipython'
timeout-minutes: 30
defaults:
run:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci_tests_dev.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ jobs:
geopandas ghostscript libnetcdf hdf5 zlib curl pcre make
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Off-topic: All Python packages are installed using pip, except that geopandas is installed using mamba. What's the reason?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at #1290 (comment), I think it was to get compatible GDAL versions between gmt, geopandas and fiona at the time. We could try doing pip install --pre geopandas, but best do it in a separate PR.

pip install --pre --prefer-binary \
numpy pandas xarray netCDF4 packaging \
build dvc ipython 'pytest>=6.0' pytest-cov \
build contextily dvc ipython 'pytest>=6.0' pytest-cov \
pytest-doctestplus pytest-mpl sphinx-gallery

# Pull baseline image data from dvc remote (DAGsHub)
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci_tests_legacy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ jobs:
run: |
mamba install gmt=${{ matrix.gmt_version }} numpy \
pandas xarray netCDF4 packaging \
geopandas ipython \
contextily geopandas ipython \
build dvc make 'pytest>=6.0' \
pytest-cov pytest-doctestplus pytest-mpl sphinx-gallery

Expand Down
1 change: 1 addition & 0 deletions ci/requirements/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ dependencies:
- netCDF4
- packaging
# Optional dependencies
- contextily
- geopandas
# Development dependencies (general)
- build
Expand Down
7 changes: 7 additions & 0 deletions doc/api/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,13 @@ and store them in GMT's user data directory.
datasets.load_earth_vertical_gravity_gradient
datasets.load_sample_data

In addition, there is also a special function to load XYZ tile maps via
:doc:`contextily <contextily:index>` to be used as base maps.

.. autosummary::
:toctree: generated

datasets.load_tile_map

.. currentmodule:: pygmt

Expand Down
5 changes: 4 additions & 1 deletion doc/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,11 +54,14 @@

# intersphinx configuration
intersphinx_mapping = {
"python": ("https://docs.python.org/3/", None),
"contextily": ("https://contextily.readthedocs.io/en/stable/", None),
"geopandas": ("https://geopandas.org/en/stable/", None),
"numpy": ("https://numpy.org/doc/stable/", None),
"python": ("https://docs.python.org/3/", None),
"pandas": ("https://pandas.pydata.org/pandas-docs/stable/", None),
"rasterio": ("https://rasterio.readthedocs.io/en/stable/", None),
"xarray": ("https://docs.xarray.dev/en/stable/", None),
"xyzservices": ("https://xyzservices.readthedocs.io/en/stable", None),
Comment on lines +62 to +64
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't worked with intersphinx_mapping before, but why are rasterio and xyzservices being added on this in addition to contextily? I'm assuming they're required dependencies for contextily?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These intersphinx links are added so that :doc:`rasterio <rasterio:index>` and :class:`xyzservices.TileProvider` can be used in the docstring.

}

# options for sphinx-copybutton
Expand Down
1 change: 1 addition & 0 deletions doc/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,7 @@ PyGMT requires the following libraries to be installed:
The following are optional dependencies:

* `IPython <https://ipython.org>`__: For embedding the figures in Jupyter notebooks (recommended).
* `Contextily <https://contextily.readthedocs.io>`__: For retrieving tile maps from the internet.
* `GeoPandas <https://geopandas.org>`__: For using and plotting GeoDataFrame objects.

Installing GMT and other dependencies
Expand Down
1 change: 1 addition & 0 deletions environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ dependencies:
- netCDF4
- packaging
# Optional dependencies
- contextily
weiji14 marked this conversation as resolved.
Show resolved Hide resolved
- geopandas
- ipython
# Development dependencies (general)
Expand Down
10 changes: 9 additions & 1 deletion pygmt/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,15 @@ def _get_ghostscript_version():
"machine": platform.platform(),
}

deps = ["numpy", "pandas", "xarray", "netCDF4", "packaging", "geopandas"]
deps = [
"numpy",
"pandas",
"xarray",
"netCDF4",
"packaging",
"contextily",
"geopandas",
]

print("PyGMT information:")
print(f" version: {__version__}")
Expand Down
1 change: 1 addition & 0 deletions pygmt/datasets/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,4 @@
load_earth_vertical_gravity_gradient,
)
from pygmt.datasets.samples import list_sample_data, load_sample_data
from pygmt.datasets.tile_map import load_tile_map
149 changes: 149 additions & 0 deletions pygmt/datasets/tile_map.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
"""
Function to load raster tile maps from XYZ tile providers, and load as
:class:`xarray.DataArray`.
"""

try:
import contextily
except ImportError:
contextily = None

import numpy as np
import xarray as xr

__doctest_requires__ = {("load_tile_map"): ["contextily"]}


def load_tile_map(region, zoom="auto", source=None, lonlat=True, wait=0, max_retries=2):
"""
Load a georeferenced raster tile map from XYZ tile providers.

The tiles that compose the map are merged and georeferenced into an
:class:`xarray.DataArray` image with 3 bands (RGB). Note that the returned
image is in a Spherical Mercator (EPSG:3857) coordinate reference system.

Parameters
----------
region : list
The bounding box of the map in the form of a list [*xmin*, *xmax*,
*ymin*, *ymax*]. These coordinates should be in longitude/latitude if
``lonlat=True`` or Spherical Mercator (EPSG:3857) if ``lonlat=False``.

zoom : int or str
Optional. Level of detail. Higher levels (e.g. ``22``) means a zoom
level closer to the Earth's surface, with more tiles covering a smaller
geographical area and thus more detail. Lower levels (e.g. ``0``) means
a zoom level further from the Earth's surface, with less tiles covering
a larger geographical area and thus less detail [Default is
``"auto"`` to automatically determine the zoom level based on the
bounding box region extent].
weiji14 marked this conversation as resolved.
Show resolved Hide resolved

Note: the maximum possible zoom level may be smaller than 22, and
depends on what is supported by the chosen web tile provider source.
weiji14 marked this conversation as resolved.
Show resolved Hide resolved

source : xyzservices.TileProvider or str
Optional. The tile source: web tile provider or path to a local file.
Provide either:

- A web tile provider in the form of a
:class:`xyzservices.TileProvider` object. See
:doc:`Contextily providers <contextily:providers_deepdive>` for a
list of tile providers [Default is
``xyzservices.providers.Stamen.Terrain``, i.e. Stamen Terrain web
tiles].
- A web tile provider in the form of a URL. The placeholders for the
XYZ in the URL need to be {x}, {y}, {z}, respectively. E.g.
``https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png``.
- A local file path. The file is read with
:doc:`rasterio <rasterio:index>` and all bands are loaded into the
basemap. See
:doc:`contextily:working_with_local_files`.

IMPORTANT: tiles are assumed to be in the Spherical Mercator projection
(EPSG:3857).
weiji14 marked this conversation as resolved.
Show resolved Hide resolved

lonlat : bool
Optional. If False, coordinates in ``region`` are assumed to be
Spherical Mercator as opposed to longitude/latitude [Default is True].
weiji14 marked this conversation as resolved.
Show resolved Hide resolved

wait : int
Optional. If the tile API is rate-limited, the number of seconds to
wait between a failed request and the next try [Default is 0].
weiji14 marked this conversation as resolved.
Show resolved Hide resolved

max_retries : int
Optional. Total number of rejected requests allowed before contextily
will stop trying to fetch more tiles from a rate-limited API [Default
is 2].
weiji14 marked this conversation as resolved.
Show resolved Hide resolved

Returns
-------
raster : xarray.DataArray
Georeferenced 3-D data array of RGB values.

Raises
------
ModuleNotFoundError
If ``contextily`` is not installed. Follow
:doc:`install instructions for contextily <contextily:index>`, (e.g.
via ``pip install contextily``) before using this function.

Examples
--------
>>> import contextily
>>> from pygmt.datasets import load_tile_map
>>> raster = load_tile_map(
... region=[103.60, 104.06, 1.22, 1.49], # West, East, South, North
... source=contextily.providers.Stamen.TerrainBackground,
... lonlat=True, # bounding box coordinates are longitude/latitude
seisman marked this conversation as resolved.
Show resolved Hide resolved
... )
>>> raster.sizes
Frozen({'band': 3, 'y': 1024, 'x': 1536})
>>> raster.coords
Coordinates:
* band (band) int64 0 1 2
* y (y) float64 1.663e+05 1.663e+05 1.663e+05 ... 1.272e+05 ...
* x (x) float64 1.153e+07 1.153e+07 1.153e+07 ... 1.158e+07 ...
"""
# pylint: disable=too-many-locals
if contextily is None:
raise ModuleNotFoundError(
"Package `contextily` is required to be installed to use this function. "
"Please use `pip install contextily` or "
"`conda install -c conda-forge contextily` "
"to install the package."
)

west, east, south, north = region
image, extent = contextily.bounds2img(
w=west,
s=south,
e=east,
n=north,
zoom=zoom,
source=source,
ll=lonlat,
wait=wait,
max_retries=max_retries,
)

# Turn RGBA img from channel-last to channel-first and get 3-band RGB only
_image = image.transpose(2, 0, 1) # Change image from (H, W, C) to (C, H, W)
rgb_image = _image[0:3, :, :] # Get just RGB by dropping RGBA's alpha channel

# Georeference RGB image into an xarray.DataArray
left, right, bottom, top = extent
dataarray = xr.DataArray(
data=rgb_image,
coords={
"band": [0, 1, 2], # Red, Green, Blue
"y": np.linspace(start=top, stop=bottom, num=rgb_image.shape[1]),
"x": np.linspace(start=left, stop=right, num=rgb_image.shape[2]),
},
dims=("band", "y", "x"),
)

# If rioxarray is installed, set the coordinate reference system
if hasattr(dataarray, "rio"):
dataarray = dataarray.rio.write_crs(input_crs="EPSG:3857")

return dataarray
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ dynamic = ["version"]

[project.optional-dependencies]
all = [
"contextily",
"geopandas",
"ipython"
]
Expand Down