Skip to content

Geoprocessing tools (Python) for efficient geometric mapping and set operations over geospatial grids

License

Notifications You must be signed in to change notification settings

eurostat/pygridmap

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

pygridmap

Geoprocessing tools for efficient geometric mapping and set operations over geospatial grids.

The pygridmap package enable you to perform some basic geometric/set operations over large/big vector datasets, including regular grid maker (i.e., rasterisation) and grid overlay (i.e. intersection and union). The code takes advantage of multiprocessor capabilities for efficient tile-based processing.

Description

The package pygridmap supports the following methods/algorithms:

Regular gridding with different reference locations
  • user-defined rasterisation of polygon regions (e.g., vector boundaries) into regular grids of various resolutions (i.e., dimension of one-size square grid cells) with the GridMaker class (see also Java implementation GridMaker),
Regular gridding striclty covering a spatial domain (left) or its interior only (right)
  • customised vector overlay operations (e.g., intersection, union, overlay/overlap and merging) between any polygon layer and a geospatial grid ("regular" or not) with the Gridoverlay class, including weighted areal interpolation.
Intersection of a regular grid with an arbitrary vector layer

Quick launch

  • Run (and update to your convenience) the test/example notebooks from the tests/ folder in binder: Binder. We provide the interactive environments with already installed packages to query and access Eurostat database for notebook resources below (current build with commit ba83e79).

  • Run the notebooks in Google colab separately, namely:

Quick install

The pygridmap package can be installed using pip:

>>> pip install pygridmap

If you wish to use the latest available version from github:

>>> pip install git+https://github.com/eurostat/pygridmap.git

Usage

In your Python script

The following classes are available:

>>> from pygridmap.base import FrameProcessor, GridProcessor
>>> from pygridmap.gridding import GridMaker
>>> from pygridmap.overlay import GridOverlay

Through the bash command

Notes

Processing

The implementation of the methods above adopts (customised, using the multiprocessing module) multitprocessor tiling processing together with (native) vector processing whenever possible in order to take advantage of multiprocessor compute capabilities. Note that the FrameProcessor class can be used to run embarassing parallel calculations on dataframe rows.

Example of block-processing tiles

Geometric operations are making an extensive use of the geometric/geospatial Python module geopandas since this module supports essential features (most of them derived from the shapely fiona libraries), like:

These implementations are probably not optimal since they are wrappers to the GEOS library, itself a port of the Java Topology Suite (JTS), similarly to what is used in QGis.

Algorithms

Given a source geospatial representation with various geometries, the simple way to create a regular grid with unit cells of dimension [cellx, celly] covering the spatial domain of the source vector layer is:

>>> from pygridmap import gridding
>>> grid = gridding.grid_maker(source, [cellx, celly]) 

See documentation of method gridding.grid_maker for additional parameters.

On areal interpolation, the package tobler that is distributed with the pysal library enables to perform more generic operations (i.e., not only considering grid as a target layer). Whenever one needs to project/interpolate an extensive feature attribute from a source vector layer onto a target vector layer (e.g., a regular grid), the following command:

>>> from pygridmap import overlay
>>> estimate = overlay.area_interpolate(source, target, attribute) 

is equivalent to running:

>>> from tobler import area_weighted
>>> estimate = area_weighted.area_interpolate(source, target, extensive_variables = attribute)

See documentation of method overlay.area_interpolate for additional parameters.

Areal interpolation of an extensive variable (left) from coarse to fine using tobler code (left) and pygridmap algorithm (right)

With respect to the latter implementation, the current algorithm supports a tile-based multicore approach for the interpolation (through the setting of the tile parameter). The tobler algorithm will however help you "project" intensive variables as well.

About

documentation in construction
status since 2020 – ongoing
contributors
license EUPL

Requirements

Other resources

  • Project python-geospatial, a collection of Python packages for geospatial analysis, and earthdatascience, a repository of Python tutorials and workshop.
  • Library pysal for geospatial data science with an emphasis on geospatial vector data.
  • Software GridMaker for alternative grid maker.

References

About

Geoprocessing tools (Python) for efficient geometric mapping and set operations over geospatial grids

Resources

License

Stars

Watchers

Forks

Packages

No packages published