This project does currently not contain meaningful functionality, and it is currently lacking a clear vision. After some consideration I decided that the original intended scope of the project was too far-reaching.
An updated idea could sound like this:
This project aims to provide examples and utilities for common tasks related to DSM. For some of these tasks, we also want to work on high-level user interfaces, accessible via python APIs or the command line. The actual legwork is done by established libraries. Custom solutions are implemented where needed or more convenient.
For more info, see Roadmap.
For now, you might find the examples in the docs or the list of Resources useful.
This is a list of some useful python packages and other resources in the context of DSM:
Data formats:
- geopandas: for vector data (shapefiles, .gpkg, etc.). I recommend saving data as .gpkg (not shapefiles) to get a single self-contained file.
- rasterio: for GeoTIFF files
- xarray for NetCDF (.nc) files. It can also read GeoTIFF if rasterio is installed.
- GDAL of course… (
import osgeo.gdal) - (exifread can read metadata from TIFFs)
- (pyshp: read/write raw shapefiles, without the overhead or convenience of geopandas)
- (xlrd to read excel files via
pandas.read_excel)
Covariates:
- elevation: downloads SRTM30/SRTM90 Digital Elevation Models (DEM) for arbitrary locations
- pysheds: watershed delineation
- richdem: set of DEM hydrologic analysis tools
- xarray-spatial: common raster analysis functions, DEM generation
- xclim: derived climate variables built with xarray
Google earth:
- earthengine-api: Google Earth Engine API
- geemap: interactive mapping (in jupyter) using Google Earth Engine
- geedim: search, compose, and download GEE imagery without size limits
Machine learning:
- scipy: General purpose scientific library, optimizers, image filters
- scikit-learn: Many easy-to-use general purpose machine learning algorithms
- imbalanced-learn: resampling for datasets with strong class imbalance
- PyKrige: Kriging toolkit
- GSTools: GeoStatTools provides geostatistical tools for various purposes
- GPflow: Gaussian processes in tensorflow
- gpytorch: Gaussian processes in pytorch
- (scikit-image: collection of algorithms for image processing)
Coordinates and shapes:
- shapely: Geometric objects, predicates, and operations
- pyproj: cartographic projections and coordinate transformations library
- (affine: affine transformation of the plane. Usually indirectly installed through shapely or others)
- (rtree: spatial indexing, i.e. fast lookup of polygons by coordinates)
- (descartes: plotting geometric objects with matplotlib)
Jupyter:
- jupytext: command line tool to convert .ipynb to and from different file formats (.html, .py), execute a notebook or clear all outputs
- (jupyterlab: more interactive interface for jupyter notebooks. Warning: interactive widgets often work only either in lab or notebook)
- (ipympl: interactive matplotlib figures for jupyterlab)
- (ipywidgets: interactive widgets for jupyterlab - progress bar, buttons, checkboxes etc)
Dataset catalogs:
- Earth Engine Data Catalog: Catalog of various environmental datasets
- Free GIS Data: Large collection of references to datasets
- Useful Geographic Datasets
- List of GIS data sources: wikipedia entry
- ArcGIS Hub
- EarthExplorer
Misc:
- PySAL: "Python Spatial Analysis Library". Looks generally useful, but I didn't check it out in detail yet.
In the future, we plan on working the following topics:
- common easy-to-use interface (like scikit-learn) for several machine learning models for linear regression, kriging, regression-kriging, random forests, ...
- examples for plotting raster and vector data
- examples for reprojection, resampling (using rasterio/gdal)
- sampling points from vector datasets
- custom (non-stationary) GP kernels for DSM
- algorithms for experimental design (DOE)
- 3D terrain survey
- ...?
Once the first release is available, install via:
pip install pysoilmapTBD