2021.11.1

• core

  • use generators to start processing as soon as possible and to reduce memory consumption when processing large areas
  • add preprocessing_tasks tiles_tasks attributes to Job
  • reduce code smells

• package

  • fix http extra in setup.py

2021.11.0

• CLI

  • convert: enable --output-pyramid to read custom grid from JSON file
  • stac create-item: add --relative-paths flag for asset URL

• package

  • add mapchete.testing including convenience tools
  • use temporary directories for outputs where possible when testing processes

2021.10.3

• core
  • mapchete.Executor: fix call in DaskExecutor.as_completed() by not providing current client as loop

2021.10.2

• core
  • mapchete.Executor: add max_submitted_futures option to DaskExecutor.as_completed() to cap the number of tasks sent to the scheduler; also immediately yield finished tasks once they are available

2021.10.1

• core

  • mapchete.Executor: make sure futures are removed after yield; track time spent to submit tasks
  • mapchete.Executor: remove task submission delay introduced for debugging
  • mapchete.commands.convert: set process metatiling so output tiles cannot be larger than process metatiles
  • mapchete.commmands.convert: fix overwrite flag on remote single GeoTIFFs
  • mapchete.commands.cp: add point and point-crs options
  • don't write all STAC information into metadata.json
  • don't initialize InputTile objects when processing baselevel tile

2021.10.0

• packaging

  • change version numbering scheme to YYYY.MM.x

• core

  • make sure Executor.running_futures is cleared after each as_completed() and cancel() call
  • fix fargs and fkwargs ommission in Executor.map()

• testing

  • skip COG tests if driver is not available in GDAL

0.44

• core

  • add warnings when using Python<3.7 with usage of start methods other than spawn
  • mapchete.Executor keep track of running and finished futures and remove finished futures to release memory (on local machine as well as on dask cluster)

• CLI

  • add mapchete stac create-item command to create a STAC tiled-assets file

0.43

• core

  • mapchete.io.vector.reproject_geometry():

    • use pyproj to determine CRS bounds to clip geometries when reprojecting
    • enable geometry segmentation before geometry is clipped (segmentize_on_clip=False and segmentize_fraction=100 args)

  • suppress rasterio warnings when reading rasters (too many rasterio.errors.NodataShadowWarnings)

• packaging

  • add pyproj to dependencies

0.42

• core

  • add option for input drivers to let processing-heavy tasks be handled by mapchete.Executor by implementing InputData.add_preprocessing_task() and InputData.get_preprocessing_task_result() methods
  • check futures submitted to mapchete.Executor for exceptions before yielding
  • remove deprecated multi (now called workers), distributed and max_chunksize arguments

• CLI

  • remove deprecated --max-chunksize option
  • replace "tiles" with "tasks" in progress

0.41

• core

  • add mapchete.commands package
  • add dask as optional Executor
  • expose futures in Executor class to facilitate job cancellation
  • use spawn as default multiprocessing start method (#351)
  • fix missing worker logs (#185)
  • rename mapchete.cli.utils to mapchete.cli.options
  • enable providing process code from within process config

• packaging

  • updated API doc

• CLI

  • CLI: change --multi option to --worker
  • enable optional concurrency for mapchete cp