GSoC 2016 Ideas

Jachym Cepicky edited this page Mar 23, 2016 · 3 revisions

General info

PyWPS is one of the first implementations of OGC WPS standard on the server side, written in Python programming language.

During time, (geo-)python environment has changed a lot and we took this a opportunity to rewrite PyWPS from scratch, with experiences from our previous work.

We would like to offer several ideas, which could be added to new created PyWPS core, we are focusing on and we know, our users would like to see.

For more details, please join our mailing list

GSOC 2016 ideas

Database output storage - PostGIS

PyWPS has to store resulting data files at some place. Currently, simple file storage is implemented - all resulting files are stored to defined output directory on the server. In the future, we shall support more sophisticated methods of data storage, like database.

This includes raster, vector and "other" output data. They need to be effectively stored, together with their metadata, for later usage (e.g. deleting old files, usage statistics, etc.)

Mentor name: Jorge de Jesus (jorge.mendesdejesus wur nl)

Required skills: Python, PostgreSQL and PostGIS intermediate user, awaraness of OGC OWS services

Database output storage - SQLite

PyWPS has to store resulting data files at some place. Currently, simple file storage is implemented - all resulting files are stored to defined output directory on the server. In the future, we shall support more sophisticated methods of data storage, like database.

This includes raster, vector and "other" output data. They need to be effectively stored, together with their metadata, for later usage (e.g. deleting old files, usage statistics, etc.)

Mentor name: Jorge de Jesus (jorge.mendesdejesus wur nl)

Required skills: Python, SQLite intermediate user, awareness of OGC OWS services

Remote output storage

PyWPS has to store resulting data files at some place. Currently, simple file storage is implemented - all resulting files are stored to defined output directory on the server. In the future, we shall support more sophisticated methods of data storage, like external cloud service.

Several services are in place - proprietary(-cloud) (Dropbox, Google Drive, Box, etc.) or standards(-protocols)-oriented (FTP, SFTP, WEBDAV, ...).

Mentor name: Jachym Cepicky, jachym.cepicky gmail com

Required skills: Python intermediate coder, awareness of OGC OWS services

Publishing data to MapServer and pycsw

OGC WPS 1.0.0 Standard and its best-practice document enable to publish OGC OWS services as result. Naturally that would be OGC WCS and WFS (and WMS). PyWPS will never provide such functionality by itself, but it will use external services. In current stable version of PyWPS (3.x), basic steps for MapServer support are done.

We would like to implement support for usage of MapServer, to be used for output data serving, via OGC services. This should be done using standard MapServer Mapfile configuration.

As part of MapServer OWS publishing, if a metadata document is available, PyWPS can publish directly to CSW given CSW's Transaction operation. Deeper integration with pycsw is also possible given pycsw is also a Python implementation.

Having a rich publishing workflow to data and metadata services from PyWPS makes this workflow really attractive to so many downstream possibilities (GeoNode, QGIS) either directly in Python or via WPS proper.

Mentor name: Luís de Sousa, luis.a.de.sousa gmail com

Required skills: Python intermediate coder with user experience in MapServer, awareness of pycsw and OGC Web services,

Publishing data with help of GeoServer

OGC WPS 1.0.0 Standard and it's best-practice document do enable to publish OGC OWS services as result. Naturally that would be OGC WCS and WFS (and WMS). PyWPS will never provide such functionality by itself, but it will use external services. In current stable version of PyWPS (3.x), basic steps for UMN MapServer support are done.

We would like to implement support for usage of GeoServer, to be used for output data serving, via OGC services. This should be done using REST interface of GeoServer

Mentor name: Jachym Cepicky, jachym.cepicky gmail com

Required skills: Python intermediate coder with experience in REST APIs, GeoServer user is advantage, awareness of OGC OWS services

Publishing data with help of QGIS MapServer

OGC WPS 1.0.0 Standard and it's best-practice document do enable to publish OGC OWS services as result. Naturally that would be OGC WCS and WFS (and WMS). PyWPS will never provide such functionality by itself, but it will use external services. In current stable version of PyWPS (3.x), basic steps for UMN MapServer support are done.

We would like to implement support for usage of QGIS MapServer, to be used for output data serving, via OGC services.

Mentor name: Jachym Cepicky, jachym.cepicky gmail com

Required skills: Python intermediate coder with experience with QGIS, awareness of OGC OWS services

Process chaining capabilities

Chaining geo processing services based on OGC WPS Standard can be achieved in different ways based on the complexity of the application. Indeed there are existing possibilities mainly for SOAP-based web services in general (Taverna, BPEL), but in most cases a lot of effort needs to be invested to implement such services and process chains. 

A good enhancement of PyWPS would be the integration of chaining geo processing services either published as a further process or defined by an input parameter (e.g., XML-based definition for process chaining). Following features could / should be included (further ideas are welcome): 

  • Status updating of parent process (=process of process chain), 
  • Provide access to results of subprocess, even if chain is still running, 
  • If/else conditions and for loops based on output values of subprocess, 
  • Logical data transfer if subprocesses are on the same machine (no unnecessary copy of data) 

Mentor name: Jonas Eberle, jonas.eberle gmx de

Required skills: Python intermediate coder with experience XML and lxml, awareness of OGC OWS services

Web-based administration & process management

Providing operational running processing services requires often a tool to list executed processes with their current status to have an overview about usage and possible errors. Users and administrators could easily view a process log file, if generated from the process, and further information about the executed processes. Furthermore process actions - like an easy way to restart or delete an executed process - could be very helpful for the process management. An administration part could extend PyWPS to make it more user-friendly in creating services and changing settings. 

The administration part could include: 

  • Create services, change settings of services
  • Add new processes (e.g., provide a template process for further manual adjustments)
  • Change process settings
  • User management (?)

The process management part could include:

  • List executed processes with their status (e.g., running, failed, succeeded)
  • View process details (e.g., inputs, outputs, log file)
  • Provide process actions (e.g., restart, delete)
  • further ideas are welcome

WPS 2.0.0 implementation

PyWPS supports OGC Web Processing Service 1.0.0. We would like to add support for newer version too

http://opengeospatial.org/standards/wps

Mentor name: Jachym Cepicky, jachym.cepicky gmail com