WebServicesHowto

Working with WOUDC Web Services

Overview

WOUDC Web Services provide machine access to observations via WFS, WMS, CSW and WPS. These services can be accessed via HTTP. All WOUDC Web Services are CORS enabled.

Data Download

TODO

Processing Services

WOUDC Processing Services are realized via the WOUDC Web Processing Service, which provides support for OGC:WPS 1.0.0. To interact with a WPS, you must submit an XML file as the HTTP request as part of the request/response workflow. The following validation services exist:

Process Discovery

To discover WOUDC's process offerings, invoke https://geo.woudc.org/wps?service=WPS&version=1.0.0&request=GetCapabilities. Each //wps:Process element represents an advertised process available for use.

Process Execution

To execute a process, the client must invoke an HTTP POST/XML request. HTTP POST requests with XML are a bit different then the traditional HTTP POST approach (key=value). An HTTP client opens a connection to the server and sends XML directly. WPS implements HTTP POST in this manner with XML requests.

There are numerous ways to make this type of request, but here are a few:

Python

import requests
requests.post('https://geo.woudc.org/wps', data=open('/path/to/request.xml').read()).text

Command line tools:

# curl
curl -X POST -d @/path/to/request.xml https://geo.woudc.org/wps

# lwp-request
cat /path/to/request.xml | POST https://geo.woudc.org/wps

# wget
wget https://geo.woudc.org/wps --post-file=/path/to/request.xml

Transformation Services

WOUDC Transformation Services provide real-time format conversion for non-standard Extended CSV formats submitted to the data centre. Transformation services allow the contributor to convert their data on demand to Extended CSV for submission to WOUDC in an agile and lightweight fashion. The following Transformation services exist:

• nsd2extcsv: convert a non-standard submission (SHADOZ, NASA-AMES, NASA-AMES-2160, BAS) to ExtendedCSV format

Let's try running the nsd2extcsv Process against some SHADOZ data:

Sample XML request:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<wps:Execute service="WPS" version="1.0.0" xmlns:wps="http://www.opengis.net/wps/1.0.0" xmlns:ows="http://www.opengis.net/ows/1.1" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/wps/1.0.0
http://schemas.opengis.net/wps/1.0.0/wpsExecute_request.xsd">
	<ows:Identifier>nsd2extcsv</ows:Identifier>
	<wps:DataInputs>
		<wps:Input>
			<ows:Identifier>format</ows:Identifier>
			<wps:Data>
				<wps:LiteralData>SHADOZ</wps:LiteralData>
			</wps:Data>
		</wps:Input>
		<wps:Input>
			<ows:Identifier>data</ows:Identifier>
			<wps:Data>
				<wps:ComplexData>$RAW_SHADOZ</wps:ComplexData>
			</wps:Data>
		</wps:Input>
	</wps:DataInputs>
</wps:Execute>

Notes:

• Save the following file to disk as foo.xml
• replace $RAW_SHADOZ with an actual SHADOZ file (open Extended CSV file in a text editor, select all, copy/paste)
• run the command as per one of the HTTP POST/XML examples, e.g.
• wget https://geo.woudc.org/wps --post-file=/path/to/request.xml
• the response is an XML document with the results of the Process, in this case the SHADOZ data converted to Extended CSV
• TODO: describe outputs on WOUDC website

Validation Services

WOUDC Validation Services provide real-time format checking as well as quality assessment for potential data submissions. Validation services do not ingest data into WOUDC; rather, they are agile and lightweight tools which expose the functionality of WOUDC data processing to support contributors running real-time checks on their data before submitting to WOUDC as a means to improve data quality over time.

Validation Services are realized via the WOUDC Web Processing Service, which provides support for OGC:WPS 1.0.0. To interact with a WPS, you must submit an XML file as the HTTP request as part of the request/response workflow. The following validation services exist:

• data-extcsv: format checker for Extended CSV format
• data-qa: executes Quality assessment against an Extended CSV file

Let's try running the ExtendedCSV format checkout Process.

Sample XML request:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<wps:Execute service="WPS" version="1.0.0" xmlns:wps="http://www.opengis.net/wps/1.0.0" xmlns:ows="http://www.opengis.net/ows/1.1" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/wps/1.0.0
http://schemas.opengis.net/wps/1.0.0/wpsExecute_request.xsd">
	<ows:Identifier>data-extcsv</ows:Identifier>
	<wps:DataInputs>
		<wps:Input>
			<ows:Identifier>data</ows:Identifier>
			<wps:Data>
				<wps:ComplexData>$RAW_EXTCSV</wps:ComplexData>
			</wps:Data>
		</wps:Input>
	</wps:DataInputs>
</wps:Execute>

Notes:

• Save the following file to disk as foo.xml
• replace $RAW_EXTCSV with an actual ExtendedCSV file (open Extended CSV file in a text editor, select all, copy/paste)
• run the command as per one of the HTTP POST/XML examples, e.g.
• wget https://geo.woudc.org/wps --post-file=/path/to/request.xml
• the response is an XML document with the results of the Process
• TODO: describe outputs on WOUDC website

Web Services Use Cases

Checking for new files processed into WOUDC

Overview

This use case covers those who poll WOUDC at given intervals for data updates as part of their downstream processing workflows. The idea here is process new files periodically (i.e. weekly).

Example URLs

See all new files in the index since 2018-01-29

https://geo.woudc.org/ows?service=WFS&version=1.1.0&request=GetFeature&typename=filelist&filter=<Filter><PropertyIsGreaterThanOrEqualTo><PropertyName>processed_datetime</PropertyName><Literal>2018-01-29</Literal></PropertyIsGreaterThanOrEqualTo></Filter>

This provides GML (XML) output. If you wish, you can add 'outputformat=CSV', or 'outputformat=GeoJSON' to get CSV/GeoJSON formats instead.

Filter Stations and Visualize on http://geojson.io

Overview

Use WOUDC Web Services to download a station list in GeoJSON format using temporal filters and load the result in http://geojson.io. This will allow a user to visualize stations reporting within a specified timeframe on a map.

Example URLs

https://geo.woudc.org/ows?service=WFS&version=1.1.0&request=GetFeature&typename=stations&filter=<Filter><And><PropertyIsGreaterThanOrEqualTo><PropertyName>date_from</PropertyName><Literal>YYYY-MM-DD HH:MM:SS</Literal></PropertyIsGreaterThanOrEqualTo><PropertyIsLessThanOrEqualTo><PropertyName>date_to</PropertyName><Literal>YYYY-MM-DD HH:MM:SS</Literal></PropertyIsLessThanOrEqualTo></And></Filter>

..where 'YYYY-MM-DD HH:MM:SS' are the "date from" and "date to" respectively. You can also download CSV, KML, or GIS formats using WOUDC Web Services.

Example querying for stations reporting between 2000 and 2011 in various formats:

GML (XML) https://geo.woudc.org/ows?service=WFS&version=1.1.0&request=GetFeature&typename=stations&filter=<Filter><And><PropertyIsGreaterThanOrEqualTo><PropertyName>date_from</PropertyName><Literal>2000-01-01 00:00:00</Literal></PropertyIsGreaterThanOrEqualTo><PropertyIsLessThanOrEqualTo><PropertyName>date_to</PropertyName><Literal>2011-01-01 00:00:00</Literal></PropertyIsLessThanOrEqualTo></And></Filter>&

CSV: https://geo.woudc.org/ows?service=WFS&version=1.1.0&request=GetFeature&typename=stations&filter=<Filter><And><PropertyIsGreaterThanOrEqualTo><PropertyName>date_from</PropertyName><Literal>2000-01-01 00:00:00</Literal></PropertyIsGreaterThanOrEqualTo><PropertyIsLessThanOrEqualTo><PropertyName>date_to</PropertyName><Literal>2011-01-01 00:00:00</Literal></PropertyIsLessThanOrEqualTo></And></Filter>&outputformat=CSV

NetCDF: https://geo.woudc.org/ows?service=WFS&version=1.1.0&request=GetFeature&typename=stations&filter=<Filter><And><PropertyIsGreaterThanOrEqualTo><PropertyName>date_from</PropertyName><Literal>2000-01-01 00:00:00</Literal></PropertyIsGreaterThanOrEqualTo><PropertyIsLessThanOrEqualTo><PropertyName>date_to</PropertyName><Literal>2011-01-01 00:00:00</Literal></PropertyIsLessThanOrEqualTo></And></Filter>&outputformat=NetCDF

GeoJSON: https://geo.woudc.org/ows?service=WFS&version=1.1.0&request=GetFeature&typename=stations&filter=<Filter><And><PropertyIsGreaterThanOrEqualTo><PropertyName>date_from</PropertyName><Literal>2000-01-01 00:00:00</Literal></PropertyIsGreaterThanOrEqualTo><PropertyIsLessThanOrEqualTo><PropertyName>date_to</PropertyName><Literal>2011-01-01 00:00:00</Literal></PropertyIsLessThanOrEqualTo></And></Filter>&outputformat=GeoJSON

KML: https://geo.woudc.org/ows?service=WFS&version=1.1.0&request=GetFeature&typename=stations&filter=<Filter><And><PropertyIsGreaterThanOrEqualTo><PropertyName>date_from</PropertyName><Literal>2000-01-01 00:00:00</Literal></PropertyIsGreaterThanOrEqualTo><PropertyIsLessThanOrEqualTo><PropertyName>date_to</PropertyName><Literal>2011-01-01 00:00:00</Literal></PropertyIsLessThanOrEqualTo></And></Filter>&outputformat=OGRKML

Steps:

1. determine a start date and end date to support a temporal filter (format YYYY-MM-DD HH:MM:SS)
2. run a WFS query with a temporal filter: https://geo.woudc.org/ows?service=WFS&version=1.1.0&request=GetFeature&typename=stations&filter=<Filter><And><PropertyIsGreaterThanOrEqualTo><PropertyName>date_from</PropertyName><Literal>$DATE_FROM</Literal></PropertyIsGreaterThanOrEqualTo><PropertyIsLessThanOrEqualTo><PropertyName>date_to</PropertyName><Literal>$DATE_TO</Literal></PropertyIsLessThanOrEqualTo></And></Filter>&outputformat=GeoJSON
3. download the result (result.json) as a file on your desktop
4. open a web browser and visit http://geojson.io
5. using the File/Open menu, navigate to the downloaded file
6. click on a given point to see the associated properties
7. click on the Table link (top right) to visualize the station data in a tabular display

Expected Result:

WFS based station query can be visualized using online web applications.