-
Notifications
You must be signed in to change notification settings - Fork 5
WebServicesHowto
WOUDC Web Services provide machine access to observations via WFS, WMS, CSW and WPS. These services can be accessed via HTTP. All WOUDC Web Services are CORS enabled.
TODO
WOUDC Processing Services are realized via the WOUDC Web Processing Service, which provides support for OGC:WPS 1.0.0. To interact with a WPS, you must submit an XML file as the HTTP request as part of the request/response workflow. The following validation services exist:
To discover WOUDC's process offerings, invoke https://geo.woudc.org/wps?service=WPS&version=1.0.0&request=GetCapabilities. Each //wps:Process
element represents an advertised process available for use.
To execute a process, the client must invoke an HTTP POST/XML request. HTTP POST requests with XML are a bit different then the traditional HTTP POST approach (key=value). An HTTP client opens a connection to the server and sends XML directly. WPS implements HTTP POST in this manner with XML requests.
There are numerous ways to make this type of request, but here are a few:
Python
import requests
requests.post('https://geo.woudc.org/wps', data=open('/path/to/request.xml').read()).text
Command line tools:
# curl
curl -X POST -d @/path/to/request.xml https://geo.woudc.org/wps
# lwp-request
cat /path/to/request.xml | POST https://geo.woudc.org/wps
# wget
wget https://geo.woudc.org/wps --post-file=/path/to/request.xml
WOUDC Transformation Services provide real-time format conversion for non-standard Extended CSV formats submitted to the data centre. Transformation services allow the contributor to convert their data on demand to Extended CSV for submission to WOUDC in an agile and lightweight fashion. The following Transformation services exist:
-
nsd2extcsv
: convert a non-standard submission (SHADOZ, NASA-AMES, NASA-AMES-2160, BAS) to ExtendedCSV format
Let's try running the nsd2extcsv
Process against some SHADOZ data:
Sample XML request:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<wps:Execute service="WPS" version="1.0.0" xmlns:wps="http://www.opengis.net/wps/1.0.0" xmlns:ows="http://www.opengis.net/ows/1.1" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/wps/1.0.0
http://schemas.opengis.net/wps/1.0.0/wpsExecute_request.xsd">
<ows:Identifier>nsd2extcsv</ows:Identifier>
<wps:DataInputs>
<wps:Input>
<ows:Identifier>format</ows:Identifier>
<wps:Data>
<wps:LiteralData>SHADOZ</wps:LiteralData>
</wps:Data>
</wps:Input>
<wps:Input>
<ows:Identifier>data</ows:Identifier>
<wps:Data>
<wps:ComplexData>$RAW_SHADOZ</wps:ComplexData>
</wps:Data>
</wps:Input>
</wps:DataInputs>
</wps:Execute>
Notes:
- Save the following file to disk as
foo.xml
- replace
$RAW_SHADOZ
with an actual SHADOZ file (open Extended CSV file in a text editor, select all, copy/paste) - run the command as per one of the HTTP POST/XML examples, e.g.
wget https://geo.woudc.org/wps --post-file=/path/to/request.xml
- the response is an XML document with the results of the Process, in this case the SHADOZ data converted to Extended CSV
- TODO: describe outputs on WOUDC website
WOUDC Validation Services provide real-time format checking as well as quality assessment for potential data submissions. Validation services do not ingest data into WOUDC; rather, they are agile and lightweight tools which expose the functionality of WOUDC data processing to support contributors running real-time checks on their data before submitting to WOUDC as a means to improve data quality over time.
Validation Services are realized via the WOUDC Web Processing Service, which provides support for OGC:WPS 1.0.0. To interact with a WPS, you must submit an XML file as the HTTP request as part of the request/response workflow. The following validation services exist:
-
data-extcsv
: format checker for Extended CSV format -
data-qa
: executes Quality assessment against an Extended CSV file
Let's try running the ExtendedCSV format checkout Process.
Sample XML request:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<wps:Execute service="WPS" version="1.0.0" xmlns:wps="http://www.opengis.net/wps/1.0.0" xmlns:ows="http://www.opengis.net/ows/1.1" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/wps/1.0.0
http://schemas.opengis.net/wps/1.0.0/wpsExecute_request.xsd">
<ows:Identifier>data-extcsv</ows:Identifier>
<wps:DataInputs>
<wps:Input>
<ows:Identifier>data</ows:Identifier>
<wps:Data>
<wps:ComplexData>$RAW_EXTCSV</wps:ComplexData>
</wps:Data>
</wps:Input>
</wps:DataInputs>
</wps:Execute>
Notes:
- Save the following file to disk as
foo.xml
- replace
$RAW_EXTCSV
with an actual ExtendedCSV file (open Extended CSV file in a text editor, select all, copy/paste) - run the command as per one of the HTTP POST/XML examples, e.g.
wget https://geo.woudc.org/wps --post-file=/path/to/request.xml
- the response is an XML document with the results of the Process
- TODO: describe outputs on WOUDC website
This use case covers those who poll WOUDC at given intervals for data updates as part of their downstream processing workflows. The idea here is process new files periodically (i.e. weekly).
See all new files in the index since 2018-01-29
https://geo.woudc.org/ows?service=WFS&version=1.1.0&request=GetFeature&typename=filelist&filter=<Filter><PropertyIsGreaterThanOrEqualTo><PropertyName>processed_datetime</PropertyName><Literal>2018-01-29</Literal></PropertyIsGreaterThanOrEqualTo></Filter>
This provides GML (XML) output. If you wish, you can add 'outputformat=CSV', or 'outputformat=GeoJSON' to get CSV/GeoJSON formats instead.
Filter Stations and Visualize on http://geojson.io
Use WOUDC Web Services to download a station list in GeoJSON format using temporal filters and load the result in http://geojson.io. This will allow a user to visualize stations reporting within a specified timeframe on a map.
https://geo.woudc.org/ows?service=WFS&version=1.1.0&request=GetFeature&typename=stations&filter=<Filter><And><PropertyIsGreaterThanOrEqualTo><PropertyName>date_from</PropertyName><Literal>YYYY-MM-DD HH:MM:SS</Literal></PropertyIsGreaterThanOrEqualTo><PropertyIsLessThanOrEqualTo><PropertyName>date_to</PropertyName><Literal>YYYY-MM-DD HH:MM:SS</Literal></PropertyIsLessThanOrEqualTo></And></Filter>
..where 'YYYY-MM-DD HH:MM:SS' are the "date from" and "date to" respectively. You can also download CSV, KML, or GIS formats using WOUDC Web Services.
Example querying for stations reporting between 2000 and 2011 in various formats:
GML (XML) https://geo.woudc.org/ows?service=WFS&version=1.1.0&request=GetFeature&typename=stations&filter=<Filter><And><PropertyIsGreaterThanOrEqualTo><PropertyName>date_from</PropertyName><Literal>2000-01-01 00:00:00</Literal></PropertyIsGreaterThanOrEqualTo><PropertyIsLessThanOrEqualTo><PropertyName>date_to</PropertyName><Literal>2011-01-01 00:00:00</Literal></PropertyIsLessThanOrEqualTo></And></Filter>&
CSV: https://geo.woudc.org/ows?service=WFS&version=1.1.0&request=GetFeature&typename=stations&filter=<Filter><And><PropertyIsGreaterThanOrEqualTo><PropertyName>date_from</PropertyName><Literal>2000-01-01 00:00:00</Literal></PropertyIsGreaterThanOrEqualTo><PropertyIsLessThanOrEqualTo><PropertyName>date_to</PropertyName><Literal>2011-01-01 00:00:00</Literal></PropertyIsLessThanOrEqualTo></And></Filter>&outputformat=CSV
NetCDF: https://geo.woudc.org/ows?service=WFS&version=1.1.0&request=GetFeature&typename=stations&filter=<Filter><And><PropertyIsGreaterThanOrEqualTo><PropertyName>date_from</PropertyName><Literal>2000-01-01 00:00:00</Literal></PropertyIsGreaterThanOrEqualTo><PropertyIsLessThanOrEqualTo><PropertyName>date_to</PropertyName><Literal>2011-01-01 00:00:00</Literal></PropertyIsLessThanOrEqualTo></And></Filter>&outputformat=NetCDF
GeoJSON: https://geo.woudc.org/ows?service=WFS&version=1.1.0&request=GetFeature&typename=stations&filter=<Filter><And><PropertyIsGreaterThanOrEqualTo><PropertyName>date_from</PropertyName><Literal>2000-01-01 00:00:00</Literal></PropertyIsGreaterThanOrEqualTo><PropertyIsLessThanOrEqualTo><PropertyName>date_to</PropertyName><Literal>2011-01-01 00:00:00</Literal></PropertyIsLessThanOrEqualTo></And></Filter>&outputformat=GeoJSON
KML: https://geo.woudc.org/ows?service=WFS&version=1.1.0&request=GetFeature&typename=stations&filter=<Filter><And><PropertyIsGreaterThanOrEqualTo><PropertyName>date_from</PropertyName><Literal>2000-01-01 00:00:00</Literal></PropertyIsGreaterThanOrEqualTo><PropertyIsLessThanOrEqualTo><PropertyName>date_to</PropertyName><Literal>2011-01-01 00:00:00</Literal></PropertyIsLessThanOrEqualTo></And></Filter>&outputformat=OGRKML
- determine a start date and end date to support a temporal filter (format
YYYY-MM-DD HH:MM:SS
) - run a WFS query with a temporal filter:
https://geo.woudc.org/ows?service=WFS&version=1.1.0&request=GetFeature&typename=stations&filter=<Filter><And><PropertyIsGreaterThanOrEqualTo><PropertyName>date_from</PropertyName><Literal>$DATE_FROM</Literal></PropertyIsGreaterThanOrEqualTo><PropertyIsLessThanOrEqualTo><PropertyName>date_to</PropertyName><Literal>$DATE_TO</Literal></PropertyIsLessThanOrEqualTo></And></Filter>&outputformat=GeoJSON
- download the result (
result.json
) as a file on your desktop - open a web browser and visit http://geojson.io
- using the
File/Open
menu, navigate to the downloaded file - click on a given point to see the associated properties
- click on the
Table
link (top right) to visualize the station data in a tabular display
WFS based station query can be visualized using online web applications.