[DATA] Access to global datasets from 3rd parties

[matamadio]

Overview

Global datasets are oftern the only source of data for country scale risk analysis in developing countries.
GFDRR gets a lot of request for these kind of data to be used in projects.
We need to address if and how these datasets are going to be accessed via RDL and its upcoming workflows.

Example data

Exposure:

• Worldpop
• Open Street Map

Hazard (see also sheet)

• GAR17 hazards
• IbTracs (Cyclones)
• FWI (Wildfires)
• FAO-ASI (Droughts)
• WRI Flood maps

Use cases

• Download data to use in GIS analytics
• Visualisation of map in GIS or WebGIS
• Processing in online analytical tools (e.g. clip to country extent, overlay with other data)

Options

Options for the inclusion of data have balanced pros and cons.

1. Include in the catalogue as metadata; download links points to original source (download page); link to API (data access) and WMS (data view) whenever available from source (e.g. OSM)
(+) No storage used
(+) Always up to date (new versions, etc)
(-) TIed to original data format, which may not be optimal (in particular for any pre-set analytical tool)
(-) Become inaccessible if source is discontinued

2. Include in the catalogue pointing to a copy of the data in the RDL storage
(+) Reformat as optimal for workflow and alignment to schema
(+) Access granted independently of source changes
(-) Storage used
(-) Need manual updating of versions

[stufraser1]

In Short term, let's create metadata in RDL format and provide original URL. Longer term we could look to bring some data across but this is not urgent and would require agreement of each data provider.

…

________________________________ From: Mattia Amadio ***@***.***> Sent: Monday, March 6, 2023 2:32:45 PM To: GFDRR/rdl-standard ***@***.***> Cc: Stuart Fraser ***@***.***>; Assign ***@***.***> Subject: [GFDRR/rdl-standard] Access to global datasets from 3rd parties (#31) Overview Global datasets are oftern the only source of data for country scale risk analysis in developing countries. GFDRR gets a lot of request for these kind of data to be used in projects. We need to address if and how these datasets are going to be accessed via RDL and its upcoming workflows. Example data Exposure: * Worldpop<https://www.worldpop.org/project/categories?id=3> * Open Street Map<https://wiki.openstreetmap.org/wiki/Databases_and_data_access_APIs> Hazard (see also sheet<https://worldbankgroup-my.sharepoint.com/:x:/g/personal/mamadio_worldbank_org1/EdmT0D0YD1BOqlBYOJNKiioBFsOmaC1mnzcGPPBSxzA5Zw?e=EdN7WZ>) * GAR17 hazards<https://risk.preventionweb.net/capraviewer/> * IbTracs<https://www.ncei.noaa.gov/access/metadata/landing-page/bin/iso?id=gov.noaa.ncdc:C00834> (Cyclones) * FWI (Wildfires) * FAO-ASI<http://www.fao.org/giews/earthobservation/asis/index_1.jsp?lang=en> (Droughts) * WRI Flood maps<https://www.wri.org/data/aqueduct-floods-hazard-maps> Use cases * Download data to use in GIS analytics * Visualisation of map in GIS or WebGIS * Processing in online analytical tools (e.g. clip to country extent, overlay with other data) Options Options for the inclusion of data have balanced pros and cons. 1. Include in the catalogue as metadata; download links points to original source (download page); link to API (data access) and WMS (data view) whenever available from source (e.g. OSM) (+) No storage used (+) Always up to date (new versions, etc) (-) TIed to original data format, which may not be optimal (in particular for any pre-set analytical tool) (-) Become inaccessible if source is discontinued 2. Include in the catalogue pointing to a copy of the data in the RDL storage (+) Reformat as optimal for workflow and alignment to schema (+) Access granted independently of source changes (-) Storage used (-) Need manual updating of versions — Reply to this email directly, view it on GitHub<#31>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AC7PNYURPBC77DCEOPXWYZLW2XYQ3ANCNFSM6AAAAAAVRGNYZA>. You are receiving this because you were assigned.Message ID: ***@***.***>

[pzwsk]

Agree that option 1, i.e. referencing and pointing to the original source through the Risk Data Library catalog would already be a nice added value. I would imagine we could communicate on the fact the World Bank has improved its curation of global hazard layers thanks to the Risk Data Library Standard. Let's add this to FY24 workplan.

For option 2, I would need a better understanding of the efforts needed and the potential workflow but I don't think our role would be to become a data warehouse whereby we transform and maintain a copy of the datasets. Rather, we should seek collaboration with global data producers for them to provide the data according to our standards and/or transformation tools.