-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
assess the WIS 1.0 catalogue for availability of NWP data #103
Comments
Update 2021-05-21: @efucile will provide list of centers providing NWP. @josusky and @tomkralidis to develop MVP. |
@tomkralidis and @josusky this is a list of GDPFS centres. There is a column saying which of them is supposed to provide data in WIS. This can be used to check which of the centres in the list is also providing data in the catalogue. |
Just for the record,, the whole WIS metadata catalogue can be downloaded from https://gisc.dwd.de/oaidownload/wis-catalogue.tar.gz. I guess that can be considered and authoritative source :-) |
Yes, it's everything. At https://gisc.dwd.de/wisportal/# (see "Metadata") you can choose between all or only GISC Toulouse or only GISC Offenbach. Should be updated every night. |
I am pasting here part of the discussion with Tom over Slack: |
Aside: I've captured the WIS catalogue location information in the wiki for future use: https://github.com/wmo-im/wcmp/wiki/WISMetadataCatalogue#wis-10-metadata-catalogue Please feel free to update. |
An initial implementation can be found in https://github.com/wmo-im/pywiscat; @josusky to review/update accordingly, at which point we'll be able to provide an analysis of NWP data. |
Different search result numbers occure due to differences in search query Search over [SolR-supported operators: AND – alternative symbol: &&, NOT – alternative symbol: !, OR – alternative symbol: || [DEFAULT]] Of interest could possibly also be our new REST API in the WIS Portal. A first version of a doc is available at |
COR: /search/startSearch --> GET |
Hi, a first very crude test (looking only for the word "GRIB") yielded the following result (in the JSON output non-ASCII characters are encoded): {
"NMC FRANCE - M\u00e9t\u00e9o-France": 1531,
"NMC UNITED KINGDOM - Met Office": 15069,
"ECMWF": 2755,
"GISC Tokyo - Japan Meteorological Agency": 21372,
"Max-Planck-Institut fuer Meteorologie": 46,
"Deutscher Wetterdienst": 1246,
"National Meteorological Information Center, CMA": 126,
"WMO Lead Centre for Long-Range Forecast Multi-Model Ensemble": 120,
"FSBE \u00abAviamettelecom of Roshydromet\u00bb": 2182,
"Deutsches Klimarechenzentrum": 8,
"Japan Meteorological Agency": 193,
"OSI SAF": 28,
"European Centre for Medium-Range Weather Forecasts": 7,
"Forschungszentrum Karlsruhe": 3,
"Commonwealth Scientific & Industrial Research Organisation": 4,
"University of Hohenheim": 2,
"Canadian Centre for Climate Modelling and Analysis": 2,
"NOAA": 1,
"EUMETSAT": 25,
"H SAF": 14,
"WMO/WIS/GISC Tokyo": 10,
"Deutscher Wetterdienst (RD)": 2,
"Institute for Meteorology, Freie Universit\u00e4t Berlin": 1,
"DCPC-Adriatic Marine Meteorological Centre": 8,
"ZAMG - Central Institute for Meteorology and Geodynamics": 2,
"Agenzia Regionale Preventione e Ambiente dell'Emilia-Romagna": 2,
"Deutscher Wetterdienst (ZAK)": 2,
"CNMCA (Pratica di Mare)": 2,
"NMC BULGARIA - National Institute of Meteorology and Hydrology": 3,
"University of Toulouse": 2,
"Max-Planck-Institut fuer Meteorologie (MD)": 5,
"National Institute for Environmental Studies": 6,
"Met Office Hadley Centre": 4,
"Institut f\u00fcr Meteorologie der Freien Universit\u00e4t Berlin": 3,
"Instituto Nacional de Meteorologia": 1,
"Centre National de Recherches M\u00e9t\u00e9orologiques": 2,
"Istituto Superiore per la Protezione e la Ricerca Ambientale (ex APAT)": 2,
"Met Office": 2,
"Institute of Atmospheric Sciences and Climate": 2,
"Meteo-France": 1,
"Geophysical Fluid Dynamics Laboratory/NOAA": 2,
"Environment Canada": 2,
"WMO/WIS/DCPC Tokyo (Global Producing Centre for long-range forecast)": 1,
"Federal Office of Meteorology and Climatology MeteoSwiss": 2,
"South East European Virtual Climate Change Center (SEEVCCC)": 3,
"ARPA-Servizio IdroMeteorologico": 2,
"Agenzia Regionale per la Protezione dell'Ambiente Ligure": 1,
"Helmholtz-Zentrum Geesthacht, Zentrum f\u00fcr Material- und K\u00fcstenforschung GmbH": 1,
"WMO": 1,
"DCPC Rome (RTH)": 1,
"Czech hydrometeorological institude": 1,
"NMC KENYA - Kenya Meteorological Department": 1
} this shows several potential issues. Several centers are listed more than once, for example, "ECMWF" is obviously the same thing as "European Centre for Medium-Range Weather Forecasts". And there is granularity again. JMA published 21372 records while DWD "only" 1246, but that does not mean that JMA is doing its job by an order of magnitude better :-) |
@efucile , sorry for the delay. I have run an updated version of pywiscat (that groups the output by citation authority extracted from the file identifier URI) after our last teleconference but forgot to publish the result. Here it is (this time UTF-8 encoded, thus the non-ASCII characters are more readable):
|
cc @efucile
As a result of the forthcoming WMO data policy, assess WIS 1.0 catalogue to determine the availability of NWP data on WIS.
The text was updated successfully, but these errors were encountered: