Produce Heat Wave Hazard Events for Europe

[p-a-s-c-a-l]

we need some sort of loook-up table for determining for each area in Europe the event-probability (maybe implemented as REST service).
-->action: ZAMG will take care of generating it --> For each grid point we have one frequent event, occasional and rare event (this is done for the baseline scenario + the 3 rcps scenarios (and for each rcp there are 3 future scenarios))
--> the idea is to keep it as possible for the screening phase. More complex solution for selecting/applying the hazard events to the study area could be part of the definition of a future business model (e.g., provide better methods of calculation in a pay mode)

[p-a-s-c-a-l]

@RobAndGo:
Sorry for my late reply, but I had to remove some errors from the method I was using and then confirm that it would work.
Anyhow, here is just some preliminary results of what I thought I could deliver.

I am in the process of updating the heat wave hazard events and determining the events:

• "Frequent" (Probability = 1.0; yearly event),
• "Occasional" (Probability = 0.2; 1 in 5 year event),
• "Rare" (Probability = 0.05; 1 in 20 year event)

A subset of the data looks like this:

RCP_Period_Xcoord_Ycoord;Frequent(1.0);Occasional(0.2);Rare(0.05);
rcp45_20110101-20401231_X1_Y1;2d_30C;2d_38C;19d_36C;
rcp45_20110101-20401231_X1_Y2;2d_34C;3d_36C;9d_36C;
rcp45_20110101-20401231_X1_Y3;2d_28C;12d_30C;6d_34C;
rcp45_20110101-20401231_X1_Y4;3d_28C;11d_32C;12d_34C;
rcp45_20110101-20401231_X1_Y5;2d_36C;7d_34C;2d_38C;
rcp45_20110101-20401231_X1_Y6;4d_34C;5d_36C;14d_36C;
rcp45_20110101-20401231_X1_Y7;3d_38C;5d_40C;12d_40C;
rcp45_20110101-20401231_X1_Y8;2d_40C;12d_36C;19d_38C;
rcp45_20110101-20401231_X1_Y9;3d_34C;3d_38C;10d_36C;
rcp45_20110101-20401231_X1_Y10;2d_28C;8d_28C;2d_32C;
rcp45_20110101-20401231_X1_Y11;3d_24C;5d_26C;13d_26C;
rcp45_20110101-20401231_X1_Y12;4d_28C;6d_30C;2d_32C;
rcp45_20110101-20401231_X1_Y13;3d_32C;7d_32C;11d_32C;
rcp45_20110101-20401231_X1_Y14;2d_40C;4d_40C;5d_40C;
rcp45_20110101-20401231_X1_Y15;3d_38C;5d_40C;8d_40C;
rcp45_20110101-20401231_X1_Y16;3d_38C;6d_40C;11d_40C;
rcp45_20110101-20401231_X1_Y17;3d_36C;9d_38C;10d_40C;
rcp45_20110101-20401231_X1_Y18;3d_38C;7d_40C;14d_38C;
rcp45_20110101-20401231_X1_Y19;4d_36C;6d_40C;9d_40C;
rcp45_20110101-20401231_X1_Y20;2d_40C;7d_38C;9d_40C;
rcp45_20110101-20401231_X1_Y21;2d_38C;8d_36C;7d_40C;
rcp45_20110101-20401231_X1_Y22;2d_38C;5d_40C;7d_40C;
rcp45_20110101-20401231_X1_Y23;3d_36C;5d_40C;9d_38C;
rcp45_20110101-20401231_X1_Y24;2d_38C;5d_38C;15d_32C;
rcp45_20110101-20401231_X1_Y25;5d_28C;3d_38C;7d_34C;
rcp45_20110101-20401231_X1_Y26;7d_26C;3d_34C;2d_38C;
rcp45_20110101-20401231_X1_Y27;5d_24C;2d_32C;2d_34C;
rcp45_20110101-20401231_X1_Y28;;;2d_24C;
rcp45_20110101-20401231_X1_Y29;;;;
rcp45_20110101-20401231_X1_Y30;;;;

The fields are defined in the header (first) line and are separated by the delimiter ;
Field description:

1. rcp26, rcp45, rcp85 or baseline _
   period: 19710101-20001231, 20110101-20401231, 20410101-20701231, 20710101-21001230 _
   X coordinate _
   Y coordinate
2. Frequent event represented as duration in days and temperature in C, i.e. 2d_30C
3. Occasional event represented as duration in days and temperature in C, i.e. 2d_30C
4. Rare event represented as duration in days and temperature in C, i.e. 19d_36C

Note that not all grid points will have heat wave cases (e.g. grid points X1_Y28, X1_Y29, X1_Y30...).

I have the coordinates for each grid box within the EURO-CORDEX data set. The EURO-CORDEX files themselves reference grid coordinates in the form X,Y. The corresponding latitude,longitude I have in a separate file of the form:

x y : latitude longitude
1 1 : 29.952276 -8.075616
1 2 : 30.055091 -8.120772
1 3 : 30.157890 -8.166022
1 4 : 30.260673 -8.211366
1 5 : 30.363441 -8.256806
1 6 : 30.466193 -8.302341
1 7 : 30.568929 -8.347972
1 8 : 30.671650 -8.393699
1 9 : 30.774354 -8.439524
1 10 : 30.877042 -8.485447
1 11 : 30.979714 -8.531469
1 12 : 31.082369 -8.577590
1 13 : 31.185008 -8.623810
1 14 : 31.287631 -8.670131
1 15 : 31.390237 -8.716553
1 16 : 31.492826 -8.763076
1 17 : 31.595398 -8.809701
1 18 : 31.697953 -8.856429
1 19 : 31.800491 -8.903261
1 20 : 31.903012 -8.950197
1 21 : 32.005516 -8.997237
1 22 : 32.108002 -9.044383
1 23 : 32.210471 -9.091634
1 24 : 32.312922 -9.138992
1 25 : 32.415356 -9.186458
1 26 : 32.517771 -9.234031
1 27 : 32.620169 -9.281713
1 28 : 32.722548 -9.329504
1 29 : 32.824910 -9.377405
1 30 : 32.927253 -9.425416

What I want to do was to replace the X,Y grid coordinates in the first file with the latitude and longitude, so that all the information is in one file. However, I realised that the original data was not quite right, which is why I was delayed with this response.

The plan here is to have separate data files showing the heat wave events for:

1. baseline 19710101-20001231
2. rcp26 20110101-20401231
3. rcp26 20410101-20701231
4. rcp26 20710101-21001231
5. rcp45 20110101-20401231 - presented above
6. rcp45 20410101-20701231
7. rcp45 20710101-21001231
8. rcp85 20110101-20401231
9. rcp85 20410101-20701231
10. rcp85 20710101-21001231

That is, there will be a total of 10 files. Each file will have the data for 345 x 351 = 121095 grid points. At the moment I will have the files in ASCII format and each would have a size of about 10Mb.

I hope this comment makes sense. I will be away from work for the rest of the week, so any responses from my side won't come until next week.

[p-a-s-c-a-l]

A subset of the data looks like this:
...

@humerh is this what you expected? Can you work with this in Emikat?

Next steps:

1. select the event of one grid point/raster cell that is representatives for all raster cells the study area.
2. calculate LE for all unique Hazard Events in this cell, regardless of RCP and time period. Unfortunately, it's doesn't seem enough to calculate just the 3 event of the baseline period and then work with probabilities. Future events may be different from those occurring in baseline. Right? @RobAndGo

[p-a-s-c-a-l]

Example:
at the same grid point you may have Frequent;Occasional;Rare:

• Baseline: 2d_30C, 2d_38C, 19d_36C;
• rcp85 20710101-21001231: 4d_40C, 8d_42C, 21d_40C;

So it's not to enough to consider just the baseline.

[humerh]

Some remarks to the proposed data set:

• From my point of view this data set seems to be complete. Thanks to Robert for this quick implementation.

Nevertheless I have some remarks:

• How are these points defined? Are these points the middle points of the 12km*12km Areas?

• As the concept of "Events" is not only bounded to "Heat-Wave" Events, this kind of mapping may also be applyable to other Hazard types. For this reason the mapping file can also be expanded by context information, which binds this mapping to a specific HAZARD_TYPE. A very simple approach whould be to add a prefix to the Event-Name (Instead 6d_40C --> HW.6d_40C). Another solution can be to add a column expressing the hazard type (like: HW, or the long taxonomy term "hazard:temperature:heat:heat-wave:index:csu").

• I asume, that the CSIS-user should know, which event is "rare" or "frequent". So also CSIS has to resolve this mapping. For this reason there should be a service (REST or geoserver), which returns these "events" to the user. A possible signature of the function can be:
  Request parameters: longitute, latitude, scenario-name(rcp), time-period-name [, hazard-type]
  Response fields: names of the events for frequent/occasional/rare

• If all aggree to the previous point someone has to implement this mapping service. Can this be implemented by ATOS/Meteogrid?
  Currently EMIKAT does not need this information for the calculation, as we can also calculate for ALL events. CSIS has to select the proper events, depending on the request context (rcp/time/location).

[p-a-s-c-a-l]

Considering time and resource left I'm not very keen to introduce another service if there isn't a strong need for it now, unless ATOS/Meteogrid tell us that this service can be implemented in no time. I'm also not sure for which purpose this service is used. Is it only needed if you use the same table (?) for different event types? Where is the event data stored, btw? In EMIKAT? Files? PostGIS?

In the context tab of the study, should the user preselect one RCP and one time period.? Then only the baseline and one rcp/time period would have to be calculated.

Any comments to this comment? :)

calculate LE for all unique Hazard Events in this cell, regardless of RCP and time period. Unfortunately, it's doesn't seem enough to calculate just the 3 event of the baseline period and then work with probabilities. Future events may be different from those occurring in baseline. Right?

[ghilbrae]

We are a bit confused about the need and goal of this service too. What you are suggesting is that we'd need a new database or file or whatever in which to store a set of tables to query, isn't it? Shouldn't this information be stored along with the calculations for the LE or in the CSIS if the user is to select this information beforehand?
Thanks for any clarification!

[humerh]

This mapping is in my eyes needed for dynamically select the right Events, which are dependend of the location, RCP-Scenario and Time-Periode.

I have the picture, this is needed in the user front-end CSIS.

If the CSIS client is not interested in showing the list of the representative events for an arbitratry Study area, then Emikat can also handle this information as private information.

In this case only Emikat knows, which Events are used for the calculation of Local Events and Impacts.

The CSIS client would only be able to select LE, Impact and riscs by seelcting one of following parameters:

• by the name of an EVENT (without knowing, if this event is representative)
• by the triple (RCP-Scenario, Time-Period, Hazard-Type) without knowing, which Event is used for calculation.

[RobAndGo]

OK, I'm back. I hope I don't confuse things as I reply to the comments in their order.

From @p-a-s-c-a-l

Next steps:

1. select the event of one grid point/raster cell that is representatives for all raster cells the study area.
2. calculate LE for all unique Hazard Events in this cell, regardless of RCP and time period. Unfortunately, it's doesn't seem enough to calculate just the 3 event of the baseline period and then work with probabilities. Future events may be different from those occurring in baseline. Right? @RobAndGo

Yes, the future events may be different from those occurring in the baseline. This is why I need the 10 files I outlined above (baseline period, rcp26 x 3 future periods, rcp45 x 3 future periods, rcp85 x 3 future periods).

Example:
at the same grid point you may have Frequent;Occasional;Rare:

• Baseline: 2d_30C, 2d_38C, 19d_36C;
• rcp85 20710101-21001231: 4d_40C, 8d_42C, 21d_40C;

So it's not to enough to consider just the baseline.

Exactly!

[RobAndGo]

From @humerh

• How are these points defined? Are these points the middle points of the 12km*12km Areas?

I believe these points represent the middle points of the 12km*12km areas. I will check to see that this is the case.

• As the concept of "Events" is not only bounded to "Heat-Wave" Events, this kind of mapping may also be applyable to other Hazard types. For this reason the mapping file can also be expanded by context information, which binds this mapping to a specific HAZARD_TYPE. A very simple approach whould be to add a prefix to the Event-Name (Instead 6d_40C --> HW.6d_40C). Another solution can be to add a column expressing the hazard type (like: HW, or the long taxonomy term "hazard:temperature:heat:heat-wave:index:csu").

This makes sense(!) and I can implement it.

[RobAndGo]

Here are some segments of the data file that I am producing:

europe_rcp45_20110101-20401231_X001-049.csv.txt
europe_rcp45_20110101-20401231_X110-159.csv.txt
europe_rcp45_20110101-20401231_X240-289.csv.txt

They are .csv files (I had to add the .txt extension in order to upload the file). The column headings give the description of the data.

At the moment this is only the RCP45 scenario for the period 20110101-20401231 and is only a section of the complete EURO-CORDEX grid (i.e. it represents nearly 50% of the entire grid - x-coordinates 001-049, 110-159, 240-289). Hopefully it is of use now and can help with further developments!

[humerh]

From my point of view this makes sense!

The only "improvement suggestion" would be to add information about the "Hazard Type" (context) of the event, because this approach can also be applied to other Hazard Types. (see discussion above)

[RobAndGo]

I finally have finished products for the baseline period (19710101-20001231), RCP26, RCP45 and RCP85, periods 20110101-20401231, 20410101-20701231, 20710101-21001231 and have attached them here:

europe_baseline_19710101-20001231.csv.txt
europe_rcp26_20110101-20401231.csv.txt
europe_rcp26_20410101-20701231.csv.txt
europe_rcp26_20710101-21001231.csv.txt
europe_rcp45_20110101-20401231.csv.txt
europe_rcp45_20410101-20701231.csv.txt
europe_rcp45_20710101-21001231.csv.txt
europe_rcp85_20110101-20401231.csv.txt
europe_rcp85_20410101-20701231.csv.txt
europe_rcp85_20710101-21001231.csv.txt

Once again they are .csv files (remove the .txt extension). The first two columns are latitude and longitude. I have added 2 additional columns in the position of column 3 and 4. Column 3 shows the urban_area name as defined in the Copernicus Urban Atlas and column 4 shows the country. As it turned out, each urban_area is represented by only 1 grid point.

[see https://land.copernicus.eu/local/urban-atlas/urban-atlas-2012?tab=download for a list of available towns within Urban Atlas]

At the moment I have only included the urban areas from Italy, Sweden, Austria, Spain, Germany, and Belgium. If this format is desired, I could extend this to include all of the Urban Atlas towns.

I have noticed that there are some omissions from my data as only 164 from the available 282 towns are included, e.g. Madrid is not present. This omission is no doubt due to my search distance for the nearest grid point to the town being perhaps too small. I will look into this further.

Note that there may be issues with special characters in the town names (e.g. for Germany characters like ß, ä,ö, ü may not come out right. For Italy, Spain, and Sweden I converted the special characters to their normal characters, e.g. for o, i, e, etc).

[p-a-s-c-a-l]

Thank you, Robert. So next step is to feed this data into Emikat, right?

[p-a-s-c-a-l]

@RobAndGo @humerh
Are all events now available in EMIKAT? Can we close this issue?

[p-a-s-c-a-l]

done

[p-a-s-c-a-l]

Changes in technical implementation:

• Heat waves are defined as a period of 2 days or more exceeding a certain threshold temperature.
• Frequent, Occasional, and Rare events will still exist with the same probability definitions as before (1.0, 0.2, 0.05), but will be called “heat waves of 2 days or more”. This means that the new data files will be similar to the old ones and will look like this:
  latitude;longitude;urban area;country;RCP;Period;Tx95p;Frequent(1.0);Occasional(0.2);Rare(0.05)
  30.055091;-8.120772;;;baseline;19710101-20001231;39C;36C;38C;40C

NOTE 1: there are 2 changes:

• i) there is no duration in the form of “2d_” before the temperature, and

• ii) there is a new column called “Tx95p” (see point 4 below).

NOTE 2: I will have to recalculate the temperatures according to the new definition, and calculate Tx95p, but the format of the output will be as shown above. I have attached the full file of the baseline case with this email. I can make available the other files if necessary. Once again, the temperatures serve as place-holders only and are not real values.

• For each gridpoint, the Tx95p (95th percentile of the maximum temperature for the warm months [Apr-Oct]) is calculated by me. In the example above, I have set this constant to 39C, but this is just a place-holder.

• Based on PLINIVS’ model, mortality (deaths) only occur for heat wave events (frequent, occasional, rare) when that temperature shown is equal to or exceeds the Tx95p temperature. So, in the example above, the frequent heat waves and the occasional heat waves will not result in mortality (36C < 39C, 38C < 39C), but the rare heat waves will result in mortality (40C > 39C). How mortality relates to temperature is given by a curve/mathematical function from PLINIVS, with higher temperatures leading to higher mortality.

[RobAndGo]

I am sorry this has taken longer than expected, but we had some computer storage problems which meant the scripts to automatically produce the data did not work as expected. Anyways, here is the first taste of real data.

HeatWave_new_DCs.zip

These are .csv files showing the heat wave events (frequent, occasional, rare) only in terms of the maximum temperature for the baseline + 3 future RCP scenarios. Due to time contraints, I have limited myself to locations of the EURO-CORDEX data corresponding only the DC-cases.

Here is a snippet of the data:

latitude;longitude;urban area;country;RCP;Period;Tx95p;Frequent(1.0);Occasional(0.2);Rare(0.05)
41.052959;14.266241;"Naples";"Italy";rcp26;20110101-20401231;33.2;35;37.5;38

The new column "Tx95p" represents the 95th percentile of maximum temperature for the baseline period and is copied into the future scenario files as well, to help in comparing the temperatures of the heat waves.

The temperatures of the heat waves are steps of 0.5°C only.

I will try to generate such data for all of Europe, and I hope something of it will be ready by next week (today is my last day this week).

[RobAndGo]

Here are the heat wave events for the baseline climate for all of Europe:
europe_historical_19710101-20001231.csv.txt

Here are the heat wave events for the RCP26 future scenario for all of Europe:
europe_rcp26_20110101-20401231.csv.txt
europe_rcp26_20410101-20701231.csv.txt
europe_rcp26_20710101-21001231.csv.txt

Here are the heat wave events for the RCP45 future scenario for all of Europe:
europe_rcp45_20110101-20401231.csv.txt
europe_rcp45_20410101-20701231.csv.txt
europe_rcp45_20710101-21001231.csv.txt

Here are the heat wave events for the RCP85 future scenario for all of Europe:
europe_rcp85_20110101-20401231.csv.txt
europe_rcp85_20410101-20701231.csv.txt
europe_rcp85_20710101-21001231.csv.txt

[humerh]

As stated in the new document „impact calculation_mortality.docx“ we need for calculation of total mortality the durations of an event too.

As this information is currently not available we should add this information to this file for each of the included temperatures.

New: Duration_Rare, Duration_occassional, Duration_frequent
@RobAndGo : What do you mean about this?

[RobAndGo]

Hmm, I was unaware of this new "impact calculation_mortality.docx" document. Could you please make it available, @humerh.
I thought that the duration did not play a role in the new calculations, which is why I have not included it in the new data. Giulio had said that a heat wave is only important when it has a duration of 2 days or more, and then the mortality is calculated based only on the temperature.

In the table above shown by @humerh, Giulio used this in his presentation to mean that:

1. the number of heatwaves with durations of 1 day are omitted (grey row), and
2. the number of heatwaves with durations >= 2 days are summed together (arrow pointing downward). That is, exact data on the duration is no longer necessary (except that the duration is at least 2 days long).

[humerh]

You find attached the document "impact calculation_mortality.docx", which I found as summary document of plenary meeting. One of the consequences of this document is - and I asume that this document was discussed and has been agreed - that we still need the duration of events!

impact calculation_mortality.docx

Solution 1: We also import the base statistics for each raster cell provided be ZAMG some months ago

Solution 2: @RobAndGo includes this duration in the extrated records (see above)

Solution 3: We ignore this document.(???)

[RobAndGo]

My final edit with the remaining data:

• baseline
• RCP26, period 20110101-20401231, 20410101-20701231, 20710101-21001231
• RCP45, periods 20110101-20401231, 20410101-20701231, 20710101-21001231
• RCP85, period 20110101-20401231, 20410101-20701231, 20710101-21001231

for all of Europe:

europe_historical_19710101-20001231.csv.txt
europe_rcp26_20110101-20401231.csv.txt
europe_rcp26_20410101-20701231.csv.txt
europe_rcp26_20710101-21001231.csv.txt
europe_rcp45_20110101-20401231.csv.txt
europe_rcp45_20410101-20701231.csv.txt
europe_rcp45_20710101-21001231.csv.txt
europe_rcp85_20110101-20401231.csv.txt
europe_rcp85_20410101-20701231.csv.txt
europe_rcp85_20710101-21001231.csv.txt

The events are now characterised similar to the very first datasets which I produced:

latitude;longitude;urban area;country;RCP;Period;Tx95p;Frequent(1.0);Occasional(0.2);Rare(0.05)
30.157890;-8.166022;;;rcp45;20110101-20401231;36.0;38.5_2.600d;42_0.333d;43.5_0.066d

where a temperature and a duration is given, e.g.
38.5_2.600d represents 38.5°C and duration 2.600 days.

Note that this duration represents the sum of all days within heat waves with durations of 2days or longer, averaged over 30 years.

I will be on holiday from 5-9 August, so any requests to me during this time won't be solved until the following week.

[humerh]

I think, the first line should be like:
latitude;longitude;urban_area;country;RCP;Period;Tx95p;Frequent(1.0);Frequent(1.0);Occasional(0.2);Rare(0.05)

Do you agree? I need the first line for the import.

2. duration 2.600 days: '.' is the decimal point, not a thousend seperator, isn't it?

[RobAndGo]

Sorry, that was an oversight from me!! :-(
I have removed the second occurrence of "latitude;longitude;" from the header and corrected it in my previous post. So the correct header is:
latitude;longitude;urban area;country;RCP;Period;Tx95p;Frequent(1.0);Occasional(0.2);Rare(0.05)

Yes, in all heat wave durations, the "." is indeed a decimal point in 2.600 days.

[p-a-s-c-a-l]

Hm, why aren't there dedicated columns for the temperature and the duration?

[humerh]

Sorry, I did not see, that the other data files are now also available. I will import them in the next hours.

[humerh]

Comments of the coding of the Events in that way, as Robert suggested:

This coding format was not my idea, but for me this is ok.

We define here Hazard events, which are relevant for a given area.
We characterize them as FREQUENT, OCCASIONAL and RARE.
The definition of the corresponding Temperature and Duration is typically for "Heat wave" and can here be described by a String descriptor "degree_duration".
As such defintion can also be found for other HAZARD_EVENT_TYPES, we will for these cases have other characteristic parameters.
This form of a String descriptor is open for other HAZARD EVENT TYPEs, so I also used this format in EMIKAT to stay open.

[RobAndGo]

Yes, as @humerh mentioned, the durations are dependent on that threshold temperature, which is why I have linked them together into one column as "temperature_duration" for each of the events (frequent, occasional, rare). I had done this for the other data sets previously posted, and since no one complained, I thought this format was acceptable.