-
Notifications
You must be signed in to change notification settings - Fork 1
/
data-prep.Rmd
671 lines (577 loc) · 30.1 KB
/
data-prep.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
---
title: "Preparing the data"
author: "Robert Schlegel"
date: "2020-02-25"
output: workflowr::wflow_html
editor_options:
chunk_output_type: console
csl: FMars.csl
bibliography: MHWflux.bib
---
```{r global_options, include = FALSE}
knitr::opts_chunk$set(fig.width = 8, fig.align = 'center',
echo = TRUE, warning = FALSE, message = FALSE,
eval = TRUE, tidy = FALSE)
```
## Introduction
The purpose of this vignette is to walk through the steps taken to prepare the data from the NOAA OISST, ERA5, and GLORYS products for the following analyses. The statistical analyses may be found in the [MHWs vs. heat flux](https://robwschlegel.github.io/MHWflux/mhw-flux.html) vignette, and the SOM analysis may be found in the [SOM](https://robwschlegel.github.io/MHWflux/som.html) vignette.
All of the libraries and functions used in this vignette, and the project more broadly, may be found [here](https://github.com/robwschlegel/MHWflux/blob/master/code/functions.R).
```{r satrtup}
# get everything up and running in one go
source("code/functions.R")
library(ggpubr)
library(gridExtra)
library(FNN)
```
## Study area
A reminder of what the study area looks like. It has been cut into 6 regions, adapted from work by @Richaud2016.
```{r region-fig}
frame_base +
geom_polygon(data = NWA_coords, alpha = 0.7, size = 2,
aes(fill = region, colour = region)) +
geom_polygon(data = map_base, aes(group = group))
```
## Pixels per region
We will now calculate which pixels fall within which region so we can later determine how to create our average SST time series per region as well as the other averaged heat flux, atmospheric, and oceanic time series.
For the SOM we will want to have everything on the same 1/4 degree grid. For the ERA5 data this is an easy shift to make, but for the GLORYS 1/12 degree data we want to ensure that these smaller pixels are being matched correctly to the nearest centre of an OISST pixels. To do this we will create an index of pairings using a fastest nearest neighbour search.
```{r grid-points, eval=FALSE}
# Load one OISST file to extract the lon/lat coords
OISST_grid <- readRDS("../data/OISST/daily/1982/daily.1982-01-01.Rda") %>%
dplyr::select(lon, lat) %>%
filter(lon >= NWA_corners[1], lon <= NWA_corners[2],
lat >= NWA_corners[3], lat <= NWA_corners[4]) %>%
unique() %>%
mutate(OISST_index = 1:n()) # For merging to GLORYS grid
saveRDS(OISST_grid[,1:2], "metadata/OISST_grid.Rda")
# Load one high-res GLORYS file to extract the lon/lat coords
GLORYS_grid_base <- tidync(GLORYS_files[1]) %>%
hyper_tibble() %>%
dplyr::rename(lon = longitude, lat = latitude) %>%
filter(lon >= NWA_corners[1], lon <= NWA_corners[2],
lat >= NWA_corners[3], lat <= NWA_corners[4]) %>%
dplyr::select(lon, lat) %>%
unique()
# Add on the nearest OISST coords
GLORYS_grid <- GLORYS_grid_base %>%
mutate(OISST_index = as.vector(knnx.index(as.matrix(OISST_grid[,c("lon", "lat")]),
as.matrix(GLORYS_grid_base[,c("lon", "lat")]), k = 1))) %>%
left_join(OISST_grid, by = "OISST_index") %>%
dplyr::select(-OISST_index) %>%
dplyr::rename(lon = lon.x, lat = lat.x, lon_OISST = lon.y, lat_OISST = lat.y)
saveRDS(GLORYS_grid, "metadata/GLORYS_grid.Rda")
# Load one ERA5 file to get the lon/lat coords
ERA5_grid <- tidync(ERA5_lhf_files[1]) %>%
hyper_filter(latitude = dplyr::between(latitude, NWA_corners[3], NWA_corners[4]),
longitude = dplyr::between(longitude, NWA_corners[1]+360, NWA_corners[2]+360),
time = index == 1) %>%
hyper_tibble() %>%
dplyr::rename(lon = longitude, lat = latitude) %>%
dplyr::select(lon, lat) %>%
mutate(lon = lon-360) %>% # Change back to +-180 scale
mutate(lon = lon+0.125, lat = lat-0.125) %>% # Regrid to match OISST coords
filter(lon >= NWA_corners[1], lon <= NWA_corners[2],
lat >= NWA_corners[3], lat <= NWA_corners[4]) %>%
unique()
saveRDS(ERA5_grid, "metadata/ERA5_grid.Rda")
# Test visuals
# Choose one
# OISST_grid %>%
# GLORYS_grid %>%
ERA5_grid %>%
ggplot(aes(x = lon, y = lat)) +
geom_raster() +
coord_quickmap(xlim = NWA_corners[1:2], ylim = NWA_corners[3:4], expand = F)
# Function for finding and cleaning up points within a given region polygon
points_in_region <- function(region_in, product_grid){
region_sub <- NWA_coords %>%
filter(region == region_in)
coords_in <- product_grid %>%
mutate(in_grid = sp::point.in.polygon(point.x = product_grid[["lon"]], point.y = product_grid[["lat"]],
pol.x = region_sub[["lon"]], pol.y = region_sub[["lat"]])) %>%
filter(in_grid >= 1) %>%
mutate(region = region_in) %>%
dplyr::select(lon, lat, region)
return(coords_in)
}
# Run the function
registerDoParallel(cores = 10)
OISST_regions <- plyr::ldply(unique(NWA_coords$region), points_in_region,
.parallel = T, product_grid = OISST_grid)
saveRDS(OISST_regions, "metadata/OISST_regions.Rda")
GLORYS_regions <- plyr::ldply(unique(NWA_coords$region), points_in_region,
.parallel = T, product_grid = GLORYS_grid)
saveRDS(GLORYS_regions, "metadata/GLORYS_regions.Rda")
ERA5_regions <- plyr::ldply(unique(NWA_coords$region), points_in_region,
.parallel = T, product_grid = OISST_grid) # We only want ocean pixels
saveRDS(ERA5_regions, "metadata/ERA5_regions.Rda")
# Test visuals
# Choose one
# OISST_regions %>%
# GLORYS_regions %>%
ERA5_regions %>%
ggplot(aes(x = lon, y = lat)) +
geom_raster(aes(fill = region)) +
scale_x_continuous(breaks = seq(-80, -40, 2)) +
scale_y_continuous(breaks = seq(32, 52, 2)) +
coord_quickmap(xlim = NWA_corners[1:2], ylim = NWA_corners[3:4], expand = T)
```
```{r grid-points-visual}
OISST_regions <- readRDS("metadata/OISST_regions.Rda")
GLORYS_regions <- readRDS("metadata/GLORYS_regions.Rda")
ERA5_regions <- readRDS("metadata/ERA5_regions.Rda")
# Combine for visual
ALL_regions <- rbind(OISST_regions, GLORYS_regions, ERA5_regions) %>%
mutate(product = c(rep("OISST", nrow(OISST_regions)),
rep("GLORYS", nrow(GLORYS_regions)),
rep("ERA5", nrow(ERA5_regions))))
# Visualise to ensure success
ggplot(NWA_coords, aes(x = lon, y = lat)) +
geom_polygon(data = map_base, aes(group = group), show.legend = F) +
geom_point(data = ALL_regions, aes(colour = region), size = 0.1) +
coord_quickmap(xlim = NWA_corners[1:2], ylim = NWA_corners[3:4], expand = F) +
labs(x = NULL, y = NULL, colour = "Region") +
facet_wrap(~product) +
guides(colour = guide_legend(override.aes = list(shape = 15, size = 5)))
# ggsave(filename = "output/NWA_product_regions.pdf", height = 5, width = 18)
```
With our pixels per region determined we may now go about creating the average time series for each region from the OISST, GLORYS, and ERA5 data.
## OISST data processing
Up first for processing are the OISST data. The full brick of data within the study area will be loaded first. Then the pixels within the regions are pulled out and individual time series are made from each and saved. Finally the climatologies and anomalies for each pixel in the study area are calculated and saved. The region clims+anoms are calculated for all of the data at the same time near the end of this vignette.
```{r OISST_prep, eval=FALSE}
# The files with data in the study area
OISST_files_sub <- data.frame(files = OISST_files,
lon = c(seq(0.125, 179.875, by = 0.25),
seq(-179.875, -0.125, by = 0.25))) %>%
filter(lon >= NWA_corners[1], lon <= NWA_corners[2]) %>%
mutate(files = as.character(files))
# Load the full grid for the study area
system.time(
OISST_all <- plyr::ldply(OISST_files_sub$files, .fun = load_OISST, .parallel = TRUE)
) # 16 seconds
# Check that the coords were subsetted correctly
NWA_corners
min(OISST_all$lon); max(OISST_all$lon)
min(OISST_all$lat); max(OISST_all$lat)
# test visual
OISST_all %>%
filter(t == "1993-01-01") %>%
ggplot(aes(x = lon, y = lat)) +
geom_raster(aes(fill = temp)) +
scale_x_continuous(breaks = seq(-80, -40, 2)) +
scale_y_continuous(breaks = seq(32, 52, 2)) +
coord_quickmap(xlim = NWA_corners[1:2], ylim = NWA_corners[3:4], expand = T)
# Create the region time series and save
system.time(
OISST_all_ts <- OISST_all %>%
right_join(OISST_regions, by = c("lon", "lat")) %>%
group_by(region, t) %>%
summarise(temp = mean(temp, na.rm = T), .groups = "drop")
) # 14 seconds
saveRDS(OISST_all_ts, "data/OISST_all_ts.Rda")
# test visual
OISST_all_ts %>%
ggplot(aes(x = t, y = temp)) +
geom_line(aes(colour = region)) +
facet_wrap(~region) +
labs(x = NULL, y = "Temp. (C)")
# Calculate the clims and anoms for each pixel
registerDoParallel(cores = 25)
system.time(
OISST_all_anom <- OISST_all %>%
mutate(var = "temp") %>%
dplyr::rename(val = temp) %>%
plyr::ddply(., c("lon", "lat", "var"), calc_clim_anom, .parallel = T, point_accuracy = 2)
) # 214 seconds
saveRDS(OISST_all_anom, "data/OISST_all_anom.Rda")
# test visual
OISST_all_anom %>%
filter(t == "1998-06-18") %>%
ggplot(aes(x = lon, y = lat)) +
geom_polygon(data = map_base, aes(group = group), show.legend = F) +
geom_tile(aes(fill = anom)) +
scale_x_continuous(breaks = seq(-80, -40, 2)) +
scale_y_continuous(breaks = seq(32, 52, 2)) +
scale_fill_gradient2(low = "blue", high = "red") +
labs(x = NULL, y = NULL) +
coord_quickmap(xlim = NWA_corners[1:2], ylim = NWA_corners[3:4], expand = T)
```
## GLORYS data processing
We are using the 1/12 degree GLORYS product for the calculations of the region time series in order to better capture the sub-mesoscale processes that may be associated with the driving of MHWs. For the broad synoptic scale data given to the SOM we will be using the 1/4 degree GLORYS product as it needs to be the same resolution as the other products being used.
```{r GLORYS-prep, eval=FALSE}
# NB: This is very RAM heavy, be careful with core use
# Process and save the region time series
registerDoParallel(cores = 25)
system.time(
GLORYS_all_ts <- plyr::ldply(GLORYS_files, load_GLORYS, .parallel = T, region = T) %>%
dplyr::arrange(region, t) %>%
mutate(cur_spd = round(sqrt(u^2 + v^2), 4),
cur_dir = round((270-(atan2(v, u)*(180/pi)))%%360))
) # 202 seconds on 25 cores
saveRDS(GLORYS_all_ts, "data/GLORYS_all_ts.Rda")
# test visual
GLORYS_all_ts %>%
ggplot(aes(x = t, y = mld)) +
geom_line(aes(colour = region)) +
facet_wrap(~region) +
labs(x = NULL, y = "MLD (m)")
# Load and prep the GLORYS data for the entire study area
registerDoParallel(cores = 25)
system.time(
GLORYS_all <- plyr::ldply(GLORYS_files, load_GLORYS,
.parallel = T, .paropts = c(.inorder = FALSE))
) # 293 seconds on 25 cores
# test visual
GLORYS_all %>%
filter(t == "1993-01-01") %>%
ggplot(aes(x = lon, y = lat)) +
geom_raster(aes(fill = mld)) +
scale_x_continuous(breaks = seq(-80, -40, 2)) +
scale_y_continuous(breaks = seq(32, 52, 2)) +
coord_quickmap(xlim = NWA_corners[1:2], ylim = NWA_corners[3:4], expand = T)
# Calculates clims+anoms and save
registerDoParallel(cores = 25)
system.time(
GLORYS_all_anom <- GLORYS_all %>%
pivot_longer(cols = c(-lon, -lat, -t), names_to = "var", values_to = "val") %>%
plyr::ddply(., c("lon", "lat", "var"), calc_clim_anom, .parallel = T,
point_accuracy = 6, .paropts = c(.inorder = FALSE))
) # 732 seconds on 25 cores
saveRDS(GLORYS_all_anom, "data/GLORYS_all_anom.Rda")
# test visual
GLORYS_all_anom %>%
filter(t == "1993-06-18",
var == "mld") %>%
ggplot(aes(x = lon, y = lat)) +
geom_polygon(data = map_base, aes(group = group), show.legend = F) +
geom_tile(aes(fill = anom)) +
scale_x_continuous(breaks = seq(-80, -40, 2)) +
scale_y_continuous(breaks = seq(32, 52, 2)) +
scale_fill_gradient2(low = "blue", high = "red") +
labs(x = NULL, y = NULL) +
coord_quickmap(xlim = NWA_corners[1:2], ylim = NWA_corners[3:4], expand = T)
```
## ERA5 data processing
Note that the ERA5 data are on an hourly 1/4 degree spatiotemporal grid. This loading process constrains them to a daily 1/4 degree grid that matches the OISST data before finally converting them to a single time series per region.
```{r ERA5-prep, eval=FALSE}
# See the code/workflow script for the code used for ERA5 data prep
# There is too much code to run from an RMarkdown document as each variable must loaded individually
```
## MHWs per region
We will be using the SST values from OISST for calculating the MHWs and will use the standard Hobday definition with a base period of 1993-01-01 to 2018-12-25. We are using an uneven length year as the GLORYS data do not quite extend to the end of December. It was decided that the increased accuracy of the climatology from the 2018 year outweighed the negative consideration of having a clim period that excludes a few days of winter.
```{r MHW-calc, eval=FALSE}
# Load the data
OISST_all_ts <- readRDS("data/OISST_all_ts.Rda")
# Calculate the MHWs
OISST_region_MHW <- OISST_all_ts %>%
mutate(region = toupper(region)) %>%
group_by(region) %>%
nest() %>%
mutate(clims = map(data, ts2clm,
climatologyPeriod = c("1993-01-01", "2018-12-25")),
events = map(clims, detect_event),
cats = map(events, category, S = FALSE)) %>%
select(-data, -clims)
# Save
saveRDS(OISST_region_MHW, "data/OISST_region_MHW.Rda")
saveRDS(OISST_region_MHW, "shiny/OISST_region_MHW.Rda")
```
## MHW results
There were some minor changes between the OISST grid used for the original SOM work and this version of the project. So first we want to compare the results of both to see by how much they differ. It should be very little.
```{r}
# New and old MHW results
OISST_MHW_new <- readRDS("data/OISST_region_MHW.Rda") %>%
select(-cats) %>%
unnest(events) %>%
filter(row_number() %% 2 == 0) %>%
unnest(events)
OISST_MHW_old <- readRDS("../MHWNWA/data/OISST_region_MHW.Rda") %>%
select(-cats) %>%
unnest(events) %>%
filter(row_number() %% 2 == 0) %>%
unnest(events)
## Max count of events per region
# New
OISST_MHW_new %>%
group_by(region) %>%
summarise(count = max(event_no))
# Old
OISST_MHW_old %>%
group_by(region) %>%
summarise(count = max(event_no))
## Average metrics
# New
OISST_MHW_new %>%
dplyr::select(region, duration, intensity_mean, intensity_max, intensity_cumulative, rate_onset, rate_decline) %>%
group_by(region) %>%
summarise_all("mean", na.rm = T, .groups = "drop")
# Old
OISST_MHW_old %>%
dplyr::select(region, duration, intensity_mean, intensity_max, intensity_cumulative, rate_onset, rate_decline) %>%
group_by(region) %>%
summarise_all("mean", na.rm = T, .groups = "drop")
```
We can see from the results above that there are some minor differences between the results when using the different grids. This is regrettable as it will almost certainly have some sort of impact on the SOM results. But I think it is necessary that we press forward with results based on the OISST grid as it makes the most sense w.r.t. picking pixels that fall within the region polygons. This will also allow for better subsetting of pixels within the chosen study area.
Because the MHW results themselves are not really the focus of the paper, I will quickly go over them here. Specifically we are looking at the summary stats for MHWs per region and season, and how they may differ between those classifying groups. The stats created here are used in the results section of the manuscript and some of the musings will be used for the discussion.
```{r MHW-results}
# Annual count of MHWs - first MHW
OISST_MHW_event %>%
mutate(year = year(date_peak)) %>%
group_by(year) %>%
summarise(count = n(), .groups = "drop") %>%
data.frame()
# Differences between regions
summary_region <- OISST_MHW_event %>%
dplyr::select(region, duration, intensity_mean, intensity_max,
intensity_cumulative, rate_onset, rate_decline) %>%
mutate(region = toupper(region)) %>%
dplyr::rename(group = region) %>%
group_by(group) %>%
# mutate(count = n()) %>%
ungroup() %>%
pivot_longer(duration:rate_decline) %>%
group_by(group, name) %>%
summarise(value_mean = round(mean(value), 1),
value_sd = round(sd(value), 1), .groups = "drop") %>%
unite("value_summary", value_mean:value_sd, sep = " ± ") %>%
pivot_wider(names_from = name, values_from = value_summary) %>%
dplyr::rename(i_cum = intensity_cumulative, i_max = intensity_max,
i_mean = intensity_mean, r_decline = rate_decline, r_onset = rate_onset) %>%
dplyr::select(group, duration, i_mean, i_max, i_cum, r_onset, r_decline)
# Differences between seasons
summary_season <- OISST_MHW_event %>%
ungroup() %>%
dplyr::select(season, duration, intensity_mean, intensity_max,
intensity_cumulative, rate_onset, rate_decline) %>%
dplyr::rename(group = season) %>%
group_by(group) %>%
# mutate(count = n()) %>%
ungroup() %>%
pivot_longer(duration:rate_decline) %>%
group_by(group, name) %>%
summarise(value_mean = round(mean(value), 1),
value_sd = round(sd(value), 1), .groups = "drop") %>%
unite("value_summary", value_mean:value_sd, sep = " ± ") %>%
pivot_wider(names_from = name, values_from = value_summary) %>%
dplyr::rename(i_cum = intensity_cumulative, i_max = intensity_max,
i_mean = intensity_mean, r_decline = rate_decline, r_onset = rate_onset) %>%
dplyr::select(group, duration, i_mean, i_max, i_cum, r_onset, r_decline)
# Table showing the mean +- SD per region and season
summary_region_season <- rbind(summary_region, summary_season)
knitr::kable(summary_region_season)
```
## Comparing OISST and GLORYS MHW results
In an earlier version of this analysis we had opted to use the GLORYS SST layer, but Ke pointed out that it may be better to use the NOAA OISST data. The reasoning being that because we are not fully closing the heat budget with GLORYS, there is no particular benefit to using the SST data from that modelled ensemble product. Rather it would be better to use the remotely observed NOAA OISST product as this is a more direct measure of the surface temperature of the ocean. Then again, there is a lot of benefit to just using two products instead of three. Particularly considering that all of the marine variables used here come from the GLORYS product. To that end the GLORYS and OISST MHWs must be compared to see if they are markedly different. If not, we will use the GLORYS SST data.
```{r GLORYS-OISST-comp}
# Load the MHW calculations from the NOAA OISST data
OISST_region_MHW <- readRDS("data/OISST_region_MHW.Rda")
# Load the GLORYS MHW data
GLORYS_region_MHW <- readRDS("data/GLORYS_region_MHW.Rda")
# Extract the time series
OISST_MHW_clim <- OISST_region_MHW %>%
select(-cats) %>%
unnest(events) %>%
filter(row_number() %% 2 == 1) %>%
unnest(events) %>%
mutate(product = "OISST")
GLORYS_MHW_clim <- GLORYS_region_MHW %>%
select(-cats) %>%
unnest(events) %>%
filter(row_number() %% 2 == 1) %>%
unnest(events) %>%
mutate(product = "GLORYS",
region = toupper(region))
MHW_clim <- rbind(OISST_MHW_clim, GLORYS_MHW_clim) %>%
mutate(anom = temp-seas)
# Extract the events
OISST_MHW_event <- OISST_region_MHW %>%
select(-cats) %>%
unnest(events) %>%
filter(row_number() %% 2 == 0) %>%
unnest(events) %>%
mutate(product = "OISST")
GLORYS_MHW_event <- GLORYS_region_MHW %>%
select(-cats) %>%
unnest(events) %>%
filter(row_number() %% 2 == 0) %>%
unnest(events) %>%
mutate(product = "GLORYS",
region = toupper(region))
MHW_event <- rbind(OISST_MHW_event, GLORYS_MHW_event) %>%
mutate(month_peak = lubridate::month(date_peak, label = T),
season = case_when(month_peak %in% c("Jan", "Feb", "Mar") ~ "Winter",
month_peak %in% c("Apr", "May", "Jun") ~ "Spring",
month_peak %in% c("Jul", "Aug", "Sep") ~ "Summer",
month_peak %in% c("Oct", "Nov", "Dec") ~ "Autumn"),
season = factor(season, levels = c("Spring", "Summer", "Autumn", "Winter"))) %>%
select(-month_peak)
# Compare time series
MHW_clim_wide <- MHW_clim %>%
dplyr::select(product, region, doy, t, anom) %>%
pivot_wider(names_from = product, values_from = anom) %>%
mutate(t_diff = OISST-GLORYS)
MHW_clim_wide_monthly <- MHW_clim_wide %>%
mutate(t = round_date(t, unit = "month")) %>%
group_by(region, t) %>%
summarise(t_diff = mean(t_diff, na.rm = T), .groups = "drop")
# Plot regional anomaly comparison
ts_comp <- ggplot(data = MHW_clim_wide, aes(x = t, y = t_diff)) +
geom_line(aes(colour = region), alpha = 0.5, show.legend = F) +
geom_line(data = MHW_clim_wide_monthly, show.legend = F,
aes(colour = region)) +
geom_smooth(method = "lm", show.legend = F) +
facet_wrap(~region) +
labs(x = "Date", y = "OISST anom. - GLORYS anom.",
title = "Daily anomaly comparisons",
subtitle = paste0("Faint line shows daily differences, bold line shows monthly.",
"\nStraight blue line shows linear trend in daily differences."))
# Plot the comparison of the seasonal and threshold signals
seas_thresh_comp <- MHW_clim %>%
dplyr::select(product, region, doy, seas, thresh) %>%
unique() %>%
pivot_wider(names_from = product, values_from = c(seas, thresh)) %>%
mutate(seas_diff = seas_OISST-seas_GLORYS,
thresh_diff = thresh_OISST-thresh_GLORYS) %>%
ggplot(aes(x = doy)) +
geom_line(aes(y = seas_diff, colour = region), linetype = "solid", show.legend = F) +
geom_line(aes(y = thresh_diff, colour = region), linetype = "dashed", show.legend = F) +
facet_wrap(~region) +
labs(x = "Day-of-year (doy)", y = "OISST clim. - GLORYS clim.",
title = "Difference per day-of-year (doy)",
subtitle = paste0("Solid line shows seasonal climatology,",
"\ndashed line shows threshold."))
# Plot average doy difference histogram
doy_comp <- MHW_clim_wide %>%
group_by(region, doy) %>%
summarise(doy_diff = mean(t_diff), .groups = "drop") %>%
ggplot(aes(x = doy_diff)) +
geom_histogram(aes(fill = region), bins = 20, show.legend = F) +
facet_wrap(~region) +
labs(x = "Mean difference (OISST - GLORYS) per doy",
title = "Distribution of mean differences per doy")
# Combine
OISST_GLORYS_ts_comp <- ggarrange(ts_comp,
ggarrange(seas_thresh_comp, doy_comp, ncol = 2, nrow = 1, align = "hv", labels = c("B", "C")),
nrow = 2, labels = "A", align = "hv")
OISST_GLORYS_ts_comp
# ggsave(plot = OISST_GLORYS_ts_comp, filename = "output/OISST_GLORYS_ts_comp.png", height = 8, width = 10)
# Compare MHW results
MHW_event_comp <- MHW_event %>%
group_by(product, region) %>%
summarise(event_count = n(),
dur = mean(duration),
int_mean = mean(intensity_mean),
int_cum_mean = mean(intensity_cumulative),
int_max = max(intensity_max),
onset = mean(rate_onset),
decline = mean(rate_decline), .groups = "drop") %>%
ungroup() %>%
arrange(region, product) %>%
mutate_if(is.numeric, round, 2) #%>%
# pivot_wider(names_from = product, values_from = c(event_count:decline))
# tableGrob(rows = NULL)
event_count_table <- MHW_event_comp %>%
dplyr::select(product:event_count) %>%
pivot_wider(names_from = region, values_from = event_count) %>%
tableGrob(rows = NULL)
# Boxplot of key variables
box_comp <- MHW_event %>%
dplyr::select(product, region, duration, intensity_mean,
intensity_cumulative, intensity_max, rate_onset, rate_decline) %>%
pivot_longer(cols = duration:rate_decline) %>%
ggplot(aes(x = region, y = value, fill = region)) +
geom_boxplot(aes(colour = product), notch = TRUE) +
scale_colour_manual(values = c("black", "red")) +
facet_wrap(~name, scales = "free_y") +
labs(fill = "Region", colour = "Product", x = NULL, y = "Value for given facet",
title = "Boxplots showing range of values for MHWs in each region")
# box_comp
OISST_GLORYS_MHW_comp <- ggarrange(box_comp, event_count_table, ncol = 1, nrow = 2,
heights = c(8, 1), labels = c("A", "B"), align = "hv")
OISST_GLORYS_MHW_comp
# ggsave(plot = OISST_GLORYS_MHW_comp, filename = "output/OISST_GLORYS_MHW_comp.png", height = 8, width = 10)
# Seasons within regions
MHW_event_season_comp <- MHW_event %>%
group_by(product, region, season) %>%
summarise(event_count = n(),
dur = mean(duration),
int_mean = mean(intensity_mean),
int_cum_mean = mean(intensity_cumulative),
int_max = max(intensity_max),
onset = mean(rate_onset),
decline = mean(rate_decline), .groups = "drop") %>%
ungroup() %>%
arrange(region, season, product) %>%
mutate_if(is.numeric, round, 2)
knitr::kable(MHW_event_season_comp)
# Compare top 3 events per region
MHW_event_top <- MHW_event %>%
dplyr::select(product, everything()) %>%
group_by(product, region) %>%
dplyr::top_n(3, intensity_cumulative) %>%
ungroup() %>%
arrange(region, product) %>%
dplyr::select(product, region, event_no, date_start, date_peak, date_end, duration,
intensity_mean, intensity_cumulative, intensity_max, rate_onset, rate_decline)
knitr::kable(MHW_event_top)
```
From the figures and tables output from this comparison analysis we may see that there are some larger differences than were expected. Most importantly perhaps is the the MHWs in the OISST data are more numerous, intense, and shorter in duration. It appears that the GLORYS data assimilation methodology smooths the data more than what we see in the remotely sensed SST. In the peer-reviewed write-up this difference between OISST and GLORYS smoothness will need to be discussed. I think it still best to use the GLORYS data as the SST should match more closely to the flux terms considering they are also likely smoothed more than a different more direct sensing would report. There are arguments for and against the use of the SST from GLORYS or NOAA but looking at the SOM results with the different SST products it appears that the OISST allows for more meaningful nodes. So I'm now swapping out the GLORYS SST for NOAA and will see if the correlation results still hold up. (They did)
## Region clims + anoms per variable
The analyses to come are going to be performed on anomaly values, not the original time series. In order to calculate the anomalies we are first going to need the climatologies for each variable. We will use the Hobday definition of climatology creation and then subtract the expected climatology from the observed values. We are again using the 1993-01-01 to 2018-12-25 base period for these calculations to ensure consistency throughout the project. The daily air-sea heat flux values are first divided by the MLD on that day. The MLD value used to divide the daily variables accounts for the water density and specific heat constant: Q/(rho x Cp x hmld), where rho = 1024 and Cp ~= 4000. The Qnet term calculated this way approximates the air-sea flux term.
```{r clims, eval=FALSE}
# Load the data
GLORYS_all_ts <- readRDS("data/GLORYS_all_ts.Rda")
ERA5_all_ts <- readRDS("data/ERA5_all_ts.Rda")
OISST_all_ts <- readRDS("data/OISST_all_ts.Rda")
ALL_ts <- left_join(ERA5_all_ts, GLORYS_all_ts, by = c("region", "t")) %>%
left_join(OISST_all_ts, by = c("region", "t")) %>%
filter(t <= "2018-12-31") %>%
mutate(region = toupper(region))
# Calculate all clims and anoms
# Also give better names to the variables
# Also convert the Qx terms from seconds to days by multiplying by 86,400
ALL_ts_anom <- ALL_ts %>%
dplyr::rename(lwr = msnlwrf, swr = swr_down, lhf = mslhf,
shf = msshf, mslp = msl, sst = temp.y) %>%
dplyr::select(-wind_dir, -cur_dir, -temp.x) %>%
mutate(qnet_mld = (qnet*86400)/(mld*1024*4000),
lwr_mld = (lwr*86400)/(mld*1024*4000),
swr_mld = (swr*86400)/(mld*1024*4000),
lhf_mld = (lhf*86400)/(mld*1024*4000),
shf_mld = (shf*86400)/(mld*1024*4000),
mld_1 = 1/mld) %>%
pivot_longer(cols = c(-region, -t), names_to = "var", values_to = "val") %>%
group_by(region, var) %>%
nest() %>%
mutate(clims = map(data, ts2clm, y = val, roundClm = 10,
climatologyPeriod = c("1993-01-01", "2018-12-25"))) %>%
dplyr::select(-data) %>%
unnest(cols = clims) %>%
mutate(anom = val-seas) %>%
ungroup()
# Save
saveRDS(ALL_ts_anom, "data/ALL_ts_anom.Rda")
saveRDS(ALL_ts_anom, "shiny/ALL_ts_anom.Rda")
```
## Cumulative heat flux terms
We also need to create cumulative air-sea heat flux terms as well as a few other choice variables. This is done by taking the first day during a phase of a MHW and adding the daily values together cumulatively until the end of that phase.
The movement terms aren't very useful and may not be worth including as they don't really show advection. So rather one can say that the parts of the heating that aren't explained by anything else could be attributed to advection through the process of elimination. For the moment they are still left in here.
```{r cum-heat-flux, eval=FALSE}
# Event index used for calculations
OISST_MHW_event_index <- OISST_MHW_event %>%
ungroup() %>%
mutate(row_index = 1:n())
# Wide anomaly dataframe to be used for cumulative calculations
ALL_ts_anom_wide <- ALL_ts_anom %>%
dplyr::select(region, var, t, anom) %>%
pivot_wider(id_cols = c("region", "t"), names_from = var, values_from = anom) %>%
dplyr::select(region:tcc, mslp, qnet, p_e, mld, mld_1, qnet_mld:shf_mld)
# Calculate cumulative values based on MHW phases
ALL_ts_anom_cum <- plyr::ddply(OISST_MHW_event_index, c("row_index"), cum_phase,
df = ALL_ts_anom_wide, .parallel = T) %>%
dplyr::select(-row_index)
# Save
saveRDS(ALL_ts_anom_cum, "data/ALL_ts_anom_cum.Rda")
saveRDS(ALL_ts_anom_cum, "shiny/ALL_ts_anom_cum.Rda")
```
In the [MHWs vs. heat flux](https://robwschlegel.github.io/MHWflux/mhw-flux.html) vignette we will take the periods of time over which MHWs occurred per region and pair those up with the GLORYS and ERA5 data. This will be used to investigate which drivers are best related to the onset and decline of MHWs. In the [SOM](https://robwschlegel.github.io/MHWNWA/som.html) vignette we will look for how the mean synoptic states during MHWs can best be clustered together.
## References