CT_dynamics.Rmd

---
title: "CT dynamics"
author: "Jens Daniel Müller"
date:  "`r format(Sys.time(), '%d %B, %Y')`"
output: 
  workflowr::wflow_html:
    number_sections: true
    toc_depth: 3
    toc_float:
      collapsed: false
editor_options:
  chunk_output_type: console
---


```{r global_options, include = FALSE}
knitr::opts_chunk$set(warning=FALSE, message=FALSE)
```

```{r packages}
library(tidyverse)
library(patchwork)
library(seacarb)
library(metR)
library(scico)
# library(cmocean)
# library(broom)
# library(lubridate)
# library(tibbletime)
```

```{r ggplot_theme, include = FALSE}
theme_set(theme_bw())
```


# Sensor data

Profile data are prepared by:

- Ignoring those made on June 16 (pCO2 sensor not in operation)
- Removing HydroC Flush and Zeroing periods
- Selecting only continous downcast periods
- Gridding profiles to 1m depth intervals
- Discarding profiles with 3 or more observation missing within upper 20m
- assigning mean date_time_ID value to all profiles belonging to one cruise
- discarding "coastal" station P01, P13, P14
- Restricting profiles to upper 25m

Please note that:

- The label ID represents the start date of the cruise ("YYMMDD"), not the exact mean sampling date

## pCO~2~ profile overview

```{r read_prepare_sensor_data, fig.cap="Overview pCO2 profiles at stations (P02-P12) and cruise dates (ID). y-axis restricted to displayed range.", fig.asp = 1.5}

ts <-
 read_csv(here::here("data/_merged_data_files",
                      "BloomSail_CTD_HydroC_track_RT.csv"),
               col_types = cols(ID = col_character(),
                                pCO2_analog = col_double(),
                                pCO2 = col_double(),
                                Zero = col_character(),
                                Flush = col_character(),
                                mixing = col_character(),
                                Zero_ID = col_integer(),
                                deployment = col_integer(),
                                lon = col_double(),
                                lat = col_double(),
                                pCO2_RT = col_double()))


# Filter relevant rows and columns

ts_profiles <- ts %>% 
  filter(type == "P",
         Flush == "0",
         Zero == "0",
         !ID %in% c("180616","180820"),
         !(station %in% c("PX1", "PX2", "P14", "P13", "P01"))) %>% 
  select(date_time, ID, station, lat, lon, dep, sal, tem, pCO2_raw = pCO2, pCO2 = pCO2_RT_mean, duration)


# Assign meta information

ts_profiles <- ts_profiles %>% 
  group_by(ID, station) %>% 
  mutate(duration = as.numeric(date_time - min(date_time))) %>%
  arrange(date_time) %>% 
  ungroup()

meta <- read_csv(here::here("Data/_summarized_data_files",
                          "Tina_V_Sensor_meta.csv"),
                 col_types = cols(ID = col_character()))

meta <- meta %>% 
    filter(!ID %in% c("180616","180820"),
           !(station %in% c("PX1", "PX2", "P14", "P13", "P01")))

ts_profiles <- full_join(ts_profiles, meta)
rm(meta)


# creating descriptive variables
ts_profiles <- ts_profiles %>% 
  mutate(phase = "standby",
         phase = if_else(duration >= start & duration < down & !is.na(down) & !is.na(start),   "down", phase),
         phase = if_else(duration >= down  & duration < lift & !is.na(lift) & !is.na(down ),   "low",  phase),
         phase = if_else(duration >= lift  & duration < up   & !is.na(up  ) & !is.na(lift  ),  "mid",  phase),
         phase = if_else(duration >= up    & duration < end  & !is.na(end ) & !is.na(up   ),   "up",   phase))

ts_profiles <- ts_profiles %>% 
  select(-c(start, down, lift, up, end, comment, p_type, duration))


# select downcasst only
ts_profiles <- ts_profiles %>% 
  filter(phase == "down") %>% 
  select(-phase)



# ts_profiles_highres <- ts_profiles

# grid observation to 1m depth intervals
ts_profiles <- ts_profiles %>% 
  mutate(dep_grid = as.numeric(as.character( cut(dep, seq(0,40,1), seq(0.5,39.5,1))))) %>% 
  group_by(ID, station, dep_grid) %>%
  summarise_all("mean", na.rm = TRUE) %>% 
  ungroup() %>% 
  select(-dep, dep=dep_grid)

# subset complete profiles
profiles_in <- ts_profiles %>% 
  filter(dep < 20) %>% 
  group_by(ID, station) %>% 
  summarise(nr = n()) %>% 
  mutate(select = if_else(nr > 18 | station == "P14", "in", "out")) %>% 
  select(-nr) %>% 
  ungroup()

ts_profiles <- full_join(ts_profiles, profiles_in)
rm(profiles_in)

ts_profiles %>% 
  arrange(date_time) %>% 
  ggplot(aes(pCO2, dep, col=select))+
  geom_hline(yintercept = 25)+
  geom_point()+
  geom_path()+
  scale_y_reverse()+
  scale_x_continuous(breaks = c(0, 600), labels = c(0, 600))+
  scale_color_brewer(palette = "Set1", direction = -1)+
  coord_cartesian(xlim = c(0,600))+
  facet_grid(ID~station)

ts_profiles <- ts_profiles %>%
  filter(select == "in") %>%
  select(-select) %>% 
  filter(dep < 25)

# assign mean date_time stamp

cruise_dates <- ts_profiles %>% 
  group_by(ID) %>% 
  summarise(date_time_ID = mean(date_time)) %>% 
  ungroup()


# inner_join remove P14 observations lacking date_time_ID 
ts_profiles <- inner_join(cruise_dates, ts_profiles)

```


## Station map

```{r map, fig.cap="Location of stations sampled between the east coast of Gotland and Gotland deep."}

map <- read_csv(here::here("data/Maps","Bathymetry_Gotland_east_small.csv"))

# ggplot()+
#   geom_contour_fill(data=map, aes(x=lon, y=lat, z=-elev), na.fill = TRUE)+
#   coord_quickmap(expand = 0, xlim = c(18.7, 19.9), ylim = c(57.25,57.6))+
#   theme_bw()+
#   theme(legend.position="bottom")

ts_profiles %>% 
  group_by(station) %>% 
  summarise(lat = mean(lat),
            lon = mean(lon)) %>% 
  ungroup() %>% 
  ggplot()+
  geom_raster(data=map, aes(lon, lat, fill=-elev))+
  scale_fill_scico(palette = "oslo", na.value = "grey", 
                   name="Depth [m]", direction = -1)+
  geom_label(aes(lon, lat, label=station))+
  coord_quickmap(expand = 0, xlim = c(18.7, 19.9), ylim = c(57.25,57.6))+
  theme_bw()

rm(map)

```

## Data coverage

```{r data_coverage, fig.cap="Spatio-temporal data coverage, indicated as station visits over time. ID (color) refers to the starting date of the cruise, except for P14, which was visited twice during each cruise."}

cover <- ts_profiles %>% 
  group_by(ID, station) %>% 
  summarise(date = mean(date_time),
            date_time_ID = mean(date_time_ID)) %>% 
  ungroup()

cover %>% 
  ggplot(aes(date, station, fill=ID))+
  geom_vline(aes(xintercept = date_time_ID, col=ID))+
  geom_point(shape=21)+
  scale_color_viridis_d()+
  scale_fill_viridis_d()

rm(cover)

```

# Bottle C~T~ and A~T~

At stations P07 and P10 discrete samples for lab measurments of C~T~ and A~T~ were collected. Please note that - in contrast to the pCO~2~ profiles - samples were taken on June 16, but removed here for harmonization of results.

```{r read_bottle_CO2}

tb <-
  read_csv(here::here("Data/_summarized_data_files", "Tina_V_Bottle_CO2_lab.csv"),
           col_types = cols(ID = col_character()))

tb <- tb %>% 
  filter(station %in% c("P07", "P10")) %>% 
  select(-pH_Mosley) %>% 
  mutate(CT_AT_ratio = CT/AT)

tb <- inner_join(tb, cruise_dates)

```

## Vertical profiles

```{r plot_bottle_CO2_profiles}

tb_long <- tb %>% 
  pivot_longer(4:7, names_to = "var", values_to = "value")

tb_long %>% 
  ggplot(aes(value, dep))+
  geom_path(aes(col=ID))+
  geom_point(aes(fill=ID), shape=21)+
  scale_y_reverse()+
  scale_fill_viridis_d()+
  scale_color_viridis_d()+
  facet_grid(station~var, scales = "free_x")+
  theme(legend.position = "bottom")

```

Important notes:
- Spatio-temporal variation of AT is small, which jusitfies conversion of pCO~2~ to C~T~ based on a fixed mean AT - On July 30 we see a drop in surface salinity, associated with a rise in A~T~, clearly pointing at exchange of water masses, presumably later 

## Surface time series

```{r plot_bottle_CO2_surface_timeseries, fig.cap="Time series of bottle data. Shown are mean values of samples collected at water depths < 10m (usually collected at 0 and 5 m).", fig.asp=1.2}

tb_surface <- tb_long %>% 
  filter(dep<10) %>% 
  group_by(ID, date_time_ID, var, station) %>% 
  summarise(value = mean(value, na.rm = TRUE)) %>% 
  ungroup()

rm(tb_long)

tb_surface %>% 
  ggplot(aes(date_time_ID, value, col=station))+
  #geom_point(aes(lubridate::ymd(ID), value, col=station))+
  geom_point()+
  geom_path()+
  scale_fill_viridis_d()+
  scale_color_brewer(palette = "Set1")+
  facet_grid(var~., scales = "free_y")+
  labs(x="Mean transect date")


```

```{r plot_bottle_CT_surface_timeseries, fig.cap="CT timeseries, derived by multiplying the CT-AT-ratio with mean AT", fig.asp=0.4}

AT_mean <- tb_surface %>% 
  filter(var == "AT") %>% 
  summarise(AT = mean(value, na.rm = TRUE)) %>%
  pull()

tb_surface %>% 
  filter(var == "CT_AT_ratio") %>% 
  ggplot(aes(lubridate::ymd(ID), value*AT_mean, col=station))+
  geom_point()+
  geom_path()+
  scale_fill_viridis_d()+
  scale_color_brewer(palette = "Set1")+
  labs(x="Mean transect date", y="CT-AT-ratio * mean AT")

```

Important notes:
- C~T~ drop and temporal patterns observed in the C~T~/A~T~ time series agrees well with those found in the C~T~ time series derived from pCO~2~ measurements


## Mean alkalinity

In order to derive C~T~ from measured pCO~2~ profiles, the mean alkalinity in the upper 20 m and both stations was calculated as:

```{r bottle_AT}

AT_mean <- tb %>% 
  filter(dep <= 20) %>% 
  summarise(AT = mean(AT, na.rm = TRUE)) %>%
  pull()

AT_mean

```

Likewise, the mean salinity amounts to:

```{r bottle_sal}

sal_mean <- tb %>% 
  filter(dep <= 20) %>% 
  summarise(sal = mean(sal, na.rm = TRUE)) %>%
  pull()

sal_mean

```

```{r write_fixed_values}

bind_cols(start = min(ts$date_time), 
          end = max(ts$date_time),
          AT = AT_mean,
          sal = sal_mean) %>% 
  write_csv(here::here("Data/_summarized_data_files", "tb_fix.csv"))

```



# C~T~ profiles

## Calculation from pCO~2~

C~T~ profiles were calculated from sensor pCO~2~ and T profiles, and constant salinity and alkalinity values. Note that the impact of fixed vs. measured salinity has only a negligible impact on C~T~ profiles.

```{r CT_calculation}

ts_profiles <- ts_profiles %>% 
  drop_na()

ts_profiles <- ts_profiles %>% 
  filter(pCO2 > 0)

ts_profiles <- ts_profiles %>% 
  mutate(CT = carb(24, var1=pCO2, var2=1720*1e-6,
                   S=sal_mean, T=tem, P=dep/10, k1k2="m10", kf="dg", ks="d",
                   gas="insitu")[,16]*1e6)

rm(sal_mean, AT_mean)

ts_profiles %>% 
  write_csv(here::here("Data/_merged_data_files", "ts_profiles_CT.csv"))

```

## Mean profiles

Mean vertical profiles were calculated for each cruise day (ID).

```{r plot_CT_profiles_all, fig.cap="Mean vertical profiles per cruise day across all stations."}

ts_profiles_ID_mean <- ts_profiles %>% 
  select(-c(station,lat, lon, pCO2_raw, date_time)) %>% 
  group_by(ID, date_time_ID, dep) %>% 
  summarise_all(list(mean), na.rm=TRUE) %>% 
  ungroup()

ts_profiles_ID_sd <- ts_profiles %>% 
  select(-c(station,lat, lon, pCO2_raw, date_time)) %>% 
  group_by(ID, date_time_ID, dep) %>% 
  summarise_all(list(sd), na.rm=TRUE) %>% 
  ungroup()

ts_profiles_ID_sd_long <- ts_profiles_ID_sd %>% 
  pivot_longer(4:7, names_to = "var", values_to = "sd")
  
ts_profiles_ID_mean_long <- ts_profiles_ID_mean %>% 
  pivot_longer(4:7, names_to = "var", values_to = "value")
  
ts_profiles_ID_long <- inner_join(ts_profiles_ID_mean_long, ts_profiles_ID_sd_long)

rm(ts_profiles_ID_sd_long, ts_profiles_ID_sd, ts_profiles_ID_mean_long, ts_profiles_ID_mean)

ts_profiles_ID_long %>% 
  ggplot(aes(value, dep, col=ID))+
  geom_point()+
  geom_path()+
  scale_y_reverse()+
  scale_color_viridis_d()+
  facet_wrap(~var, scales = "free_x")
  
```

```{r plot_CT_profiles_individual, fig.cap="Mean vertical profiles per cruise day across all stations plotted indivdually. Ribbons indicate the standard deviation observed across all profiles at each depth and transect.", fig.asp=1.5}

all <- ts_profiles_ID_long %>% 
  filter(var %in% c("CT", "tem")) %>% 
  rename(group = ID)

ts_profiles_ID_long %>% 
  filter(var %in% c("CT", "tem")) %>% 
  ggplot()+
  geom_path(data=all, aes(value, dep, group=group))+
  geom_ribbon(aes(xmin = value-sd, xmax=value+sd, y=dep, fill=ID), alpha=0.5)+
  geom_path(aes(value, dep, col=ID))+
  scale_y_reverse()+
  scale_color_viridis_d()+
  scale_fill_viridis_d()+
  facet_grid(ID~var, scales = "free_x")

rm(all)

```

Important notes:

- the standard deviation of C~T~ in the upper 10m increases on June 30.

## Individual profiles

C~T~, pCO~2~, S, and T profiles were plotted individually 
[pdf here](https://github.com/jens-daniel-mueller/BloomSail/tree/master/output/Plots/CT_dynamics/ts_profiles_pCO2_tem_sal_CT.pdf){target="_blank"} 
and grouped by ID 
[pdf here](https://github.com/jens-daniel-mueller/BloomSail/tree/master/output/Plots/CT_dynamics/ts_profiles_ID_pCO2_tem_sal_CT.pdf){target="_blank"}. The later gives an idea of the differences between stations at one point in time.

```{r plot_all_discretized_profiles_individual, eval=FALSE}

pdf(file=here::here("output/Plots/CT_dynamics",
    "ts_profiles_pCO2_tem_sal_CT.pdf"), onefile = TRUE, width = 9, height = 5)

for(i_ID in unique(ts_profiles$ID)){
  for(i_station in unique(ts_profiles$station)){

    if (nrow(ts_profiles %>% filter(ID == i_ID, station == i_station)) > 0){
      
      
      # i_ID      <-      unique(ts_profiles$ID)[1]
      # i_station <- unique(ts_profiles$station)[1]

      p_pCO2 <- 
        ts_profiles %>%
        arrange(date_time) %>% 
        filter(ID == i_ID,
               station == i_station) %>%
        ggplot(aes(pCO2, dep, col="grid_RT"))+
        geom_point(data = ts_profiles_highres %>% arrange(date_time) %>% filter(ID == i_ID, station == i_station),
                   aes(pCO2_raw, dep, col="raw"))+
        geom_point(data = ts_profiles_highres %>% arrange(date_time) %>% filter(ID == i_ID, station == i_station),
                   aes(pCO2, dep, col="raw_RT"))+
        geom_point(aes(pCO2_raw, dep, col="grid"))+
        geom_point()+
        geom_path()+
        scale_y_reverse()+
        scale_color_brewer(palette = "Set1")+
        labs(y="Depth [m]", x="pCO2 [µatm]", title = str_c(i_ID," | ",i_station))+
        coord_cartesian(xlim = c(0,200), ylim = c(30,0))+
        theme_bw()+
        theme(legend.position = "left")
      
      p_tem <- 
        ts_profiles %>%
        arrange(date_time) %>% 
        filter(ID == i_ID,
               station == i_station) %>%
        ggplot(aes(tem, dep))+
        geom_point()+
        geom_path()+
        scale_y_reverse()+
        labs(y="Depth [m]", x="Tem [°C]")+
        coord_cartesian(xlim = c(14,26), ylim = c(30,0))+
        theme_bw()
      
      p_sal <- 
        ts_profiles %>%
        arrange(date_time) %>% 
        filter(ID == i_ID,
               station == i_station) %>%
        ggplot(aes(sal, dep))+
        geom_point()+
        geom_path()+
        scale_y_reverse()+
        labs(y="Depth [m]", x="Tem [°C]")+
        coord_cartesian(xlim = c(6.5,7.5), ylim = c(30,0))+
        theme_bw()
      
      p_CT <- 
        ts_profiles %>%
        arrange(date_time) %>% 
        filter(ID == i_ID,
               station == i_station) %>%
        ggplot(aes(CT, dep))+
        geom_point()+
        geom_path()+
        scale_y_reverse()+
        labs(y="Depth [m]", x="CT* [µmol/kg]")+
        coord_cartesian(xlim = c(1400,1700), ylim = c(30,0))+
        theme_bw()
      

      print(
            p_pCO2 + p_tem + p_sal + p_CT
            )
      
      rm(p_pCO2, p_sal, p_tem, p_CT)

      
    }
  }
}

dev.off()

rm(i_ID, i_station, ts_profiles_highres)



```

```{r plot_all_discretized_profiles_by_ID, eval=FALSE}

ts_profiles_long <- ts_profiles %>%
  select(-c(lat, lon, pCO2_raw)) %>% 
  pivot_longer(6:9, values_to = "value", names_to = "var")


pdf(file=here::here("output/Plots/CT_dynamics",
    "ts_profiles_ID_pCO2_tem_sal_CT.pdf"), onefile = TRUE, width = 9, height = 5)

for(i_ID in unique(ts_profiles$ID)){

  #i_ID <- unique(ts_profiles$ID)[1]
  
  sub_ts_profiles_long <- ts_profiles_long %>% 
        arrange(date_time) %>% 
        filter(ID == i_ID)
 
  print(
  
  sub_ts_profiles_long %>% 
    ggplot()+
    geom_path(data = ts_profiles_long, aes(value, dep, group=interaction(station, ID)), col="grey")+
    geom_path(aes(value, dep, col=station))+
    scale_y_reverse()+
    labs(y="Depth [m]", title = str_c("ID: ", i_ID))+
    theme_bw()+
    facet_wrap(~var, scales = "free_x")
   
  )  
  rm(sub_ts_profiles_long)
}

dev.off()

rm(i_ID, ts_profiles_long)

```



## Profiles of incremental changes

Changes of seawater vars at each depth are calculated from one cruise day to the next and divided by the number of days inbetween.

```{r incremental_changes_profiles}

ts_profiles_ID_long <- ts_profiles_ID_long %>%
    group_by(var, dep) %>%
    arrange(date_time_ID) %>%
    mutate(date_time_ID_diff = as.numeric(date_time_ID - lag(date_time_ID)),
           date_time_ID_ref  = date_time_ID - (date_time_ID - lag(date_time_ID))/2,
           value_diff = value     - lag(value, default = first(value)),
           value_diff_daily = value_diff / date_time_ID_diff,
           value_cum = cumsum(value_diff)) %>% 
  ungroup()


ts_profiles_ID_long %>% 
  arrange(dep) %>% 
  ggplot(aes(value_diff_daily, dep, col=ID))+
  geom_vline(xintercept = 0)+
  geom_point()+
  geom_path()+
  scale_y_reverse()+
  scale_color_viridis_d()+
  facet_wrap(~var, scales = "free_x")+
  labs(x="Change of value inbetween cruises per day")
  
```

## Profiles of cumulative changes

Cumulative changes of seawater vars were calculated at each depth relative to the first cruise day on July 5.

```{r cumulative_changes_profiles}

ts_profiles_ID_long %>% 
  arrange(dep) %>% 
  ggplot(aes(value_cum, dep, col=ID))+
  geom_vline(xintercept = 0)+
  geom_point()+
  geom_path()+
  scale_y_reverse()+
  scale_color_viridis_d()+
  facet_wrap(~var, scales = "free_x")+
  labs(x="Cumulative change of value")

```

Important notes:

- Salinity in the upper 10m decreases by >0.1 on June 30, and returns to average conditions already on Aug 02.

Cumulative positive and negative changes of seawater vars were calculated separately at each depth relative to the first cruise day on July 5.

```{r cumulative_directional_changes_profiles}

ts_profiles_ID_long <- ts_profiles_ID_long %>% 
  mutate(sign = if_else(value_diff < 0, "neg", "pos")) %>% 
  group_by(var, dep, sign) %>%
  arrange(date_time_ID) %>%
  mutate(value_cum_sign = cumsum(value_diff)) %>% 
  ungroup()

ts_profiles_ID_long %>%
  write_csv(here::here("Data/_merged_data_files", "ts_profiles_ID_long_cum.csv"))

ts_profiles_ID_long %>% 
  arrange(dep) %>% 
  ggplot(aes(value_cum_sign, dep, col=ID))+
  geom_vline(xintercept = 0)+
  geom_point()+
  geom_path()+
  scale_y_reverse()+
  scale_color_viridis_d()+
  scale_fill_viridis_d()+
  facet_wrap(~interaction(sign, var), scales = "free_x", ncol=4)+
  labs(x="Cumulative directional change of value")

```


# Timeseries

## Timeseries depth intervals

Mean seawater parameters were calculated for 5m depth intervals.

```{r observed_timeseries, fig.asp=1.3}

ts_profiles_ID_long_grid <- ts_profiles_ID_long %>% 
  mutate(dep = cut(dep, seq(0,30,5))) %>% 
  group_by(ID, date_time_ID, dep, var)  %>% 
  summarise_all(list(mean), na.rm=TRUE) 

ts_profiles_ID_long_grid %>% 
  ggplot(aes(date_time_ID, value, col=as.factor(dep)))+
  geom_path()+
  #geom_errorbar(aes(date_time_ID, ymax=value+sd, ymin=value-sd, col=as.factor(dep)))+
  geom_point()+
  scale_color_viridis_d(name="Depth [m]")+
  facet_wrap(~var, scales = "free_y", ncol=1)

rm(ts_profiles_ID_long_grid)
```


## Hovmoeller plots

### Absolute values

```{r plot_hovmoeller_absolut, fig.cap="Hovmoeller plots of absolute changes in C~T~ and temperature."}

bin_CT <- 20

CT_hov <- ts_profiles_ID_long %>% 
  filter(var == "CT") %>% 
  ggplot()+
  geom_contour_fill(aes(x=date_time_ID, y=dep, z=value),
                    breaks = MakeBreaks(bin_CT),
                    col="black")+
  geom_point(aes(x=date_time_ID, y=c(24.5)), size=3, shape=24, fill="white")+
  scale_fill_viridis_c(breaks = MakeBreaks(bin_CT),
                       guide = "colorstrip",
                       name="CT (µmol/kg)")+
  scale_y_reverse()+
  theme_bw()+
  labs(y="Depth (m)")+
  coord_cartesian(expand = 0)+
  theme(axis.title.x = element_blank(),
        axis.text.x = element_blank())

bin_Tem <- 2

Tem_hov <- ts_profiles_ID_long %>% 
  filter(var == "tem") %>% 
  ggplot()+
  geom_contour_fill(aes(x=date_time_ID, y=dep, z=value),
                    breaks = MakeBreaks(bin_Tem),
                    col="black")+
  geom_point(aes(x=date_time_ID, y=c(24.5)), size=3, shape=24, fill="white")+
  scale_fill_viridis_c(breaks = MakeBreaks(bin_Tem),
                       guide = "colorstrip",
                       name="Tem (°C)",
                       option = "inferno")+
  scale_y_reverse()+
  theme_bw()+
  labs(x="",y="Depth (m)")+
  coord_cartesian(expand = 0)

CT_hov / Tem_hov

rm(CT_hov, bin_CT, Tem_hov, bin_Tem)

```


### Incremental changes

```{r plot_hovmoeller_daily, fig.cap="Hovmoeller plots of daily changes in C~T~ and temperature. Note that calculated  value of change (in contrast to absolute and cumulative values) are referred to the mean dates inbetween cruise, and are not extrapolated to the full observational period."}

bin_CT <- 2.5

CT_hov <- ts_profiles_ID_long %>% 
  filter(var == "CT") %>% 
  ggplot()+
  geom_contour_fill(aes(x=date_time_ID_ref, y=dep, z=value_diff_daily),
                    breaks = MakeBreaks(bin_CT),
                    col="black")+
  geom_point(aes(x=date_time_ID, y=c(24.5)), size=3, shape=24, fill="white")+
  scale_fill_divergent(breaks = MakeBreaks(bin_CT),
                       guide = "colorstrip",
                       name="CT (µmol/kg)")+
  scale_y_reverse()+
  theme_bw()+
  labs(y="Depth (m)")+
  coord_cartesian(expand = 0)+
  theme(axis.title.x = element_blank(),
        axis.text.x = element_blank())

bin_Tem <- 0.1

Tem_hov <- ts_profiles_ID_long %>% 
  filter(var == "tem") %>% 
  ggplot()+
  geom_contour_fill(aes(x=date_time_ID_ref, y=dep, z=value_diff_daily),
                    breaks = MakeBreaks(bin_Tem),
                    col="black")+
  geom_point(aes(x=date_time_ID, y=c(24.5)), size=3, shape=24, fill="white")+
  scale_fill_divergent(breaks = MakeBreaks(bin_Tem),
                       guide = "colorstrip",
                       name="Tem (°C)")+
  scale_y_reverse()+
  theme_bw()+
  labs(x="",y="Depth (m)")+
  coord_cartesian(expand = 0)

CT_hov / Tem_hov

rm(CT_hov, bin_CT, Tem_hov, bin_Tem)

```


### Cumulative changes

```{r plot_hovmoeller_cumulative, fig.cap="Hovmoeller plots of cumulative changes in C~T~ and temperature."}

bin_CT <- 20

CT_hov <- ts_profiles_ID_long %>% 
  filter(var == "CT") %>% 
  ggplot()+
  geom_contour_fill(aes(x=date_time_ID, y=dep, z=value_cum),
                    breaks = MakeBreaks(bin_CT),
                    col="black")+
  geom_point(aes(x=date_time_ID, y=c(24.5)), size=3, shape=24, fill="white")+
  scale_fill_divergent(breaks = MakeBreaks(bin_CT),
                       guide = "colorstrip",
                       name="CT (µmol/kg)")+
  scale_y_reverse()+
  theme_bw()+
  labs(y="Depth (m)")+
  coord_cartesian(expand = 0)+
  theme(axis.title.x = element_blank(),
        axis.text.x = element_blank())

bin_Tem <- 2

Tem_hov <- ts_profiles_ID_long %>% 
  filter(var == "tem") %>% 
  ggplot()+
  geom_contour_fill(aes(x=date_time_ID, y=dep, z=value_cum),
                    breaks = MakeBreaks(bin_Tem),
                    col="black")+
  geom_point(aes(x=date_time_ID, y=c(24.5)), size=3, shape=24, fill="white")+
  scale_fill_divergent(breaks = MakeBreaks(bin_Tem),
                       guide = "colorstrip",
                       name="Tem (°C)")+
  scale_y_reverse()+
  theme_bw()+
  labs(x="",y="Depth (m)")+
  coord_cartesian(expand = 0)

CT_hov / Tem_hov

rm(CT_hov, bin_CT, Tem_hov, bin_Tem)

```



# Depth-integration C~T~

A critical first step for the determination of net community production (NCP) is the integration of observed changes in C~T~ over depth to derive iC~T~. Two approaches were tested:

- Integration of changes in CT over a predefined, fixed water depth
- Integration of changes in CT over a mixed layer depth (MLD)

Both aproaches deliver depth-integrated, incremental changes of C~T~ inbetween cruise dates. Those were summed up to derive a trajectory of cummulative iC~T~ changes.

## Fixed depths approach

Incremental and cumulative C~T~ changes inbetween cruise dates were integrated across the water colums down to predefined depth limits. This was done separately for observed positive/negative changes in C~T~, as well as for the total observed changes.

```{r integrated_iCT_fixed_depth, fig.asp=1.1}


iCT_grid_sign <- ts_profiles_ID_long %>% 
  select(ID, date_time_ID, date_time_ID_ref) %>% 
  unique() %>% 
  expand_grid(sign = c("pos", "neg"))

iCT_grid_total <- ts_profiles_ID_long %>% 
  select(ID, date_time_ID, date_time_ID_ref) %>% 
  unique() %>% 
  expand_grid(sign = c("total"))

# dep_i <- 10
rm(iCT, dep_i)

for (dep_i in seq(5,20,5)) {


iCT_sign_temp <- ts_profiles_ID_long %>% 
  filter(var == "CT", dep < dep_i) %>% 
  mutate(sign = if_else(ID == "180705" & dep == 0.5, "neg", sign)) %>% 
  group_by(ID, date_time_ID, date_time_ID_ref, sign) %>% 
  summarise(CT_i_diff = sum(value_diff)/1000) %>% 
  ungroup()

iCT_sign_temp <- iCT_sign_temp %>% 
  group_by(sign) %>%
  arrange(date_time_ID) %>% 
  mutate(CT_i_cum = cumsum(CT_i_diff)) %>% 
  ungroup()

iCT_sign_temp <- full_join(iCT_sign_temp, iCT_grid_sign) %>% 
  arrange(sign, date_time_ID) %>% 
  fill(CT_i_cum)


iCT_total_temp <- ts_profiles_ID_long %>% 
  filter(var == "CT", dep < dep_i) %>% 
  group_by(ID, date_time_ID, date_time_ID_ref) %>% 
  summarise(CT_i_diff = sum(value_diff)/1000) %>% 
  ungroup()

iCT_total_temp <- iCT_total_temp %>% 
  arrange(date_time_ID) %>% 
  mutate(CT_i_cum = cumsum(CT_i_diff)) %>% 
  ungroup() %>% 
  mutate(sign = "total")

iCT_total_temp <- full_join(iCT_total_temp, iCT_grid_total) %>% 
  arrange(sign, date_time_ID) %>% 
  fill(CT_i_cum)

iCT_temp <- bind_rows(iCT_sign_temp, iCT_total_temp) %>% 
    mutate(dep_i = dep_i)


if (exists("iCT")) {
  iCT <- bind_rows(iCT, iCT_temp)
  } else {iCT <- iCT_temp}

rm(iCT_temp, iCT_sign_temp, iCT_total_temp)

}

rm(iCT_grid_sign, iCT_grid_total)

iCT <- iCT %>% 
  mutate(dep_i = as.factor(dep_i))

iCT %>% 
  ggplot()+
  geom_point(data = cruise_dates, aes(date_time_ID, 0), shape=21)+
  geom_col(aes(date_time_ID_ref, CT_i_diff, fill=dep_i),
           position = "dodge", alpha=0.3)+
  geom_line(aes(date_time_ID, CT_i_cum, col=dep_i))+
  scale_color_viridis_d(name="Depth limit (m)")+
  scale_fill_viridis_d(name="Depth limit (m)")+
  labs(y="iCT [mol/m2]", x="")+
  facet_grid(sign~., scales = "free_y", space = "free_y")+
  theme_bw()

iCT_fixed_dep <- iCT
rm(iCT, dep_i)
# iCT %>%
#   write_csv(here::here("Data/_merged_data_files", "iCT_dep_limits.csv"))

```

## MLD approach

As an alternative to fixed depth levels, vertical integration as low as the mixed layer depth was tested.

### Density Calculation

Seawater density Rho was determined from S, T, and p according to TEOS-10.

```{r calulcate_seawater_density}

ts_profiles <- ts_profiles %>% 
  mutate(rho = swSigma(salinity = sal, temperature = tem, pressure = dep/10))

```

### Density profiles

```{r plot_hydrographical_profiles_all, fig.cap="Mean vertical profiles per cruise day across all stations."}

ts_profiles_ID_mean_hydro <- ts_profiles %>% 
  select(-c(station,lat, lon, pCO2_raw, pCO2, CT, date_time)) %>% 
  group_by(ID, date_time_ID, dep) %>% 
  summarise_all(list(mean), na.rm=TRUE) %>% 
  ungroup()

ts_profiles_ID_sd_hydro <- ts_profiles %>% 
  select(-c(station,lat, lon, pCO2_raw, pCO2, CT, date_time)) %>% 
  group_by(ID, date_time_ID, dep) %>% 
  summarise_all(list(sd), na.rm=TRUE) %>% 
  ungroup()


ts_profiles_ID_sd_hydro_long <- ts_profiles_ID_sd_hydro %>% 
  pivot_longer(4:6, names_to = "var", values_to = "sd")
  
ts_profiles_ID_mean_hydro_long <- ts_profiles_ID_mean_hydro %>% 
  pivot_longer(4:6, names_to = "var", values_to = "value")
  
ts_profiles_ID_hydro_long <- inner_join(ts_profiles_ID_mean_hydro_long, ts_profiles_ID_sd_hydro_long)

rm(ts_profiles_ID_sd_hydro_long,
   ts_profiles_ID_sd_hydro,
   ts_profiles_ID_mean_hydro_long)

ts_profiles_ID_hydro_long %>% 
  ggplot(aes(value, dep, col=ID))+
  geom_point()+
  geom_path()+
  scale_y_reverse()+
  scale_color_viridis_d()+
  facet_wrap(~var, scales = "free_x")
  
```

### MLD calculation

Mixed layer depth (MLD) was determined based on the difference between density at the surface and at depth, for a range of density criteria

```{r calculate_MLD}
# density criterion

ts_profiles_ID_mean_hydro <- expand_grid(ts_profiles_ID_mean_hydro, rho_lim = c(0.1,0.2,0.5))

MLD <- ts_profiles_ID_mean_hydro  %>% 
  arrange(dep) %>% 
  group_by(ID, date_time_ID, rho_lim) %>% 
  mutate(d_rho = rho - first(rho)) %>% 
  filter(d_rho > rho_lim) %>% 
  summarise(MLD = min(dep)) %>% 
  ungroup()

```

### Daily density profiles

```{r plot_density_profiles, fig.cap="Overview density profiles at stations (P01-P14) and cruise dates (ID). Horizontal lines indicate determined MLD", fig.asp=1.2}

ts_profiles_ID_mean_hydro <- 
  full_join(ts_profiles_ID_mean_hydro, MLD)

ts_profiles_ID_mean_hydro %>% 
  arrange(dep) %>% 
  ggplot(aes(rho, dep))+
  geom_hline(aes(yintercept = MLD, col=as.factor(rho_lim)))+
  geom_path()+
  scale_y_reverse()+
  scale_color_brewer(palette = "Set1", name= "Rho limit")+
  facet_wrap(~ID)+
  theme_bw()

```

### MLD timeseries

```{r plot_MLD_time_series, fig.asp=0.4}

MLD %>% 
  ggplot(aes(date_time_ID, MLD, col=as.factor(rho_lim)))+
  geom_point()+
  geom_path()+
  scale_color_brewer(palette = "Set1", name= "Rho limit")+
  labs(x="")

```


### iCT calculation

```{r integrated_iCT_MLD}

# iCT_grid_sign <- ts_profiles_ID_long %>% 
#   select(ID, date_time_ID, date_time_ID_ref) %>% 
#   unique() %>% 
#   expand_grid(sign = c("pos", "neg"))
# 
# iCT_grid_total <- ts_profiles_ID_long %>% 
#   select(ID, date_time_ID, date_time_ID_ref) %>% 
#   unique() %>% 
#   expand_grid(sign = c("total"))


iCT <- ts_profiles_ID_long %>% 
  filter(var == "CT")

iCT <- full_join(iCT, MLD)

iCT <- iCT %>% 
  filter(dep <= MLD)

iCT <- iCT %>% 
  group_by(ID, date_time_ID, date_time_ID_ref, rho_lim) %>% 
  summarise(CT_i_diff = sum(value_diff)/1000) %>% 
  ungroup()

iCT <- iCT %>% 
  group_by(rho_lim) %>% 
  arrange(date_time_ID) %>% 
  mutate(CT_i_cum = cumsum(CT_i_diff)) %>% 
  ungroup()

# iCT_total_temp <- full_join(iCT_total_temp, iCT_grid_total) %>% 
#   arrange(sign, date_time_ID) %>% 
#   fill(CT_i_cum)

iCT <- iCT %>% 
  mutate(rho_lim = as.factor(rho_lim))

iCT %>% 
  ggplot()+
  geom_point(data = cruise_dates, aes(date_time_ID, 0), shape=21)+
  geom_col(aes(date_time_ID_ref, CT_i_diff, fill=rho_lim),
           position = "dodge", alpha=0.3)+
  geom_line(aes(date_time_ID, CT_i_cum, col=rho_lim))+
  scale_color_viridis_d(name="Rho limit")+
  scale_fill_viridis_d(name="Rho limit")+
  labs(y="iCT [mol/m2]", x="")+
  #facet_grid(sign~., scales = "free_y", space = "free_y")+
  theme_bw()

iCT_MLD <- iCT
rm(iCT, MLD)

# iCT %>%
#   write_csv(here::here("Data/_merged_data_files", "iCT_dep_limits.csv"))

```

## Comparison of approaches

In the following, all cummulative iC~T~ trajectories are displayed. Highlighted are those obtained for the fixed depth approach with 10 m limit, and the MLD approach with a high density threshold of 0.5 kg/m3.

```{r comparing_iCT_estimates}

iCT <- full_join(iCT_fixed_dep, iCT_MLD)

iCT <- iCT %>% 
  mutate(group = paste(as.character(sign), as.character(dep_i), as.character(rho_lim)))

iCT %>% 
  arrange(date_time_ID) %>% 
  ggplot()+
  geom_hline(yintercept = 0)+
  geom_point(data = cruise_dates, aes(date_time_ID, 0), shape=21)+
  geom_line(aes(date_time_ID, CT_i_cum,
                group=group), col="grey")+
  geom_line(data = iCT_fixed_dep %>% filter(dep_i==10, sign=="total"),
            aes(date_time_ID, CT_i_cum, col="10m - total"))+
  geom_line(data = iCT_MLD %>% filter(rho_lim == 0.1),
            aes(date_time_ID, CT_i_cum, col="MLD - 0.1"))+
  scale_color_brewer(palette = "Set1", name="")+
  labs(y="iCT [mol/m2]", x="")

rm(iCT, iCT_MLD)

```

# NCP determination

In order to derive an estimate of the net community production NCP (which is equivalent to the formed organic matter that can be exported from the investigated surface layer), two steps are required:

- decision about the most appropiate iCT trajectory
- correction of quantifyable CO~2~ fluxes in and out of the investigated water volume during the period of interest, this includes:
  - Air-sea CO~2~ fluxes
  - CO~2~ fluxes due to vertical mixing
  - CO~2~ fluxes due to lateral transport of water masses (not corrected here)

# Best iC~T~ estimate

The cummulative iC~T~ trajectory determined by integration of CT to a fixed water depth of 10 m was used for NCP calculation for the following reasons:

- During the first productivity pulse that lasted until July 23:
  - >90% of iCT changes were located within the upper 10m
  - no positive iCT changes were located within the upper 10m
- Deeper (fixed() layers were affected by iCT changes
- MLD were too shallow to cover all observed negative CT changes
  

```{r iCT_depth_distribution_180723}

ts_profiles_ID_long_180723 <- ts_profiles_ID_long %>% 
  filter(ID == 180723,
         var == "CT")

p_ts_profiles_ID_long <- ts_profiles_ID_long_180723 %>% 
  arrange(dep) %>% 
  ggplot(aes(value_cum, dep, col="total"))+
  geom_vline(xintercept = 0)+
  geom_hline(yintercept = 10, col="red")+
  geom_path(aes(value_cum_sign, dep, col="directional"))+
  geom_point(aes(value_cum_sign, dep, col="directional"))+
  geom_point()+
  geom_path()+
  scale_y_reverse()+
  scale_color_brewer(palette = "Set1", name="estimate")+
  labs(x="Cumulative change of CT on July 23 (180723)")+
  theme(legend.position = "left")

ts_profiles_ID_long_180723_dep <- ts_profiles_ID_long_180723 %>% 
  select(dep, value_cum) %>% 
  filter(value_cum < 0) %>%
  arrange(dep) %>% 
  mutate(value_cum_i = sum(value_cum),
         value_cum_dep = cumsum(value_cum),
         value_cum_i_rel = value_cum_dep/value_cum_i*100)

p_ts_profiles_ID_long_rel <- ts_profiles_ID_long_180723_dep %>% 
  ggplot(aes(value_cum_i_rel, dep))+
  geom_hline(yintercept = 10, col="red")+
  geom_vline(xintercept = 90)+
  geom_point()+
  geom_line()+
  scale_y_reverse(limits = c(25,0))+
  scale_x_continuous(breaks = seq(0,100,10))+
  labs(y = "Depth (m)", x = "Relative contribution to ts_profiles_ID_long on July 23")+
  theme_bw()

p_ts_profiles_ID_long + p_ts_profiles_ID_long_rel

rm(ts_profiles_ID_long_180723,
   ts_profiles_ID_long_180723_dep,
   p_ts_profiles_ID_long,
   p_ts_profiles_ID_long_rel)

```

## Air-Sea CO~2~ flux

```{r iCT_fix_10}

iCT <- iCT_fixed_dep %>% 
  filter(dep_i == 10, sign=="total") %>% 
  select(-c(sign, dep_i))

rm(iCT_fixed_dep)

iCT %>% 
  arrange(date_time_ID) %>% 
  ggplot()+
  geom_hline(yintercept = 0)+
  geom_point(data = cruise_dates, aes(date_time_ID, 0), shape=21)+
  geom_line(aes(date_time_ID, CT_i_cum, col="iCT observed"))+
  scale_color_brewer(palette = "Set1", name="")+
  labs(y="iCT [mol/m2]", x="")

```




# Nomenclature of objectes

Should be identical for stored files and objectes in R environment

- source
  - ts: tina 5 sensor
  - tb: tina 5 bottle
  - fm: finnmaid
  - og: ostergarnsholm tower
  - mp: bathymetric map

- subsetting and modification levels
  - profiles vs surface: operation mode tina 5
  - ID: mean values for cruise (reduced temporal resolution)
  - daily: daily mean values
  - long: data stored in long format (all observations in only 2 columns "var" and "value")
  - fix: constant, unique values such as long-term means, starting dates, etc
  - iCT: includes cumulative numbers as iCT estimates
  - NCP: includes cumulative numbers as NCP estimates
  - CT_tem: data subset to analyse changes of CT with tem
  - 3d vs 2d: surface vs depth resolved GETM data
 
# Nomenclature of variables

- Descriptive variables
  - date
  - date_time
  - date_time_ID: mean date of cruise
  - date_time_ID_diff: time differences in between cruises
  - date_time_ID_ref: mean date in between cruises corresponding to observed incremental changes
  - station
  - ID: character identifying starting date of cruise as YYMMDD
  - dep: 1m depth intervals
  - lat
  - lon

- Observed variables: column names, or identifier in column "var" for long-format tables 
  - pCO2
  - sal
  - tem
  - CT
  - U10: wind speed at 10m
  - mld: mixed or mixing layer depth

- Variable name extension
  - mean
  - sd
  - min
  - max
  - int: interpolated value
  - grid: value gridded to a coarser resolution
  - diff: difference to previous value
  - diff_daily: difference to previous value divided by number of days
  - i: intergrated over depth
  - cum: cumulative value over time
  - sign: identifier for pos. or neg. differences

- p_XXX: ggplot object


# Open tasks / questions

- clean and harmonize chunk labeling (label: plot, 1 plot per chunk, etc)
- included removed stations in map and coverage plot
- Significance of changes in AT for calculated CT changes
- Harmonize selection of profiles for tau optimization and BGC interpretation (how many missing discrete depth intervals are allowed?)