toc_report_feb_2019_part7.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TOC Thematic Report - February 2019 (Part 7: summary for Task Force meeting)\n",
    "\n",
    "This notebook summarises the work done so far in preparation for the Task Force meeting in Helsinki in June.\n",
    "\n",
    "## 1. Datasets\n",
    "\n",
    "The ICPW programme comprises two main datasets: a \"core\" selection of 264 stations and a broader \"trends\" grouping of 432 stations (as of May 2019). These are organised separately in the database, but with some duplication that makes processing a little awkward/messy. These two datasets overlap, such that there are currently **556 unique stations** within the programme as a whole.\n",
    "\n",
    "Analysis for the Thematic Report initially considered data from all 556 stations, but selection criteria have been applied for checking time series length and completeness (see below). The results so far are therefore based (hopefully!) on a high quality subset of the full dataset.\n",
    "\n",
    "Where stations are present in both the \"core\" and \"trends\" datasets, I have preferentially chosen to work with data from the \"trends\" stations. This is because these stations have already been through several iterations of quality control for the TOC Trends paper drafted by Heleen, Don and John.\n",
    "\n",
    "## 2. Time period and selection criteria\n",
    "\n",
    "The main time period of interest is **1990 to 2016**. To be comnsidered in the analysis, each series for each staion and parameter must **begin before 1995**, **end after 2011** and be at least **75% complete** in the interval between these years. Completeness is assessed based on **monthly** resampled values for rivers and **seasonal** (i.e. quarterly) resampled values for lakes. The justification for this is that robust annual estimates of concentration for lakes can be obtained more easily than for rivers (i.e about 4 samples per year are required for lakes compared to ~12 for rivers).\n",
    "\n",
    "Using these criteria, 231 stations have at least some relevant data, although only a few of these have suitable records for *all* the parameters of interest.\n",
    "\n",
    "## 3. Parameters of interest\n",
    "\n",
    "The following parameters have been considered (calculated from the raw data as required):\n",
    "\n",
    " * **TOC ($mgC/l$)**. Some Focal Centres report DOC, which is assumed to be equal to TOC for the analysis here\n",
    " * **EH ($\\mu eq/l$)**. Calculated from pH as $1.10^6 * 10^{-pH}$\n",
    " * **ESO4X ($\\mu eq/l$)**. Sea-salt corrected sulphate (see below for details of the correction applied)\n",
    " * **ECaX_EMgX ($\\mu eq/l$)**. The sum of sea-salt corrected calcium and magnesium (see below for details of the correction applied)\n",
    " * **ENO3 ($\\mu eq/l$)**. Nitrate\n",
    " * **ANC ($\\mu eq/l$)**. Acid Neutralising Capacity, calculated as $(ECa+EMg+EK+ENa+ENH4) - (ECl+ESO4+ENO3)$. Note that NH4 is often not reported. In this case, it is assumed to be negligible (i.e. zero). All the other species in the calculation are assumed to be mandatory to estimate a valid ANC value\n",
    " * **ALK-E ($\\mu eq/l$)**. Alkalinity. Usually reported as \"gran alkalinity\", but there are many method variants between the various Focal Centres. These values should probably be used with caution\n",
    " \n",
    "### 3.1. Sea-salt correction\n",
    "\n",
    "Corrections (denoted by `X`) are calculated as\n",
    "\n",
    "$$EParX = EPar_{sample} - \\left[ \\left( \\frac{EPar}{ECl} \\right)_{ref} * ECl_{sample} \\right]$$\n",
    "\n",
    "Where $\\left( \\frac{EPar}{ECl} \\right)_{ref}$ is a reference ratio assumed to be constant for all locations:\n",
    "\n",
    "| Species | Molar mass | Valency | Ref. ratio |\n",
    "|:-------:|:----------:|:-------:|:----------:|\n",
    "|   SO4   |         96 |       2 |      0.103 |\n",
    "|    Cl   |         35 |       1 |          1 |\n",
    "|    Ca   |         40 |       2 |      0.037 |\n",
    "|    Mg   |         24 |       2 |      0.196 |\n",
    "|  NO3-N  |         14 |       1 |        N/A |\n",
    "\n",
    "**Note:** Using the same reference ratios everywhere produces some strange effects, especially in countires like Germany or the Czech Republic where chloride is not necessarily marine. This probably needs revisiting as the \"one-size-fits-all\" approach taken here is clearly not appropriate in some locations.\n",
    "\n",
    "## 4. Workflow overview\n",
    "\n",
    "The following notebooks describe different stages of the analysis so far:\n",
    "\n",
    " 1. **[Update \"trends\" dataset](https://github.com/JamesSample/icpw#toc-trends-paper)**. The latest work using the \"trends\" dataset is documented [here](https://github.com/JamesSample/icpw#toc-trends-paper) under the heading *\"TOC Trends paper\"*\n",
    " \n",
    " 2. **[Update \"core\" dataset](https://nbviewer.jupyter.org/github/JamesSample/icpw/blob/master/toc_report_feb_2019_part1.ipynb)**. Adding recent data for the \"core\" stations and dealing with data issues\n",
    " \n",
    " 3. **[Combining the \"core\" and \"trends\" datasets](https://nbviewer.jupyter.org/github/JamesSample/icpw/blob/master/toc_report_feb_2019_part3.ipynb)**. An overview of the unified ICPW dataset of 556 stations\n",
    " \n",
    " 4. **[Scatterplots of annual data](https://nbviewer.jupyter.org/github/JamesSample/icpw/blob/master/toc_report_feb_2019_part4_wge_plots.ipynb)**. A high-level overview of the raw data, aggregated to annual medians\n",
    " \n",
    " 5. **[Stations with high frequency monitoring](https://nbviewer.jupyter.org/github/JamesSample/icpw/blob/master/toc_report_feb_2019_part5_hi_freq.ipynb)**. Ten stations have substantially more detailed monitoring than the others (approximately 25 to 100 samples per year from 1990 to 2016). This notebook performs trend and change point analyses based on *monthly* data, using algorithms that are too \"data hungry\" to be applied elsewhere\n",
    " \n",
    " 6. **[Stations with \"standard\" monitoring](https://nbviewer.jupyter.org/github/JamesSample/icpw/blob/master/toc_report_feb_2019_part6.ipynb)**. Trend and change point analyses using *annually* aggregated data for the 231 stations with time series matching the criteria defined in section 2, above"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}