docs/source/examples/example_nbs/01_annuity_in_payment_simple/example1_annuity_simple1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d71a777b",
   "metadata": {},
   "source": [
    "# Example 1: Simple Annuity\n",
    "\n",
    "\n",
    "This example demonstrates how to get started using ``pyprotolinc``. We look at *plain vanilla annuity\n",
    "policies* already *in the payout phase*.\n",
    "\n",
    "To follow along download the repository and change into the subdirectory of the examples folder that corresponds with *example 1*.\n",
    "\n",
    "## A First Run"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "b31d5dbe",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "D:\\programming\\pyprotolinc\\examples\\01_annuity_in_payment_simple\n"
     ]
    }
   ],
   "source": [
    "# This is for Windows, otherwise use \"!pwd\"\n",
    "!echo %CD%  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dbc47c55",
   "metadata": {},
   "source": [
    "Copy the file *results_viewer_generic_template.xslx* from the parent directory to this one \n",
    "(and if using version control) rename it to *results_viewer_generic.xslx*.\n",
    "The directory is laid out as follows: <img src=\"img/directory_layout.png\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26636843",
   "metadata": {},
   "source": [
    "To start the projection run\n",
    "```\n",
    "pyprotolinc run\n",
    "```\n",
    "\n",
    "After the execution has completed a new csv file should appear in the subdirectory *results*. A convenient way to examine it is to import the file into the *results_viewer_generic.xlsx*.\n",
    "\n",
    "<img src=\"img/results.png\"/>\n",
    "\n",
    "We can also import the results programmatically:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "b5785480",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>YEAR</th>\n",
       "      <th>QUARTER</th>\n",
       "      <th>MONTH</th>\n",
       "      <th>PREMIUM</th>\n",
       "      <th>ANNUITY_PAYMENT1</th>\n",
       "      <th>ANNUITY_PAYMENT2</th>\n",
       "      <th>DEATH_PAYMENT</th>\n",
       "      <th>DI_LUMPSUM_PAYMENT</th>\n",
       "      <th>RESERVE_BOM(DIS1)</th>\n",
       "      <th>RESERVE_BOM(DEATH)</th>\n",
       "      <th>...</th>\n",
       "      <th>MV_ACTIVE_DIS1</th>\n",
       "      <th>MV_ACT_DIS2</th>\n",
       "      <th>MV_ACT_LAPSED</th>\n",
       "      <th>MV_ACT_MATURED</th>\n",
       "      <th>MV_DIS1_DEATH</th>\n",
       "      <th>MV_DIS1_DIS2</th>\n",
       "      <th>MV_DIS1_ACT</th>\n",
       "      <th>MV_DIS2_DEATH</th>\n",
       "      <th>MV_DIS2_DIS1</th>\n",
       "      <th>MV_DIS2_ACT</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2021</td>\n",
       "      <td>4</td>\n",
       "      <td>12</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2022</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-100.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>123528.208672</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000125</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2022</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-99.987500</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>123438.496499</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000125</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2022</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-99.975002</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>123348.789350</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000125</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2022</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-99.962505</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>123259.087223</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000125</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 27 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   YEAR  QUARTER  MONTH  PREMIUM  ANNUITY_PAYMENT1  ANNUITY_PAYMENT2  \\\n",
       "0  2021        4     12      0.0          0.000000               0.0   \n",
       "1  2022        1      1      0.0       -100.000000               0.0   \n",
       "2  2022        1      2      0.0        -99.987500               0.0   \n",
       "3  2022        1      3      0.0        -99.975002               0.0   \n",
       "4  2022        2      4      0.0        -99.962505               0.0   \n",
       "\n",
       "   DEATH_PAYMENT  DI_LUMPSUM_PAYMENT  RESERVE_BOM(DIS1)  RESERVE_BOM(DEATH)  \\\n",
       "0            0.0                 0.0           0.000000                 0.0   \n",
       "1            0.0                 0.0      123528.208672                 0.0   \n",
       "2            0.0                 0.0      123438.496499                 0.0   \n",
       "3            0.0                 0.0      123348.789350                 0.0   \n",
       "4            0.0                 0.0      123259.087223                 0.0   \n",
       "\n",
       "   ...  MV_ACTIVE_DIS1  MV_ACT_DIS2  MV_ACT_LAPSED  MV_ACT_MATURED  \\\n",
       "0  ...             0.0          0.0            0.0             0.0   \n",
       "1  ...             0.0          0.0            0.0             0.0   \n",
       "2  ...             0.0          0.0            0.0             0.0   \n",
       "3  ...             0.0          0.0            0.0             0.0   \n",
       "4  ...             0.0          0.0            0.0             0.0   \n",
       "\n",
       "   MV_DIS1_DEATH  MV_DIS1_DIS2  MV_DIS1_ACT  MV_DIS2_DEATH  MV_DIS2_DIS1  \\\n",
       "0       0.000000           0.0          0.0            0.0           0.0   \n",
       "1       0.000125           0.0          0.0            0.0           0.0   \n",
       "2       0.000125           0.0          0.0            0.0           0.0   \n",
       "3       0.000125           0.0          0.0            0.0           0.0   \n",
       "4       0.000125           0.0          0.0            0.0           0.0   \n",
       "\n",
       "   MV_DIS2_ACT  \n",
       "0          0.0  \n",
       "1          0.0  \n",
       "2          0.0  \n",
       "3          0.0  \n",
       "4          0.0  \n",
       "\n",
       "[5 rows x 27 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "pd.read_csv(\"results/ncf_out_generic.csv\", index_col=0).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8b3a9647",
   "metadata": {},
   "source": [
    "The interpretation of the outputs should be straight forward, the column *ANNUITY_PAYMENT1* shows the ongoing monthly annuity\n",
    "payments due and the column *RESERVE_BOM(DIS1)* shows the mathematical reserve progression calculated by *pyprotolinc*."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "edafe24b",
   "metadata": {},
   "source": [
    "## Having a Look at the Configuration"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7fd6e600",
   "metadata": {},
   "source": [
    "The subdirectory *portfolio* contains the portfolio file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "f61e228c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>DATE_PORTFOLIO</th>\n",
       "      <th>ID</th>\n",
       "      <th>DATE_OF_BIRTH</th>\n",
       "      <th>DATE_START_OF_COVER</th>\n",
       "      <th>SUM_INSURED</th>\n",
       "      <th>CURRENT_STATUS</th>\n",
       "      <th>SEX</th>\n",
       "      <th>PRODUCT</th>\n",
       "      <th>PRODUCT_PARAMETERS</th>\n",
       "      <th>SMOKERSTATUS</th>\n",
       "      <th>RESERVING_RATE</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2021-12-31</td>\n",
       "      <td>1</td>\n",
       "      <td>1942-04-23</td>\n",
       "      <td>2022-01-01</td>\n",
       "      <td>1200</td>\n",
       "      <td>DIS1</td>\n",
       "      <td>m</td>\n",
       "      <td>AnnuityInPayment</td>\n",
       "      <td>NaN</td>\n",
       "      <td>U</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  DATE_PORTFOLIO  ID DATE_OF_BIRTH DATE_START_OF_COVER  SUM_INSURED  \\\n",
       "0     2021-12-31   1    1942-04-23          2022-01-01         1200   \n",
       "\n",
       "  CURRENT_STATUS SEX           PRODUCT  PRODUCT_PARAMETERS SMOKERSTATUS  \\\n",
       "0           DIS1   m  AnnuityInPayment                 NaN            U   \n",
       "\n",
       "   RESERVING_RATE  \n",
       "0               0  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "pd.read_excel(\"portfolio/portfolio_annuity_small.xlsx\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a7baec4",
   "metadata": {},
   "source": [
    "Currently there is just one record in the portfolio file. The attributes should be clear, just some remarks:\n",
    "\n",
    "* *DATE_PORTFOLIO* is the snapshot date at which the status captured in the file was valid. It should be identical for all policies in the portfolio\n",
    "* *SUM_INSURED* holds the annual annuity amount. Note that the yearly amount divided by 12 matches the initial annuity amount due as seen in the results output.\n",
    "* *CURRENT_STATUS* is set to DIS1. This setting links back to the state model which will be discussed below. In this case DIS1 means that the policy is in the payment phase.\n",
    "* *RESERVING_RATE* is used for discounting when calculating the reserves (and is set to 0 above)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f0cd709",
   "metadata": {},
   "source": [
    "The runs are controlled by two further files, the *config.yml* and the *longevity_assumptions_simple.yml*. Let's have a look at both:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "f3741106",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "io:\n",
      "  portfolio_cache: \"portfolio/portfolio_cache\"\n",
      "  profile_out_dir: \".\"\n",
      "\n",
      "model:\n",
      "  type: \"GenericMultiState\"\n",
      "  years_to_simulate: 119\n",
      "  steps_per_month: 1\n",
      "  use_multicore: false\n",
      "\n",
      "run_type_specs:\n",
      "\n",
      "  GenericMultiState:\n",
      "    state_model: \"AnnuityRunoffStates\"\n",
      "    assumptions_spec: \"longevity_assumptions_simple.yml\"\n",
      "    outfile: \"results/ncf_out_generic.csv\"\n",
      "    portfolio_path: \"portfolio/portfolio_annuity_small.xlsx\"\n",
      "    portfolio_chunk_size: 1024\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with open('config.yml', 'r') as f:\n",
    "    print(f.read())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7636b447",
   "metadata": {},
   "source": [
    "The file *config.yml* is the main configuration file and *pyprotolinc* is looking for it by default in the current working directory while it can also be specified as a command line option as can be seen by looking at the CLI output when adding the *--help* flag:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "41dc4046",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO: Showing help with the command 'pyprotolinc run -- --help'.\n",
      "\n",
      "NAME\n",
      "    pyprotolinc run - Perform a projection run.\n",
      "\n",
      "SYNOPSIS\n",
      "    pyprotolinc run <flags>\n",
      "\n",
      "DESCRIPTION\n",
      "    Perform a projection run.\n",
      "\n",
      "FLAGS\n",
      "    --config_file=CONFIG_FILE\n",
      "        Default: 'config.yml'\n",
      "    --multi_processing_overwrite=MULTI_PROCESSING_OVERWRITE\n",
      "        Type: Optional[]\n",
      "        Default: None\n"
     ]
    }
   ],
   "source": [
    "!pyprotolinc run --help"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99277683",
   "metadata": {},
   "source": [
    "Note that among other settings made here this configuration points to the assumption specification file *longevity_assumptions_simple.yml* and to the portfolio file. Futhermore, a *state model* is specified as *AnnuityRunoffStates*. This state model is an important element for the projector and is mapped to a class in *pyprotolinc*. The docstring of this class looks like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "e06e088f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " A state model consisting of two states:\n",
      "        - DIS1 (=0) representing the annuity phase\n",
      "        - DEATH (=1)\n",
      "    \n"
     ]
    }
   ],
   "source": [
    "from pyprotolinc.models.model_annuity_runoff import AnnuityRunoffStates\n",
    "print(AnnuityRunoffStates.__doc__)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75f13d38",
   "metadata": {},
   "source": [
    "When running *pyprotolinc* the state model is used in conjunction with the product information *AnnuityInPayment* (which is extracted from the field *PRODUCT* in the portfolio file) and the assumptions which are specified in the file *longevity_assumptions.yml*:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "5b4fd1bd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# for the annuity model\n",
      "assumptions_spec:\n",
      "\n",
      "  be:\n",
      "    # annuitant's deaths\n",
      "    - [0, 1, [\"Scalar\", 0.0015]]\n",
      "\n",
      "  res:\n",
      "    # annuitant's deaths\n",
      "    - [0, 1,  [\"Scalar\", 0.0025]]\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with open('longevity_assumptions_simple.yml', 'r') as f:\n",
    "    print(f.read())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "58b03375",
   "metadata": {},
   "source": [
    "Here we are specifying the assumptions to be used for the state transitions under the *best estimate* view (be) and under the *reserving view*. In this case the projection in the best estimate view specifies assumptions for a state transition as follows:\n",
    "```\n",
    "[0, 1, [\"Scalar\", 0.0015]]\n",
    "```\n",
    "This encodes the transition from state 0 (=DIS=\"in annuity phase\") to 1(=DEATH). In this case the structure of the assumptions is trivial: use use a flat transition rate of 1.5/1000 (2.5/1000 for reserving respectively) independent on risk factors like age or gender."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}