0b - unit 07.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise: Gradient Descent\n",
    "\n",
    "Previously we identified trends in winter temperatures by fitting a linear regression model to weather data.\n",
    "\n",
    "Here, we will repeat this process focussing on the optimizer. Specifically, we will work with batch gradient descent and explore how changing the learning rate can alter its behaviour.\n",
    "\n",
    "The model we will be working with will be the same linear regression model we have used in other units. The principles we learn, however, apply to much more complex models too.\n",
    "\n",
    "## Loading data and preparing our model\n",
    "\n",
    "Let's load up our weather data from Seattle, filter to January temperatures, and make slight adjustments so that the dates are mathematically interpretable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Visual Check:\n"
     ]
    },
    {
     "output_type": "display_data",
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "hovertemplate": "year=%{x}<br>min temperature=%{y}<extra></extra>",
         "legendgroup": "",
         "marker": {
          "color": "#636efa",
          "size": 3,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "",
         "showlegend": false,
         "type": "scattergl",
         "x": [
          1948.002737850787,
          1948.0054757015744,
          1948.0082135523614,
          1948.0109514031485,
          1948.0136892539356,
          1948.0164271047229,
          1948.01916495551,
          1948.021902806297,
          1948.024640657084,
          1948.0273785078714,
          1948.0301163586585,
          1948.0328542094455,
          1948.0355920602328,
          1948.03832991102,
          1948.041067761807,
          1948.043805612594,
          1948.0465434633813,
          1948.0492813141684,
          1948.0520191649555,
          1948.0547570157426,
          1948.0574948665299,
          1948.060232717317,
          1948.062970568104,
          1948.065708418891,
          1948.0684462696784,
          1948.0711841204654,
          1948.0739219712525,
          1948.0766598220398,
          1948.0793976728269,
          1948.082135523614,
          1948.084873374401,
          1949.002737850787,
          1949.0054757015744,
          1949.0082135523614,
          1949.0109514031485,
          1949.0136892539356,
          1949.0164271047229,
          1949.01916495551,
          1949.021902806297,
          1949.024640657084,
          1949.0273785078714,
          1949.0301163586585,
          1949.0328542094455,
          1949.0355920602328,
          1949.03832991102,
          1949.041067761807,
          1949.043805612594,
          1949.0465434633813,
          1949.0492813141684,
          1949.0520191649555,
          1949.0547570157426,
          1949.0574948665299,
          1949.060232717317,
          1949.062970568104,
          1949.065708418891,
          1949.0684462696784,
          1949.0711841204654,
          1949.0739219712525,
          1949.0766598220398,
          1949.0793976728269,
          1949.082135523614,
          1949.084873374401,
          1950.002737850787,
          1950.0054757015744,
          1950.0082135523614,
          1950.0109514031485,
          1950.0136892539356,
          1950.0164271047229,
          1950.01916495551,
          1950.021902806297,
          1950.024640657084,
          1950.0273785078714,
          1950.0301163586585,
          1950.0328542094455,
          1950.0355920602328,
          1950.03832991102,
          1950.041067761807,
          1950.043805612594,
          1950.0465434633813,
          1950.0492813141684,
          1950.0520191649555,
          1950.0547570157426,
          1950.0574948665299,
          1950.060232717317,
          1950.062970568104,
          1950.065708418891,
          1950.0684462696784,
          1950.0711841204654,
          1950.0739219712525,
          1950.0766598220398,
          1950.0793976728269,
          1950.082135523614,
          1950.084873374401,
          1951.002737850787,
          1951.0054757015744,
          1951.0082135523614,
          1951.0109514031485,
          1951.0136892539356,
          1951.0164271047229,
          1951.01916495551,
          1951.021902806297,
          1951.024640657084,
          1951.0273785078714,
          1951.0301163586585,
          1951.0328542094455,
          1951.0355920602328,
          1951.03832991102,
          1951.041067761807,
          1951.043805612594,
          1951.0465434633813,
          1951.0492813141684,
          1951.0520191649555,
          1951.0547570157426,
          1951.0574948665299,
          1951.060232717317,
          1951.062970568104,
          1951.065708418891,
          1951.0684462696784,
          1951.0711841204654,
          1951.0739219712525,
          1951.0766598220398,
          1951.0793976728269,
          1951.082135523614,
          1951.084873374401,
          1952.002737850787,
          1952.0054757015744,
          1952.0082135523614,
          1952.0109514031485,
          1952.0136892539356,
          1952.0164271047229,
          1952.01916495551,
          1952.021902806297,
          1952.024640657084,
          1952.0273785078714,
          1952.0301163586585,
          1952.0328542094455,
          1952.0355920602328,
          1952.03832991102,
          1952.041067761807,
          1952.043805612594,
          1952.0465434633813,
          1952.0492813141684,
          1952.0520191649555,
          1952.0547570157426,
          1952.0574948665299,
          1952.060232717317,
          1952.062970568104,
          1952.065708418891,
          1952.0684462696784,
          1952.0711841204654,
          1952.0739219712525,
          1952.0766598220398,
          1952.0793976728269,
          1952.082135523614,
          1952.084873374401,
          1953.002737850787,
          1953.0054757015744,
          1953.0082135523614,
          1953.0109514031485,
          1953.0136892539356,
          1953.0164271047229,
          1953.01916495551,
          1953.021902806297,
          1953.024640657084,
          1953.0273785078714,
          1953.0301163586585,
          1953.0328542094455,
          1953.0355920602328,
          1953.03832991102,
          1953.041067761807,
          1953.043805612594,
          1953.0465434633813,
          1953.0492813141684,
          1953.0520191649555,
          1953.0547570157426,
          1953.0574948665299,
          1953.060232717317,
          1953.062970568104,
          1953.065708418891,
          1953.0684462696784,
          1953.0711841204654,
          1953.0739219712525,
          1953.0766598220398,
          1953.0793976728269,
          1953.082135523614,
          1953.084873374401,
          1954.002737850787,
          1954.0054757015744,
          1954.0082135523614,
          1954.0109514031485,
          1954.0136892539356,
          1954.0164271047229,
          1954.01916495551,
          1954.021902806297,
          1954.024640657084,
          1954.0273785078714,
          1954.0301163586585,
          1954.0328542094455,
          1954.0355920602328,
          1954.03832991102,
          1954.041067761807,
          1954.043805612594,
          1954.0465434633813,
          1954.0492813141684,
          1954.0520191649555,
          1954.0547570157426,
          1954.0574948665299,
          1954.060232717317,
          1954.062970568104,
          1954.065708418891,
          1954.0684462696784,
          1954.0711841204654,
          1954.0739219712525,
          1954.0766598220398,
          1954.0793976728269,
          1954.082135523614,
          1954.084873374401,
          1955.002737850787,
          1955.0054757015744,
          1955.0082135523614,
          1955.0109514031485,
          1955.0136892539356,
          1955.0164271047229,
          1955.01916495551,
          1955.021902806297,
          1955.024640657084,
          1955.0273785078714,
          1955.0301163586585,
          1955.0328542094455,
          1955.0355920602328,
          1955.03832991102,
          1955.041067761807,
          1955.043805612594,
          1955.0465434633813,
          1955.0492813141684,
          1955.0520191649555,
          1955.0547570157426,
          1955.0574948665299,
          1955.060232717317,
          1955.062970568104,
          1955.065708418891,
          1955.0684462696784,
          1955.0711841204654,
          1955.0739219712525,
          1955.0766598220398,
          1955.0793976728269,
          1955.082135523614,
          1955.084873374401,
          1956.002737850787,
          1956.0054757015744,
          1956.0082135523614,
          1956.0109514031485,
          1956.0136892539356,
          1956.0164271047229,
          1956.01916495551,
          1956.021902806297,
          1956.024640657084,
          1956.0273785078714,
          1956.0301163586585,
          1956.0328542094455,
          1956.0355920602328,
          1956.03832991102,
          1956.041067761807,
          1956.043805612594,
          1956.0465434633813,
          1956.0492813141684,
          1956.0520191649555,
          1956.0547570157426,
          1956.0574948665299,
          1956.060232717317,
          1956.062970568104,
          1956.065708418891,
          1956.0684462696784,
          1956.0711841204654,
          1956.0739219712525,
          1956.0766598220398,
          1956.0793976728269,
          1956.082135523614,
          1956.084873374401,
          1957.002737850787,
          1957.0054757015744,
          1957.0082135523614,
          1957.0109514031485,
          1957.0136892539356,
          1957.0164271047229,
          1957.01916495551,
          1957.021902806297,
          1957.024640657084,
          1957.0273785078714,
          1957.0301163586585,
          1957.0328542094455,
          1957.0355920602328,
          1957.03832991102,
          1957.041067761807,
          1957.043805612594,
          1957.0465434633813,
          1957.0492813141684,
          1957.0520191649555,
          1957.0547570157426,
          1957.0574948665299,
          1957.060232717317,
          1957.062970568104,
          1957.065708418891,
          1957.0684462696784,
          1957.0711841204654,
          1957.0739219712525,
          1957.0766598220398,
          1957.0793976728269,
          1957.082135523614,
          1957.084873374401,
          1958.002737850787,
          1958.0054757015744,
          1958.0082135523614,
          1958.0109514031485,
          1958.0136892539356,
          1958.0164271047229,
          1958.01916495551,
          1958.021902806297,
          1958.024640657084,
          1958.0273785078714,
          1958.0301163586585,
          1958.0328542094455,
          1958.0355920602328,
          1958.03832991102,
          1958.041067761807,
          1958.043805612594,
          1958.0465434633813,
          1958.0492813141684,
          1958.0520191649555,
          1958.0547570157426,
          1958.0574948665299,
          1958.060232717317,
          1958.062970568104,
          1958.065708418891,
          1958.0684462696784,
          1958.0711841204654,
          1958.0739219712525,
          1958.0766598220398,
          1958.0793976728269,
          1958.082135523614,
          1958.084873374401,
          1959.002737850787,
          1959.0054757015744,
          1959.0082135523614,
          1959.0109514031485,
          1959.0136892539356,
          1959.0164271047229,
          1959.01916495551,
          1959.021902806297,
          1959.024640657084,
          1959.0273785078714,
          1959.0301163586585,
          1959.0328542094455,
          1959.0355920602328,
          1959.03832991102,
          1959.041067761807,
          1959.043805612594,
          1959.0465434633813,
          1959.0492813141684,
          1959.0520191649555,
          1959.0547570157426,
          1959.0574948665299,
          1959.060232717317,
          1959.062970568104,
          1959.065708418891,
          1959.0684462696784,
          1959.0711841204654,
          1959.0739219712525,
          1959.0766598220398,
          1959.0793976728269,
          1959.082135523614,
          1959.084873374401,
          1960.002737850787,
          1960.0054757015744,
          1960.0082135523614,
          1960.0109514031485,
          1960.0136892539356,
          1960.0164271047229,
          1960.01916495551,
          1960.021902806297,
          1960.024640657084,
          1960.0273785078714,
          1960.0301163586585,
          1960.0328542094455,
          1960.0355920602328,
          1960.03832991102,
          1960.041067761807,
          1960.043805612594,
          1960.0465434633813,
          1960.0492813141684,
          1960.0520191649555,
          1960.0547570157426,
          1960.0574948665299,
          1960.060232717317,
          1960.062970568104,
          1960.065708418891,
          1960.0684462696784,
          1960.0711841204654,
          1960.0739219712525,
          1960.0766598220398,
          1960.0793976728269,
          1960.082135523614,
          1960.084873374401,
          1961.002737850787,
          1961.0054757015744,
          1961.0082135523614,
          1961.0109514031485,
          1961.0136892539356,
          1961.0164271047229,
          1961.01916495551,
          1961.021902806297,
          1961.024640657084,
          1961.0273785078714,
          1961.0301163586585,
          1961.0328542094455,
          1961.0355920602328,
          1961.03832991102,
          1961.041067761807,
          1961.043805612594,
          1961.0465434633813,
          1961.0492813141684,
          1961.0520191649555,
          1961.0547570157426,
          1961.0574948665299,
          1961.060232717317,
          1961.062970568104,
          1961.065708418891,
          1961.0684462696784,
          1961.0711841204654,
          1961.0739219712525,
          1961.0766598220398,
          1961.0793976728269,
          1961.082135523614,
          1961.084873374401,
          1962.002737850787,
          1962.0054757015744,
          1962.0082135523614,
          1962.0109514031485,
          1962.0136892539356,
          1962.0164271047229,
          1962.01916495551,
          1962.021902806297,
          1962.024640657084,
          1962.0273785078714,
          1962.0301163586585,
          1962.0328542094455,
          1962.0355920602328,
          1962.03832991102,
          1962.041067761807,
          1962.043805612594,
          1962.0465434633813,
          1962.0492813141684,
          1962.0520191649555,
          1962.0547570157426,
          1962.0574948665299,
          1962.060232717317,
          1962.062970568104,
          1962.065708418891,
          1962.0684462696784,
          1962.0711841204654,
          1962.0739219712525,
          1962.0766598220398,
          1962.0793976728269,
          1962.082135523614,
          1962.084873374401,
          1963.002737850787,
          1963.0054757015744,
          1963.0082135523614,
          1963.0109514031485,
          1963.0136892539356,
          1963.0164271047229,
          1963.01916495551,
          1963.021902806297,
          1963.024640657084,
          1963.0273785078714,
          1963.0301163586585,
          1963.0328542094455,
          1963.0355920602328,
          1963.03832991102,
          1963.041067761807,
          1963.043805612594,
          1963.0465434633813,
          1963.0492813141684,
          1963.0520191649555,
          1963.0547570157426,
          1963.0574948665299,
          1963.060232717317,
          1963.062970568104,
          1963.065708418891,
          1963.0684462696784,
          1963.0711841204654,
          1963.0739219712525,
          1963.0766598220398,
          1963.0793976728269,
          1963.082135523614,
          1963.084873374401,
          1964.002737850787,
          1964.0054757015744,
          1964.0082135523614,
          1964.0109514031485,
          1964.0136892539356,
          1964.0164271047229,
          1964.01916495551,
          1964.021902806297,
          1964.024640657084,
          1964.0273785078714,
          1964.0301163586585,
          1964.0328542094455,
          1964.0355920602328,
          1964.03832991102,
          1964.041067761807,
          1964.043805612594,
          1964.0465434633813,
          1964.0492813141684,
          1964.0520191649555,
          1964.0547570157426,
          1964.0574948665299,
          1964.060232717317,
          1964.062970568104,
          1964.065708418891,
          1964.0684462696784,
          1964.0711841204654,
          1964.0739219712525,
          1964.0766598220398,
          1964.0793976728269,
          1964.082135523614,
          1964.084873374401,
          1965.002737850787,
          1965.0054757015744,
          1965.0082135523614,
          1965.0109514031485,
          1965.0136892539356,
          1965.0164271047229,
          1965.01916495551,
          1965.021902806297,
          1965.024640657084,
          1965.0273785078714,
          1965.0301163586585,
          1965.0328542094455,
          1965.0355920602328,
          1965.03832991102,
          1965.041067761807,
          1965.043805612594,
          1965.0465434633813,
          1965.0492813141684,
          1965.0520191649555,
          1965.0547570157426,
          1965.0574948665299,
          1965.060232717317,
          1965.062970568104,
          1965.065708418891,
          1965.0684462696784,
          1965.0711841204654,
          1965.0739219712525,
          1965.0766598220398,
          1965.0793976728269,
          1965.082135523614,
          1965.084873374401,
          1966.002737850787,
          1966.0054757015744,
          1966.0082135523614,
          1966.0109514031485,
          1966.0136892539356,
          1966.0164271047229,
          1966.01916495551,
          1966.021902806297,
          1966.024640657084,
          1966.0273785078714,
          1966.0301163586585,
          1966.0328542094455,
          1966.0355920602328,
          1966.03832991102,
          1966.041067761807,
          1966.043805612594,
          1966.0465434633813,
          1966.0492813141684,
          1966.0520191649555,
          1966.0547570157426,
          1966.0574948665299,
          1966.060232717317,
          1966.062970568104,
          1966.065708418891,
          1966.0684462696784,
          1966.0711841204654,
          1966.0739219712525,
          1966.0766598220398,
          1966.0793976728269,
          1966.082135523614,
          1966.084873374401,
          1967.002737850787,
          1967.0054757015744,
          1967.0082135523614,
          1967.0109514031485,
          1967.0136892539356,
          1967.0164271047229,
          1967.01916495551,
          1967.021902806297,
          1967.024640657084,
          1967.0273785078714,
          1967.0301163586585,
          1967.0328542094455,
          1967.0355920602328,
          1967.03832991102,
          1967.041067761807,
          1967.043805612594,
          1967.0465434633813,
          1967.0492813141684,
          1967.0520191649555,
          1967.0547570157426,
          1967.0574948665299,
          1967.060232717317,
          1967.062970568104,
          1967.065708418891,
          1967.0684462696784,
          1967.0711841204654,
          1967.0739219712525,
          1967.0766598220398,
          1967.0793976728269,
          1967.082135523614,
          1967.084873374401,
          1968.002737850787,
          1968.0054757015744,
          1968.0082135523614,
          1968.0109514031485,
          1968.0136892539356,
          1968.0164271047229,
          1968.01916495551,
          1968.021902806297,
          1968.024640657084,
          1968.0273785078714,
          1968.0301163586585,
          1968.0328542094455,
          1968.0355920602328,
          1968.03832991102,
          1968.041067761807,
          1968.043805612594,
          1968.0465434633813,
          1968.0492813141684,
          1968.0520191649555,
          1968.0547570157426,
          1968.0574948665299,
          1968.060232717317,
          1968.062970568104,
          1968.065708418891,
          1968.0684462696784,
          1968.0711841204654,
          1968.0739219712525,
          1968.0766598220398,
          1968.0793976728269,
          1968.082135523614,
          1968.084873374401,
          1969.002737850787,
          1969.0054757015744,
          1969.0082135523614,
          1969.0109514031485,
          1969.0136892539356,
          1969.0164271047229,
          1969.01916495551,
          1969.021902806297,
          1969.024640657084,
          1969.0273785078714,
          1969.0301163586585,
          1969.0328542094455,
          1969.0355920602328,
          1969.03832991102,
          1969.041067761807,
          1969.043805612594,
          1969.0465434633813,
          1969.0492813141684,
          1969.0520191649555,
          1969.0547570157426,
          1969.0574948665299,
          1969.060232717317,
          1969.062970568104,
          1969.065708418891,
          1969.0684462696784,
          1969.0711841204654,
          1969.0739219712525,
          1969.0766598220398,
          1969.0793976728269,
          1969.082135523614,
          1969.084873374401,
          1970.002737850787,
          1970.0054757015744,
          1970.0082135523614,
          1970.0109514031485,
          1970.0136892539356,
          1970.0164271047229,
          1970.01916495551,
          1970.021902806297,
          1970.024640657084,
          1970.0273785078714,
          1970.0301163586585,
          1970.0328542094455,
          1970.0355920602328,
          1970.03832991102,
          1970.041067761807,
          1970.043805612594,
          1970.0465434633813,
          1970.0492813141684,
          1970.0520191649555,
          1970.0547570157426,
          1970.0574948665299,
          1970.060232717317,
          1970.062970568104,
          1970.065708418891,
          1970.0684462696784,
          1970.0711841204654,
          1970.0739219712525,
          1970.0766598220398,
          1970.0793976728269,
          1970.082135523614,
          1970.084873374401,
          1971.002737850787,
          1971.0054757015744,
          1971.0082135523614,
          1971.0109514031485,
          1971.0136892539356,
          1971.0164271047229,
          1971.01916495551,
          1971.021902806297,
          1971.024640657084,
          1971.0273785078714,
          1971.0301163586585,
          1971.0328542094455,
          1971.0355920602328,
          1971.03832991102,
          1971.041067761807,
          1971.043805612594,
          1971.0465434633813,
          1971.0492813141684,
          1971.0520191649555,
          1971.0547570157426,
          1971.0574948665299,
          1971.060232717317,
          1971.062970568104,
          1971.065708418891,
          1971.0684462696784,
          1971.0711841204654,
          1971.0739219712525,
          1971.0766598220398,
          1971.0793976728269,
          1971.082135523614,
          1971.084873374401,
          1972.002737850787,
          1972.0054757015744,
          1972.0082135523614,
          1972.0109514031485,
          1972.0136892539356,
          1972.0164271047229,
          1972.01916495551,
          1972.021902806297,
          1972.024640657084,
          1972.0273785078714,
          1972.0301163586585,
          1972.0328542094455,
          1972.0355920602328,
          1972.03832991102,
          1972.041067761807,
          1972.043805612594,
          1972.0465434633813,
          1972.0492813141684,
          1972.0520191649555,
          1972.0547570157426,
          1972.0574948665299,
          1972.060232717317,
          1972.062970568104,
          1972.065708418891,
          1972.0684462696784,
          1972.0711841204654,
          1972.0739219712525,
          1972.0766598220398,
          1972.0793976728269,
          1972.082135523614,
          1972.084873374401,
          1973.002737850787,
          1973.0054757015744,
          1973.0082135523614,
          1973.0109514031485,
          1973.0136892539356,
          1973.0164271047229,
          1973.01916495551,
          1973.021902806297,
          1973.024640657084,
          1973.0273785078714,
          1973.0301163586585,
          1973.0328542094455,
          1973.0355920602328,
          1973.03832991102,
          1973.041067761807,
          1973.043805612594,
          1973.0465434633813,
          1973.0492813141684,
          1973.0520191649555,
          1973.0547570157426,
          1973.0574948665299,
          1973.060232717317,
          1973.062970568104,
          1973.065708418891,
          1973.0684462696784,
          1973.0711841204654,
          1973.0739219712525,
          1973.0766598220398,
          1973.0793976728269,
          1973.082135523614,
          1973.084873374401,
          1974.002737850787,
          1974.0054757015744,
          1974.0082135523614,
          1974.0109514031485,
          1974.0136892539356,
          1974.0164271047229,
          1974.01916495551,
          1974.021902806297,
          1974.024640657084,
          1974.0273785078714,
          1974.0301163586585,
          1974.0328542094455,
          1974.0355920602328,
          1974.03832991102,
          1974.041067761807,
          1974.043805612594,
          1974.0465434633813,
          1974.0492813141684,
          1974.0520191649555,
          1974.0547570157426,
          1974.0574948665299,
          1974.060232717317,
          1974.062970568104,
          1974.065708418891,
          1974.0684462696784,
          1974.0711841204654,
          1974.0739219712525,
          1974.0766598220398,
          1974.0793976728269,
          1974.082135523614,
          1974.084873374401,
          1975.002737850787,
          1975.0054757015744,
          1975.0082135523614,
          1975.0109514031485,
          1975.0136892539356,
          1975.0164271047229,
          1975.01916495551,
          1975.021902806297,
          1975.024640657084,
          1975.0273785078714,
          1975.0301163586585,
          1975.0328542094455,
          1975.0355920602328,
          1975.03832991102,
          1975.041067761807,
          1975.043805612594,
          1975.0465434633813,
          1975.0492813141684,
          1975.0520191649555,
          1975.0547570157426,
          1975.0574948665299,
          1975.060232717317,
          1975.062970568104,
          1975.065708418891,
          1975.0684462696784,
          1975.0711841204654,
          1975.0739219712525,
          1975.0766598220398,
          1975.0793976728269,
          1975.082135523614,
          1975.084873374401,
          1976.002737850787,
          1976.0054757015744,
          1976.0082135523614,
          1976.0109514031485,
          1976.0136892539356,
          1976.0164271047229,
          1976.01916495551,
          1976.021902806297,
          1976.024640657084,
          1976.0273785078714,
          1976.0301163586585,
          1976.0328542094455,
          1976.0355920602328,
          1976.03832991102,
          1976.041067761807,
          1976.043805612594,
          1976.0465434633813,
          1976.0492813141684,
          1976.0520191649555,
          1976.0547570157426,
          1976.0574948665299,
          1976.060232717317,
          1976.062970568104,
          1976.065708418891,
          1976.0684462696784,
          1976.0711841204654,
          1976.0739219712525,
          1976.0766598220398,
          1976.0793976728269,
          1976.082135523614,
          1976.084873374401,
          1977.002737850787,
          1977.0054757015744,
          1977.0082135523614,
          1977.0109514031485,
          1977.0136892539356,
          1977.0164271047229,
          1977.01916495551,
          1977.021902806297,
          1977.024640657084,
          1977.0273785078714,
          1977.0301163586585,
          1977.0328542094455,
          1977.0355920602328,
          1977.03832991102,
          1977.041067761807,
          1977.043805612594,
          1977.0465434633813,
          1977.0492813141684,
          1977.0520191649555,
          1977.0547570157426,
          1977.0574948665299,
          1977.060232717317,
          1977.062970568104,
          1977.065708418891,
          1977.0684462696784,
          1977.0711841204654,
          1977.0739219712525,
          1977.0766598220398,
          1977.0793976728269,
          1977.082135523614,
          1977.084873374401,
          1978.002737850787,
          1978.0054757015744,
          1978.0082135523614,
          1978.0109514031485,
          1978.0136892539356,
          1978.0164271047229,
          1978.01916495551,
          1978.021902806297,
          1978.024640657084,
          1978.0273785078714,
          1978.0301163586585,
          1978.0328542094455,
          1978.0355920602328,
          1978.03832991102,
          1978.041067761807,
          1978.043805612594,
          1978.0465434633813,
          1978.0492813141684,
          1978.0520191649555,
          1978.0547570157426,
          1978.0574948665299,
          1978.060232717317,
          1978.062970568104,
          1978.065708418891,
          1978.0684462696784,
          1978.0711841204654,
          1978.0739219712525,
          1978.0766598220398,
          1978.0793976728269,
          1978.082135523614,
          1978.084873374401,
          1979.002737850787,
          1979.0054757015744,
          1979.0082135523614,
          1979.0109514031485,
          1979.0136892539356,
          1979.0164271047229,
          1979.01916495551,
          1979.021902806297,
          1979.024640657084,
          1979.0273785078714,
          1979.0301163586585,
          1979.0328542094455,
          1979.0355920602328,
          1979.03832991102,
          1979.041067761807,
          1979.043805612594,
          1979.0465434633813,
          1979.0492813141684,
          1979.0520191649555,
          1979.0547570157426,
          1979.0574948665299,
          1979.060232717317,
          1979.062970568104,
          1979.065708418891,
          1979.0684462696784,
          1979.0711841204654,
          1979.0739219712525,
          1979.0766598220398,
          1979.0793976728269,
          1979.082135523614,
          1979.084873374401,
          1980.002737850787,
          1980.0054757015744,
          1980.0082135523614,
          1980.0109514031485,
          1980.0136892539356,
          1980.0164271047229,
          1980.01916495551,
          1980.021902806297,
          1980.024640657084,
          1980.0273785078714,
          1980.0301163586585,
          1980.0328542094455,
          1980.0355920602328,
          1980.03832991102,
          1980.041067761807,
          1980.043805612594,
          1980.0465434633813,
          1980.0492813141684,
          1980.0520191649555,
          1980.0547570157426,
          1980.0574948665299,
          1980.060232717317,
          1980.062970568104,
          1980.065708418891,
          1980.0684462696784,
          1980.0711841204654,
          1980.0739219712525,
          1980.0766598220398,
          1980.0793976728269,
          1980.082135523614,
          1980.084873374401,
          1981.002737850787,
          1981.0054757015744,
          1981.0082135523614,
          1981.0109514031485,
          1981.0136892539356,
          1981.0164271047229,
          1981.01916495551,
          1981.021902806297,
          1981.024640657084,
          1981.0273785078714,
          1981.0301163586585,
          1981.0328542094455,
          1981.0355920602328,
          1981.03832991102,
          1981.041067761807,
          1981.043805612594,
          1981.0465434633813,
          1981.0492813141684,
          1981.0520191649555,
          1981.0547570157426,
          1981.0574948665299,
          1981.060232717317,
          1981.062970568104,
          1981.065708418891,
          1981.0684462696784,
          1981.0711841204654,
          1981.0739219712525,
          1981.0766598220398,
          1981.0793976728269,
          1981.082135523614,
          1981.084873374401,
          1982.002737850787,
          1982.0054757015744,
          1982.0082135523614,
          1982.0109514031485,
          1982.0136892539356,
          1982.0164271047229,
          1982.01916495551,
          1982.021902806297,
          1982.024640657084,
          1982.0273785078714,
          1982.0301163586585,
          1982.0328542094455,
          1982.0355920602328,
          1982.03832991102,
          1982.041067761807,
          1982.043805612594,
          1982.0465434633813,
          1982.0492813141684,
          1982.0520191649555,
          1982.0547570157426,
          1982.0574948665299,
          1982.060232717317,
          1982.062970568104,
          1982.065708418891,
          1982.0684462696784,
          1982.0711841204654,
          1982.0739219712525,
          1982.0766598220398,
          1982.0793976728269,
          1982.082135523614,
          1982.084873374401,
          1983.002737850787,
          1983.0054757015744,
          1983.0082135523614,
          1983.0109514031485,
          1983.0136892539356,
          1983.0164271047229,
          1983.01916495551,
          1983.021902806297,
          1983.024640657084,
          1983.0273785078714,
          1983.0301163586585,
          1983.0328542094455,
          1983.0355920602328,
          1983.03832991102,
          1983.041067761807,
          1983.043805612594,
          1983.0465434633813,
          1983.0492813141684,
          1983.0520191649555,
          1983.0547570157426,
          1983.0574948665299,
          1983.060232717317,
          1983.062970568104,
          1983.065708418891,
          1983.0684462696784,
          1983.0711841204654,
          1983.0739219712525,
          1983.0766598220398,
          1983.0793976728269,
          1983.082135523614,
          1983.084873374401,
          1984.002737850787,
          1984.0054757015744,
          1984.0082135523614,
          1984.0109514031485,
          1984.0136892539356,
          1984.0164271047229,
          1984.01916495551,
          1984.021902806297,
          1984.024640657084,
          1984.0273785078714,
          1984.0301163586585,
          1984.0328542094455,
          1984.0355920602328,
          1984.03832991102,
          1984.041067761807,
          1984.043805612594,
          1984.0465434633813,
          1984.0492813141684,
          1984.0520191649555,
          1984.0547570157426,
          1984.0574948665299,
          1984.060232717317,
          1984.062970568104,
          1984.065708418891,
          1984.0684462696784,
          1984.0711841204654,
          1984.0739219712525,
          1984.0766598220398,
          1984.0793976728269,
          1984.082135523614,
          1984.084873374401,
          1985.002737850787,
          1985.0054757015744,
          1985.0082135523614,
          1985.0109514031485,
          1985.0136892539356,
          1985.0164271047229,
          1985.01916495551,
          1985.021902806297,
          1985.024640657084,
          1985.0273785078714,
          1985.0301163586585,
          1985.0328542094455,
          1985.0355920602328,
          1985.03832991102,
          1985.041067761807,
          1985.043805612594,
          1985.0465434633813,
          1985.0492813141684,
          1985.0520191649555,
          1985.0547570157426,
          1985.0574948665299,
          1985.060232717317,
          1985.062970568104,
          1985.065708418891,
          1985.0684462696784,
          1985.0711841204654,
          1985.0739219712525,
          1985.0766598220398,
          1985.0793976728269,
          1985.082135523614,
          1985.084873374401,
          1986.002737850787,
          1986.0054757015744,
          1986.0082135523614,
          1986.0109514031485,
          1986.0136892539356,
          1986.0164271047229,
          1986.01916495551,
          1986.021902806297,
          1986.024640657084,
          1986.0273785078714,
          1986.0301163586585,
          1986.0328542094455,
          1986.0355920602328,
          1986.03832991102,
          1986.041067761807,
          1986.043805612594,
          1986.0465434633813,
          1986.0492813141684,
          1986.0520191649555,
          1986.0547570157426,
          1986.0574948665299,
          1986.060232717317,
          1986.062970568104,
          1986.065708418891,
          1986.0684462696784,
          1986.0711841204654,
          1986.0739219712525,
          1986.0766598220398,
          1986.0793976728269,
          1986.082135523614,
          1986.084873374401,
          1987.002737850787,
          1987.0054757015744,
          1987.0082135523614,
          1987.0109514031485,
          1987.0136892539356,
          1987.0164271047229,
          1987.01916495551,
          1987.021902806297,
          1987.024640657084,
          1987.0273785078714,
          1987.0301163586585,
          1987.0328542094455,
          1987.0355920602328,
          1987.03832991102,
          1987.041067761807,
          1987.043805612594,
          1987.0465434633813,
          1987.0492813141684,
          1987.0520191649555,
          1987.0547570157426,
          1987.0574948665299,
          1987.060232717317,
          1987.062970568104,
          1987.065708418891,
          1987.0684462696784,
          1987.0711841204654,
          1987.0739219712525,
          1987.0766598220398,
          1987.0793976728269,
          1987.082135523614,
          1987.084873374401,
          1988.002737850787,
          1988.0054757015744,
          1988.0082135523614,
          1988.0109514031485,
          1988.0136892539356,
          1988.0164271047229,
          1988.01916495551,
          1988.021902806297,
          1988.024640657084,
          1988.0273785078714,
          1988.0301163586585,
          1988.0328542094455,
          1988.0355920602328,
          1988.03832991102,
          1988.041067761807,
          1988.043805612594,
          1988.0465434633813,
          1988.0492813141684,
          1988.0520191649555,
          1988.0547570157426,
          1988.0574948665299,
          1988.060232717317,
          1988.062970568104,
          1988.065708418891,
          1988.0684462696784,
          1988.0711841204654,
          1988.0739219712525,
          1988.0766598220398,
          1988.0793976728269,
          1988.082135523614,
          1988.084873374401,
          1989.002737850787,
          1989.0054757015744,
          1989.0082135523614,
          1989.0109514031485,
          1989.0136892539356,
          1989.0164271047229,
          1989.01916495551,
          1989.021902806297,
          1989.024640657084,
          1989.0273785078714,
          1989.0301163586585,
          1989.0328542094455,
          1989.0355920602328,
          1989.03832991102,
          1989.041067761807,
          1989.043805612594,
          1989.0465434633813,
          1989.0492813141684,
          1989.0520191649555,
          1989.0547570157426,
          1989.0574948665299,
          1989.060232717317,
          1989.062970568104,
          1989.065708418891,
          1989.0684462696784,
          1989.0711841204654,
          1989.0739219712525,
          1989.0766598220398,
          1989.0793976728269,
          1989.082135523614,
          1989.084873374401,
          1990.002737850787,
          1990.0054757015744,
          1990.0082135523614,
          1990.0109514031485,
          1990.0136892539356,
          1990.0164271047229,
          1990.01916495551,
          1990.021902806297,
          1990.024640657084,
          1990.0273785078714,
          1990.0301163586585,
          1990.0328542094455,
          1990.0355920602328,
          1990.03832991102,
          1990.041067761807,
          1990.043805612594,
          1990.0465434633813,
          1990.0492813141684,
          1990.0520191649555,
          1990.0547570157426,
          1990.0574948665299,
          1990.060232717317,
          1990.062970568104,
          1990.065708418891,
          1990.0684462696784,
          1990.0711841204654,
          1990.0739219712525,
          1990.0766598220398,
          1990.0793976728269,
          1990.082135523614,
          1990.084873374401,
          1991.002737850787,
          1991.0054757015744,
          1991.0082135523614,
          1991.0109514031485,
          1991.0136892539356,
          1991.0164271047229,
          1991.01916495551,
          1991.021902806297,
          1991.024640657084,
          1991.0273785078714,
          1991.0301163586585,
          1991.0328542094455,
          1991.0355920602328,
          1991.03832991102,
          1991.041067761807,
          1991.043805612594,
          1991.0465434633813,
          1991.0492813141684,
          1991.0520191649555,
          1991.0547570157426,
          1991.0574948665299,
          1991.060232717317,
          1991.062970568104,
          1991.065708418891,
          1991.0684462696784,
          1991.0711841204654,
          1991.0739219712525,
          1991.0766598220398,
          1991.0793976728269,
          1991.082135523614,
          1991.084873374401,
          1992.002737850787,
          1992.0054757015744,
          1992.0082135523614,
          1992.0109514031485,
          1992.0136892539356,
          1992.0164271047229,
          1992.01916495551,
          1992.021902806297,
          1992.024640657084,
          1992.0273785078714,
          1992.0301163586585,
          1992.0328542094455,
          1992.0355920602328,
          1992.03832991102,
          1992.041067761807,
          1992.043805612594,
          1992.0465434633813,
          1992.0492813141684,
          1992.0520191649555,
          1992.0547570157426,
          1992.0574948665299,
          1992.060232717317,
          1992.062970568104,
          1992.065708418891,
          1992.0684462696784,
          1992.0711841204654,
          1992.0739219712525,
          1992.0766598220398,
          1992.0793976728269,
          1992.082135523614,
          1992.084873374401,
          1993.002737850787,
          1993.0054757015744,
          1993.0082135523614,
          1993.0109514031485,
          1993.0136892539356,
          1993.0164271047229,
          1993.01916495551,
          1993.021902806297,
          1993.024640657084,
          1993.0273785078714,
          1993.0301163586585,
          1993.0328542094455,
          1993.0355920602328,
          1993.03832991102,
          1993.041067761807,
          1993.043805612594,
          1993.0465434633813,
          1993.0492813141684,
          1993.0520191649555,
          1993.0547570157426,
          1993.0574948665299,
          1993.060232717317,
          1993.062970568104,
          1993.065708418891,
          1993.0684462696784,
          1993.0711841204654,
          1993.0739219712525,
          1993.0766598220398,
          1993.0793976728269,
          1993.082135523614,
          1993.084873374401,
          1994.002737850787,
          1994.0054757015744,
          1994.0082135523614,
          1994.0109514031485,
          1994.0136892539356,
          1994.0164271047229,
          1994.01916495551,
          1994.021902806297,
          1994.024640657084,
          1994.0273785078714,
          1994.0301163586585,
          1994.0328542094455,
          1994.0355920602328,
          1994.03832991102,
          1994.041067761807,
          1994.043805612594,
          1994.0465434633813,
          1994.0492813141684,
          1994.0520191649555,
          1994.0547570157426,
          1994.0574948665299,
          1994.060232717317,
          1994.062970568104,
          1994.065708418891,
          1994.0684462696784,
          1994.0711841204654,
          1994.0739219712525,
          1994.0766598220398,
          1994.0793976728269,
          1994.082135523614,
          1994.084873374401,
          1995.002737850787,
          1995.0054757015744,
          1995.0082135523614,
          1995.0109514031485,
          1995.0136892539356,
          1995.0164271047229,
          1995.01916495551,
          1995.021902806297,
          1995.024640657084,
          1995.0273785078714,
          1995.0301163586585,
          1995.0328542094455,
          1995.0355920602328,
          1995.03832991102,
          1995.041067761807,
          1995.043805612594,
          1995.0465434633813,
          1995.0492813141684,
          1995.0520191649555,
          1995.0547570157426,
          1995.0574948665299,
          1995.060232717317,
          1995.062970568104,
          1995.065708418891,
          1995.0684462696784,
          1995.0711841204654,
          1995.0739219712525,
          1995.0766598220398,
          1995.0793976728269,
          1995.082135523614,
          1995.084873374401,
          1996.002737850787,
          1996.0054757015744,
          1996.0082135523614,
          1996.0109514031485,
          1996.0136892539356,
          1996.0164271047229,
          1996.01916495551,
          1996.021902806297,
          1996.024640657084,
          1996.0273785078714,
          1996.0301163586585,
          1996.0328542094455,
          1996.0355920602328,
          1996.03832991102,
          1996.041067761807,
          1996.043805612594,
          1996.0465434633813,
          1996.0492813141684,
          1996.0520191649555,
          1996.0547570157426,
          1996.0574948665299,
          1996.060232717317,
          1996.062970568104,
          1996.065708418891,
          1996.0684462696784,
          1996.0711841204654,
          1996.0739219712525,
          1996.0766598220398,
          1996.0793976728269,
          1996.082135523614,
          1996.084873374401,
          1997.002737850787,
          1997.0054757015744,
          1997.0082135523614,
          1997.0109514031485,
          1997.0136892539356,
          1997.0164271047229,
          1997.01916495551,
          1997.021902806297,
          1997.024640657084,
          1997.0273785078714,
          1997.0301163586585,
          1997.0328542094455,
          1997.0355920602328,
          1997.03832991102,
          1997.041067761807,
          1997.043805612594,
          1997.0465434633813,
          1997.0492813141684,
          1997.0520191649555,
          1997.0547570157426,
          1997.0574948665299,
          1997.060232717317,
          1997.062970568104,
          1997.065708418891,
          1997.0684462696784,
          1997.0711841204654,
          1997.0739219712525,
          1997.0766598220398,
          1997.0793976728269,
          1997.082135523614,
          1997.084873374401,
          1998.002737850787,
          1998.0054757015744,
          1998.0082135523614,
          1998.0109514031485,
          1998.0136892539356,
          1998.0164271047229,
          1998.01916495551,
          1998.021902806297,
          1998.024640657084,
          1998.0273785078714,
          1998.0301163586585,
          1998.0328542094455,
          1998.0355920602328,
          1998.03832991102,
          1998.041067761807,
          1998.043805612594,
          1998.0465434633813,
          1998.0492813141684,
          1998.0520191649555,
          1998.0547570157426,
          1998.0574948665299,
          1998.060232717317,
          1998.062970568104,
          1998.065708418891,
          1998.0684462696784,
          1998.0711841204654,
          1998.0739219712525,
          1998.0766598220398,
          1998.0793976728269,
          1998.082135523614,
          1998.084873374401,
          1999.002737850787,
          1999.0054757015744,
          1999.0082135523614,
          1999.0109514031485,
          1999.0136892539356,
          1999.0164271047229,
          1999.01916495551,
          1999.021902806297,
          1999.024640657084,
          1999.0273785078714,
          1999.0301163586585,
          1999.0328542094455,
          1999.0355920602328,
          1999.03832991102,
          1999.041067761807,
          1999.043805612594,
          1999.0465434633813,
          1999.0492813141684,
          1999.0520191649555,
          1999.0547570157426,
          1999.0574948665299,
          1999.060232717317,
          1999.062970568104,
          1999.065708418891,
          1999.0684462696784,
          1999.0711841204654,
          1999.0739219712525,
          1999.0766598220398,
          1999.0793976728269,
          1999.082135523614,
          1999.084873374401,
          2000.002737850787,
          2000.0054757015744,
          2000.0082135523614,
          2000.0109514031485,
          2000.0136892539356,
          2000.0164271047229,
          2000.01916495551,
          2000.021902806297,
          2000.024640657084,
          2000.0273785078714,
          2000.0301163586585,
          2000.0328542094455,
          2000.0355920602328,
          2000.03832991102,
          2000.041067761807,
          2000.043805612594,
          2000.0465434633813,
          2000.0492813141684,
          2000.0520191649555,
          2000.0547570157426,
          2000.0574948665299,
          2000.060232717317,
          2000.062970568104,
          2000.065708418891,
          2000.0684462696784,
          2000.0711841204654,
          2000.0739219712525,
          2000.0766598220398,
          2000.0793976728269,
          2000.082135523614,
          2000.084873374401,
          2001.002737850787,
          2001.0054757015744,
          2001.0082135523614,
          2001.0109514031485,
          2001.0136892539356,
          2001.0164271047229,
          2001.01916495551,
          2001.021902806297,
          2001.024640657084,
          2001.0273785078714,
          2001.0301163586585,
          2001.0328542094455,
          2001.0355920602328,
          2001.03832991102,
          2001.041067761807,
          2001.043805612594,
          2001.0465434633813,
          2001.0492813141684,
          2001.0520191649555,
          2001.0547570157426,
          2001.0574948665299,
          2001.060232717317,
          2001.062970568104,
          2001.065708418891,
          2001.0684462696784,
          2001.0711841204654,
          2001.0739219712525,
          2001.0766598220398,
          2001.0793976728269,
          2001.082135523614,
          2001.084873374401,
          2002.002737850787,
          2002.0054757015744,
          2002.0082135523614,
          2002.0109514031485,
          2002.0136892539356,
          2002.0164271047229,
          2002.01916495551,
          2002.021902806297,
          2002.024640657084,
          2002.0273785078714,
          2002.0301163586585,
          2002.0328542094455,
          2002.0355920602328,
          2002.03832991102,
          2002.041067761807,
          2002.043805612594,
          2002.0465434633813,
          2002.0492813141684,
          2002.0520191649555,
          2002.0547570157426,
          2002.0574948665299,
          2002.060232717317,
          2002.062970568104,
          2002.065708418891,
          2002.0684462696784,
          2002.0711841204654,
          2002.0739219712525,
          2002.0766598220398,
          2002.0793976728269,
          2002.082135523614,
          2002.084873374401,
          2003.002737850787,
          2003.0054757015744,
          2003.0082135523614,
          2003.0109514031485,
          2003.0136892539356,
          2003.0164271047229,
          2003.01916495551,
          2003.021902806297,
          2003.024640657084,
          2003.0273785078714,
          2003.0301163586585,
          2003.0328542094455,
          2003.0355920602328,
          2003.03832991102,
          2003.041067761807,
          2003.043805612594,
          2003.0465434633813,
          2003.0492813141684,
          2003.0520191649555,
          2003.0547570157426,
          2003.0574948665299,
          2003.060232717317,
          2003.062970568104,
          2003.065708418891,
          2003.0684462696784,
          2003.0711841204654,
          2003.0739219712525,
          2003.0766598220398,
          2003.0793976728269,
          2003.082135523614,
          2003.084873374401,
          2004.002737850787,
          2004.0054757015744,
          2004.0082135523614,
          2004.0109514031485,
          2004.0136892539356,
          2004.0164271047229,
          2004.01916495551,
          2004.021902806297,
          2004.024640657084,
          2004.0273785078714,
          2004.0301163586585,
          2004.0328542094455,
          2004.0355920602328,
          2004.03832991102,
          2004.041067761807,
          2004.043805612594,
          2004.0465434633813,
          2004.0492813141684,
          2004.0520191649555,
          2004.0547570157426,
          2004.0574948665299,
          2004.060232717317,
          2004.062970568104,
          2004.065708418891,
          2004.0684462696784,
          2004.0711841204654,
          2004.0739219712525,
          2004.0766598220398,
          2004.0793976728269,
          2004.082135523614,
          2004.084873374401,
          2005.002737850787,
          2005.0054757015744,
          2005.0082135523614,
          2005.0109514031485,
          2005.0136892539356,
          2005.0164271047229,
          2005.01916495551,
          2005.021902806297,
          2005.024640657084,
          2005.0273785078714,
          2005.0301163586585,
          2005.0328542094455,
          2005.0355920602328,
          2005.03832991102,
          2005.041067761807,
          2005.043805612594,
          2005.0465434633813,
          2005.0492813141684,
          2005.0520191649555,
          2005.0547570157426,
          2005.0574948665299,
          2005.060232717317,
          2005.062970568104,
          2005.065708418891,
          2005.0684462696784,
          2005.0711841204654,
          2005.0739219712525,
          2005.0766598220398,
          2005.0793976728269,
          2005.082135523614,
          2005.084873374401,
          2006.002737850787,
          2006.0054757015744,
          2006.0082135523614,
          2006.0109514031485,
          2006.0136892539356,
          2006.0164271047229,
          2006.01916495551,
          2006.021902806297,
          2006.024640657084,
          2006.0273785078714,
          2006.0301163586585,
          2006.0328542094455,
          2006.0355920602328,
          2006.03832991102,
          2006.041067761807,
          2006.043805612594,
          2006.0465434633813,
          2006.0492813141684,
          2006.0520191649555,
          2006.0547570157426,
          2006.0574948665299,
          2006.060232717317,
          2006.062970568104,
          2006.065708418891,
          2006.0684462696784,
          2006.0711841204654,
          2006.0739219712525,
          2006.0766598220398,
          2006.0793976728269,
          2006.082135523614,
          2006.084873374401,
          2007.002737850787,
          2007.0054757015744,
          2007.0082135523614,
          2007.0109514031485,
          2007.0136892539356,
          2007.0164271047229,
          2007.01916495551,
          2007.021902806297,
          2007.024640657084,
          2007.0273785078714,
          2007.0301163586585,
          2007.0328542094455,
          2007.0355920602328,
          2007.03832991102,
          2007.041067761807,
          2007.043805612594,
          2007.0465434633813,
          2007.0492813141684,
          2007.0520191649555,
          2007.0547570157426,
          2007.0574948665299,
          2007.060232717317,
          2007.062970568104,
          2007.065708418891,
          2007.0684462696784,
          2007.0711841204654,
          2007.0739219712525,
          2007.0766598220398,
          2007.0793976728269,
          2007.082135523614,
          2007.084873374401,
          2008.002737850787,
          2008.0054757015744,
          2008.0082135523614,
          2008.0109514031485,
          2008.0136892539356,
          2008.0164271047229,
          2008.01916495551,
          2008.021902806297,
          2008.024640657084,
          2008.0273785078714,
          2008.0301163586585,
          2008.0328542094455,
          2008.0355920602328,
          2008.03832991102,
          2008.041067761807,
          2008.043805612594,
          2008.0465434633813,
          2008.0492813141684,
          2008.0520191649555,
          2008.0547570157426,
          2008.0574948665299,
          2008.060232717317,
          2008.062970568104,
          2008.065708418891,
          2008.0684462696784,
          2008.0711841204654,
          2008.0739219712525,
          2008.0766598220398,
          2008.0793976728269,
          2008.082135523614,
          2008.084873374401,
          2009.002737850787,
          2009.0054757015744,
          2009.0082135523614,
          2009.0109514031485,
          2009.0136892539356,
          2009.0164271047229,
          2009.01916495551,
          2009.021902806297,
          2009.024640657084,
          2009.0273785078714,
          2009.0301163586585,
          2009.0328542094455,
          2009.0355920602328,
          2009.03832991102,
          2009.041067761807,
          2009.043805612594,
          2009.0465434633813,
          2009.0492813141684,
          2009.0520191649555,
          2009.0547570157426,
          2009.0574948665299,
          2009.060232717317,
          2009.062970568104,
          2009.065708418891,
          2009.0684462696784,
          2009.0711841204654,
          2009.0739219712525,
          2009.0766598220398,
          2009.0793976728269,
          2009.082135523614,
          2009.084873374401,
          2010.002737850787,
          2010.0054757015744,
          2010.0082135523614,
          2010.0109514031485,
          2010.0136892539356,
          2010.0164271047229,
          2010.01916495551,
          2010.021902806297,
          2010.024640657084,
          2010.0273785078714,
          2010.0301163586585,
          2010.0328542094455,
          2010.0355920602328,
          2010.03832991102,
          2010.041067761807,
          2010.043805612594,
          2010.0465434633813,
          2010.0492813141684,
          2010.0520191649555,
          2010.0547570157426,
          2010.0574948665299,
          2010.060232717317,
          2010.062970568104,
          2010.065708418891,
          2010.0684462696784,
          2010.0711841204654,
          2010.0739219712525,
          2010.0766598220398,
          2010.0793976728269,
          2010.082135523614,
          2010.084873374401,
          2011.002737850787,
          2011.0054757015744,
          2011.0082135523614,
          2011.0109514031485,
          2011.0136892539356,
          2011.0164271047229,
          2011.01916495551,
          2011.021902806297,
          2011.024640657084,
          2011.0273785078714,
          2011.0301163586585,
          2011.0328542094455,
          2011.0355920602328,
          2011.03832991102,
          2011.041067761807,
          2011.043805612594,
          2011.0465434633813,
          2011.0492813141684,
          2011.0520191649555,
          2011.0547570157426,
          2011.0574948665299,
          2011.060232717317,
          2011.062970568104,
          2011.065708418891,
          2011.0684462696784,
          2011.0711841204654,
          2011.0739219712525,
          2011.0766598220398,
          2011.0793976728269,
          2011.082135523614,
          2011.084873374401,
          2012.002737850787,
          2012.0054757015744,
          2012.0082135523614,
          2012.0109514031485,
          2012.0136892539356,
          2012.0164271047229,
          2012.01916495551,
          2012.021902806297,
          2012.024640657084,
          2012.0273785078714,
          2012.0301163586585,
          2012.0328542094455,
          2012.0355920602328,
          2012.03832991102,
          2012.041067761807,
          2012.043805612594,
          2012.0465434633813,
          2012.0492813141684,
          2012.0520191649555,
          2012.0547570157426,
          2012.0574948665299,
          2012.060232717317,
          2012.062970568104,
          2012.065708418891,
          2012.0684462696784,
          2012.0711841204654,
          2012.0739219712525,
          2012.0766598220398,
          2012.0793976728269,
          2012.082135523614,
          2012.084873374401,
          2013.002737850787,
          2013.0054757015744,
          2013.0082135523614,
          2013.0109514031485,
          2013.0136892539356,
          2013.0164271047229,
          2013.01916495551,
          2013.021902806297,
          2013.024640657084,
          2013.0273785078714,
          2013.0301163586585,
          2013.0328542094455,
          2013.0355920602328,
          2013.03832991102,
          2013.041067761807,
          2013.043805612594,
          2013.0465434633813,
          2013.0492813141684,
          2013.0520191649555,
          2013.0547570157426,
          2013.0574948665299,
          2013.060232717317,
          2013.062970568104,
          2013.065708418891,
          2013.0684462696784,
          2013.0711841204654,
          2013.0739219712525,
          2013.0766598220398,
          2013.0793976728269,
          2013.082135523614,
          2013.084873374401,
          2014.002737850787,
          2014.0054757015744,
          2014.0082135523614,
          2014.0109514031485,
          2014.0136892539356,
          2014.0164271047229,
          2014.01916495551,
          2014.021902806297,
          2014.024640657084,
          2014.0273785078714,
          2014.0301163586585,
          2014.0328542094455,
          2014.0355920602328,
          2014.03832991102,
          2014.041067761807,
          2014.043805612594,
          2014.0465434633813,
          2014.0492813141684,
          2014.0520191649555,
          2014.0547570157426,
          2014.0574948665299,
          2014.060232717317,
          2014.062970568104,
          2014.065708418891,
          2014.0684462696784,
          2014.0711841204654,
          2014.0739219712525,
          2014.0766598220398,
          2014.0793976728269,
          2014.082135523614,
          2014.084873374401,
          2015.002737850787,
          2015.0054757015744,
          2015.0082135523614,
          2015.0109514031485,
          2015.0136892539356,
          2015.0164271047229,
          2015.01916495551,
          2015.021902806297,
          2015.024640657084,
          2015.0273785078714,
          2015.0301163586585,
          2015.0328542094455,
          2015.0355920602328,
          2015.03832991102,
          2015.041067761807,
          2015.043805612594,
          2015.0465434633813,
          2015.0492813141684,
          2015.0520191649555,
          2015.0547570157426,
          2015.0574948665299,
          2015.060232717317,
          2015.062970568104,
          2015.065708418891,
          2015.0684462696784,
          2015.0711841204654,
          2015.0739219712525,
          2015.0766598220398,
          2015.0793976728269,
          2015.082135523614,
          2015.084873374401,
          2016.002737850787,
          2016.0054757015744,
          2016.0082135523614,
          2016.0109514031485,
          2016.0136892539356,
          2016.0164271047229,
          2016.01916495551,
          2016.021902806297,
          2016.024640657084,
          2016.0273785078714,
          2016.0301163586585,
          2016.0328542094455,
          2016.0355920602328,
          2016.03832991102,
          2016.041067761807,
          2016.043805612594,
          2016.0465434633813,
          2016.0492813141684,
          2016.0520191649555,
          2016.0547570157426,
          2016.0574948665299,
          2016.060232717317,
          2016.062970568104,
          2016.065708418891,
          2016.0684462696784,
          2016.0711841204654,
          2016.0739219712525,
          2016.0766598220398,
          2016.0793976728269,
          2016.082135523614,
          2016.084873374401,
          2017.002737850787,
          2017.0054757015744,
          2017.0082135523614,
          2017.0109514031485,
          2017.0136892539356,
          2017.0164271047229,
          2017.01916495551,
          2017.021902806297,
          2017.024640657084,
          2017.0273785078714,
          2017.0301163586585,
          2017.0328542094455,
          2017.0355920602328,
          2017.03832991102,
          2017.041067761807,
          2017.043805612594,
          2017.0465434633813,
          2017.0492813141684,
          2017.0520191649555,
          2017.0547570157426,
          2017.0574948665299,
          2017.060232717317,
          2017.062970568104,
          2017.065708418891,
          2017.0684462696784,
          2017.0711841204654,
          2017.0739219712525,
          2017.0766598220398,
          2017.0793976728269,
          2017.082135523614,
          2017.084873374401
         ],
         "xaxis": "x",
         "y": [
          42,
          36,
          35,
          34,
          32,
          39,
          40,
          35,
          31,
          34,
          32,
          26,
          29,
          26,
          31,
          28,
          29,
          28,
          27,
          29,
          32,
          44,
          43,
          34,
          30,
          32,
          33,
          25,
          34,
          30,
          27,
          26,
          22,
          17,
          20,
          28,
          37,
          26,
          18,
          13,
          12,
          17,
          19,
          19,
          25,
          28,
          26,
          33,
          29,
          24,
          20,
          22,
          21,
          16,
          17,
          17,
          26,
          24,
          16,
          26,
          28,
          29,
          24,
          6,
          6,
          14,
          22,
          35,
          33,
          31,
          30,
          31,
          29,
          18,
          11,
          8,
          15,
          13,
          15,
          9,
          24,
          37,
          42,
          36,
          25,
          10,
          7,
          21,
          19,
          7,
          6,
          7,
          0,
          37,
          39,
          36,
          35,
          31,
          31,
          30,
          32,
          32,
          39,
          36,
          40,
          40,
          35,
          32,
          32,
          32,
          32,
          32,
          32,
          34,
          34,
          36,
          39,
          42,
          33,
          19,
          15,
          15,
          11,
          19,
          10,
          20,
          17,
          27,
          32,
          31,
          32,
          34,
          36,
          34,
          32,
          28,
          25,
          31,
          32,
          28,
          27,
          33,
          36,
          33,
          28,
          30,
          30,
          34,
          35,
          39,
          38,
          34,
          37,
          44,
          42,
          37,
          39,
          36,
          34,
          31,
          26,
          41,
          41,
          36,
          35,
          41,
          47,
          44,
          43,
          41,
          40,
          40,
          44,
          44,
          42,
          40,
          41,
          46,
          43,
          38,
          32,
          35,
          38,
          44,
          45,
          44,
          36,
          34,
          37,
          37,
          37,
          40,
          40,
          36,
          37,
          35,
          31,
          29,
          33,
          34,
          23,
          22,
          27,
          23,
          18,
          16,
          27,
          32,
          32,
          29,
          30,
          30,
          30,
          33,
          28,
          37,
          41,
          36,
          32,
          29,
          28,
          32,
          35,
          34,
          35,
          32,
          34,
          35,
          40,
          37,
          34,
          35,
          31,
          35,
          37,
          27,
          36,
          30,
          38,
          45,
          40,
          37,
          32,
          33,
          33,
          33,
          39,
          39,
          33,
          35,
          39,
          36,
          37,
          34,
          31,
          31,
          37,
          35,
          40,
          39,
          36,
          35,
          40,
          40,
          38,
          37,
          38,
          41,
          40,
          41,
          33,
          31,
          29,
          28,
          23,
          28,
          23,
          17,
          17,
          40,
          38,
          36,
          33,
          25,
          31,
          32,
          32,
          31,
          36,
          36,
          32,
          30,
          29,
          30,
          27,
          29,
          25,
          29,
          20,
          22,
          29,
          20,
          15,
          16,
          9,
          11,
          16,
          23,
          27,
          37,
          39,
          40,
          40,
          36,
          33,
          32,
          31,
          37,
          35,
          43,
          43,
          41,
          38,
          41,
          47,
          46,
          41,
          33,
          29,
          34,
          37,
          38,
          43,
          42,
          40,
          36,
          40,
          43,
          37,
          40,
          38,
          32,
          17,
          14,
          16,
          25,
          34,
          39,
          45,
          45,
          41,
          42,
          42,
          40,
          36,
          40,
          42,
          42,
          38,
          35,
          34,
          34,
          41,
          45,
          43,
          37,
          42,
          41,
          39,
          34,
          37,
          38,
          25,
          30,
          25,
          26,
          34,
          37,
          36,
          32,
          31,
          29,
          32,
          32,
          33,
          32,
          34,
          36,
          33,
          31,
          31,
          34,
          39,
          35,
          35,
          41,
          42,
          41,
          39,
          38,
          48,
          41,
          46,
          33,
          25,
          25,
          36,
          42,
          42,
          41,
          41,
          44,
          44,
          42,
          40,
          42,
          47,
          46,
          38,
          42,
          39,
          42,
          35,
          29,
          28,
          28,
          36,
          30,
          37,
          27,
          33,
          39,
          44,
          45,
          32,
          35,
          42,
          36,
          36,
          37,
          48,
          43,
          40,
          31,
          29,
          31,
          32,
          32,
          31,
          38,
          31,
          25,
          20,
          16,
          13,
          19,
          23,
          34,
          41,
          44,
          43,
          35,
          31,
          34,
          36,
          46,
          44,
          41,
          34,
          32,
          33,
          31,
          32,
          29,
          16,
          12,
          13,
          24,
          30,
          37,
          38,
          36,
          27,
          24,
          26,
          26,
          32,
          27,
          31,
          30,
          24,
          24,
          27,
          21,
          26,
          30,
          41,
          35,
          38,
          34,
          37,
          31,
          29,
          35,
          34,
          36,
          35,
          40,
          36,
          37,
          38,
          39,
          35,
          35,
          34,
          32,
          31,
          30,
          31,
          31,
          34,
          33,
          39,
          34,
          39,
          37,
          40,
          31,
          33,
          30,
          31,
          32,
          32,
          33,
          34,
          29,
          35,
          32,
          35,
          38,
          44,
          40,
          38,
          35,
          33,
          38,
          43,
          37,
          35,
          33,
          37,
          33,
          36,
          43,
          46,
          43,
          37,
          34,
          33,
          33,
          32,
          33,
          34,
          43,
          43,
          37,
          33,
          40,
          41,
          43,
          43,
          41,
          31,
          30,
          31,
          33,
          31,
          30,
          36,
          35,
          32,
          27,
          36,
          34,
          42,
          44,
          45,
          42,
          36,
          36,
          37,
          41,
          32,
          32,
          35,
          37,
          44,
          40,
          40,
          43,
          41,
          44,
          47,
          42,
          37,
          36,
          33,
          42,
          39,
          34,
          34,
          35,
          34,
          38,
          40,
          46,
          47,
          39,
          33,
          29,
          33,
          29,
          30,
          38,
          32,
          30,
          31,
          35,
          35,
          34,
          33,
          39,
          44,
          50,
          40,
          38,
          37,
          42,
          45,
          50,
          47,
          44,
          41,
          35,
          33,
          31,
          26,
          25,
          29,
          29,
          35,
          33,
          37,
          37,
          42,
          42,
          40,
          34,
          32,
          35,
          35,
          34,
          29,
          32,
          33,
          33,
          33,
          30,
          28,
          31,
          25,
          25,
          23,
          17,
          18,
          25,
          23,
          21,
          15,
          19,
          22,
          28,
          35,
          34,
          33,
          29,
          30,
          33,
          38,
          38,
          33,
          30,
          36,
          32,
          38,
          39,
          32,
          33,
          35,
          45,
          46,
          45,
          46,
          42,
          45,
          42,
          38,
          36,
          35,
          36,
          33,
          33,
          43,
          30,
          29,
          24,
          25,
          24,
          30,
          32,
          43,
          42,
          33,
          27,
          28,
          26,
          28,
          38,
          42,
          45,
          45,
          40,
          36,
          35,
          35,
          43,
          40,
          36,
          44,
          40,
          37,
          35,
          48,
          48,
          37,
          31,
          27,
          32,
          39,
          42,
          37,
          35,
          37,
          33,
          35,
          32,
          30,
          35,
          35,
          41,
          36,
          34,
          46,
          48,
          44,
          43,
          34,
          26,
          21,
          20,
          12,
          13,
          24,
          15,
          24,
          38,
          36,
          28,
          24,
          20,
          15,
          19,
          23,
          18,
          34,
          34,
          41,
          47,
          47,
          45,
          43,
          40,
          39,
          35,
          37,
          36,
          31,
          34,
          39,
          33,
          26,
          38,
          40,
          36,
          38,
          38,
          25,
          26,
          23,
          22,
          22,
          21,
          21,
          22,
          16,
          29,
          24,
          30,
          46,
          48,
          50,
          43,
          37,
          40,
          35,
          32,
          30,
          42,
          46,
          44,
          40,
          38,
          42,
          44,
          42,
          34,
          35,
          35,
          35,
          39,
          37,
          39,
          35,
          34,
          31,
          29,
          30,
          27,
          30,
          39,
          35,
          35,
          38,
          49,
          41,
          36,
          34,
          29,
          37,
          42,
          41,
          33,
          30,
          25,
          30,
          28,
          30,
          32,
          27,
          30,
          34,
          42,
          37,
          36,
          37,
          40,
          37,
          38,
          35,
          35,
          35,
          39,
          49,
          46,
          42,
          40,
          36,
          29,
          33,
          38,
          36,
          31,
          33,
          36,
          46,
          46,
          38,
          40,
          33,
          27,
          33,
          30,
          31,
          28,
          26,
          27,
          26,
          24,
          24,
          37,
          41,
          42,
          43,
          46,
          45,
          51,
          42,
          38,
          37,
          37,
          37,
          33,
          33,
          32,
          26,
          27,
          32,
          33,
          31,
          38,
          29,
          33,
          33,
          46,
          39,
          38,
          40,
          45,
          45,
          44,
          43,
          42,
          41,
          44,
          49,
          44,
          39,
          44,
          44,
          46,
          43,
          37,
          35,
          35,
          39,
          38,
          36,
          41,
          36,
          34,
          37,
          20,
          26,
          23,
          31,
          27,
          29,
          33,
          34,
          35,
          36,
          40,
          39,
          36,
          37,
          36,
          36,
          34,
          30,
          39,
          43,
          34,
          32,
          34,
          32,
          32,
          32,
          35,
          27,
          26,
          25,
          26,
          44,
          37,
          33,
          34,
          30,
          26,
          24,
          26,
          26,
          28,
          31,
          35,
          34,
          38,
          38,
          40,
          31,
          25,
          26,
          30,
          32,
          32,
          31,
          33,
          32,
          21,
          19,
          19,
          15,
          19,
          33,
          42,
          38,
          32,
          34,
          37,
          41,
          35,
          35,
          41,
          38,
          33,
          34,
          34,
          32,
          31,
          29,
          41,
          45,
          46,
          45,
          50,
          47,
          43,
          40,
          41,
          39,
          34,
          35,
          38,
          37,
          37,
          33,
          31,
          30,
          28,
          15,
          14,
          24,
          37,
          35,
          35,
          39,
          38,
          41,
          44,
          43,
          40,
          39,
          35,
          35,
          38,
          32,
          31,
          42,
          39,
          37,
          41,
          38,
          40,
          43,
          43,
          37,
          29,
          36,
          40,
          41,
          42,
          40,
          47,
          41,
          40,
          46,
          39,
          37,
          37,
          30,
          29,
          34,
          43,
          48,
          42,
          41,
          31,
          37,
          44,
          45,
          45,
          46,
          40,
          38,
          47,
          42,
          39,
          35,
          42,
          47,
          49,
          47,
          49,
          42,
          42,
          43,
          43,
          41,
          38,
          34,
          31,
          27,
          26,
          25,
          21,
          25,
          28,
          34,
          43,
          44,
          49,
          43,
          44,
          45,
          46,
          41,
          38,
          33,
          24,
          26,
          27,
          26,
          35,
          33,
          32,
          29,
          29,
          25,
          25,
          32,
          29,
          34,
          39,
          33,
          37,
          33,
          41,
          39,
          36,
          33,
          31,
          31,
          34,
          26,
          29,
          35,
          29,
          26,
          31,
          36,
          34,
          36,
          32,
          41,
          38,
          35,
          43,
          44,
          43,
          39,
          33,
          39,
          37,
          41,
          42,
          41,
          42,
          40,
          37,
          39,
          40,
          40,
          36,
          35,
          35,
          41,
          37,
          42,
          46,
          43,
          41,
          42,
          43,
          37,
          29,
          27,
          25,
          23,
          28,
          39,
          47,
          40,
          33,
          31,
          25,
          28,
          34,
          34,
          32,
          30,
          30,
          30,
          38,
          39,
          40,
          40,
          42,
          42,
          43,
          40,
          39,
          28,
          23,
          28,
          37,
          37,
          36,
          34,
          34,
          36,
          40,
          37,
          37,
          38,
          41,
          38,
          34,
          35,
          33,
          33,
          37,
          35,
          35,
          35,
          28,
          35,
          32,
          37,
          44,
          36,
          33,
          29,
          39,
          40,
          45,
          38,
          33,
          31,
          28,
          31,
          39,
          35,
          34,
          42,
          34,
          34,
          39,
          45,
          42,
          37,
          33,
          33,
          36,
          30,
          25,
          32,
          36,
          30,
          39,
          35,
          42,
          39,
          32,
          33,
          33,
          40,
          44,
          45,
          43,
          43,
          43,
          42,
          36,
          36,
          39,
          43,
          44,
          41,
          40,
          35,
          26,
          28,
          32,
          40,
          40,
          37,
          37,
          37,
          32,
          38,
          40,
          38,
          33,
          32,
          40,
          33,
          31,
          28,
          26,
          30,
          33,
          39,
          39,
          40,
          46,
          48,
          47,
          46,
          41,
          42,
          45,
          40,
          33,
          31,
          27,
          28,
          34,
          32,
          29,
          27,
          26,
          30,
          26,
          29,
          36,
          37,
          39,
          37,
          35,
          35,
          32,
          25,
          31,
          40,
          42,
          37,
          34,
          38,
          42,
          42,
          38,
          33,
          28,
          32,
          33,
          38,
          37,
          46,
          47,
          42,
          37,
          45,
          44,
          47,
          48,
          47,
          24,
          28,
          32,
          31,
          24,
          22,
          24,
          28,
          30,
          26,
          24,
          23,
          32,
          32,
          33,
          32,
          29,
          29,
          36,
          38,
          37,
          32,
          31,
          35,
          47,
          47,
          44,
          45,
          41,
          35,
          28,
          45,
          45,
          46,
          42,
          39,
          37,
          41,
          42,
          44,
          44,
          46,
          47,
          46,
          39,
          44,
          43,
          38,
          33,
          35,
          34,
          36,
          47,
          46,
          42,
          41,
          40,
          39,
          38,
          33,
          32,
          27,
          38,
          30,
          37,
          29,
          31,
          31,
          33,
          43,
          46,
          44,
          44,
          42,
          44,
          43,
          41,
          39,
          36,
          43,
          34,
          49,
          45,
          40,
          43,
          40,
          42,
          43,
          40,
          46,
          47,
          48,
          50,
          45,
          47,
          42,
          37,
          36,
          39,
          46,
          41,
          41,
          44,
          37,
          33,
          41,
          46,
          42,
          36,
          33,
          32,
          32,
          37,
          32,
          33,
          33,
          33,
          32,
          31,
          30,
          25,
          18,
          15,
          20,
          48,
          42,
          39,
          37,
          34,
          37,
          45,
          42,
          42,
          44,
          38,
          28,
          23,
          21,
          23,
          31,
          41,
          42,
          47,
          41,
          36,
          38,
          37,
          33,
          26,
          22,
          24,
          36,
          42,
          46,
          45,
          37,
          32,
          32,
          35,
          39,
          42,
          36,
          28,
          25,
          24,
          23,
          25,
          34,
          41,
          38,
          38,
          43,
          45,
          41,
          39,
          39,
          37,
          44,
          43,
          39,
          44,
          42,
          45,
          47,
          41,
          41,
          39,
          32,
          30,
          35,
          35,
          40,
          40,
          42,
          41,
          43,
          40,
          42,
          39,
          43,
          40,
          39,
          39,
          43,
          43,
          42,
          41,
          38,
          32,
          30,
          34,
          35,
          37,
          37,
          35,
          34,
          34,
          37,
          37,
          36,
          42,
          37,
          33,
          40,
          36,
          35,
          34,
          34,
          32,
          36,
          37,
          34,
          36,
          35,
          30,
          29,
          35,
          39,
          37,
          33,
          33,
          38,
          35,
          35,
          29,
          34,
          39,
          37,
          34,
          39,
          40,
          45,
          40,
          36,
          34,
          37,
          33,
          41,
          41,
          39,
          37,
          37,
          30,
          27,
          28,
          39,
          37,
          37,
          40,
          34,
          47,
          38,
          31,
          34,
          29,
          28,
          37,
          40,
          38,
          41,
          41,
          38,
          40,
          42,
          42,
          51,
          46,
          40,
          39,
          42,
          39,
          34,
          35,
          35,
          30,
          36,
          33,
          37,
          39,
          35,
          35,
          35,
          42,
          33,
          31,
          28,
          25,
          25,
          33,
          37,
          41,
          45,
          45,
          46,
          40,
          33,
          33,
          36,
          39,
          37,
          37,
          42,
          42,
          35,
          32,
          37,
          37,
          35,
          39,
          38,
          41,
          43,
          47,
          45,
          45,
          43,
          45,
          40,
          39,
          48,
          46,
          30,
          32,
          23,
          20,
          20,
          22,
          30,
          37,
          40,
          38,
          33,
          35,
          37,
          46,
          45,
          41,
          35,
          43,
          41,
          36,
          33,
          32,
          41,
          34,
          33,
          40,
          41,
          44,
          48,
          39,
          37,
          37,
          30,
          28,
          28,
          24,
          26,
          34,
          33,
          32,
          27,
          23,
          39,
          31,
          27,
          26,
          32,
          42,
          53,
          50,
          49,
          49,
          50,
          45,
          42,
          37,
          40,
          41,
          40,
          42,
          41,
          42,
          46,
          41,
          38,
          42,
          48,
          50,
          43,
          39,
          46,
          46,
          44,
          44,
          45,
          38,
          37,
          41,
          45,
          42,
          43,
          39,
          42,
          44,
          45,
          41,
          38,
          41,
          40,
          40,
          40,
          43,
          41,
          34,
          45,
          40,
          30,
          31,
          36,
          40,
          38,
          34,
          28,
          22,
          19,
          21,
          20,
          23,
          28,
          33,
          34,
          38,
          37,
          37,
          40,
          43,
          35,
          33,
          36,
          32,
          28,
          28,
          33,
          28,
          41,
          37,
          40,
          41,
          37,
          33,
          34,
          33,
          36,
          37,
          41,
          41,
          38,
          33,
          30,
          33,
          34,
          34,
          35,
          33,
          25,
          25,
          26,
          33,
          25,
          26,
          30,
          31,
          35,
          34,
          36,
          36,
          33,
          32,
          30,
          34,
          42,
          50,
          37,
          36,
          39,
          44,
          46,
          40,
          36,
          35,
          32,
          32,
          30,
          29,
          30,
          31,
          32,
          31,
          29,
          29,
          25,
          30,
          37,
          35,
          31,
          34,
          45,
          43,
          39,
          44,
          44,
          42,
          40,
          39,
          43,
          44,
          45,
          47,
          46,
          45,
          44,
          37,
          42,
          49,
          48,
          42,
          45,
          40,
          40,
          37,
          41,
          38,
          35,
          41,
          44,
          43,
          45,
          26,
          25,
          27,
          28,
          35,
          44,
          37,
          34,
          31,
          30,
          30,
          33,
          43,
          49,
          47,
          50,
          42,
          37,
          35,
          33,
          44,
          40,
          41,
          44,
          43,
          39,
          37,
          39,
          43,
          37,
          34,
          41,
          37,
          45,
          42,
          37,
          36,
          37,
          37,
          41,
          33,
          30,
          29,
          27,
          33,
          26,
          27,
          32,
          27,
          27,
          30,
          38,
          36,
          34,
          36,
          40,
          34,
          28,
          33,
          39,
          43,
          43,
          27,
          30,
          29,
          36,
          40,
          37,
          40,
          42,
          35,
          31,
          27,
          25,
          24,
          28,
          31,
          25,
          27,
          30,
          31,
          31,
          29,
          29,
          36,
          34,
          37,
          39,
          39,
          38,
          41,
          44,
          45,
          38,
          43,
          37,
          33,
          31,
          31,
          41,
          45,
          42,
          47,
          45,
          42,
          50,
          45,
          42,
          40,
          37,
          33,
          38,
          37,
          35,
          42,
          37,
          34,
          34,
          33,
          35,
          43,
          45,
          43,
          42,
          26,
          32,
          35,
          38,
          49,
          43,
          42,
          35,
          38,
          43,
          45,
          40,
          37,
          33,
          34,
          42,
          38,
          45,
          43,
          38,
          31,
          43,
          47,
          52,
          45,
          43,
          47,
          41,
          38,
          34,
          38,
          28,
          25,
          31,
          35,
          36,
          37,
          34,
          36,
          30,
          38,
          35,
          40,
          41,
          36,
          33,
          42,
          43,
          43,
          42,
          40,
          45,
          44,
          43,
          37,
          38,
          46,
          49,
          46,
          41,
          38,
          37,
          28,
          26,
          21,
          22,
          21,
          20,
          29,
          35,
          34,
          32,
          28,
          25,
          25,
          26,
          27,
          30,
          39,
          44,
          41,
          37,
          40,
          37,
          34,
          31,
          35,
          41,
          37,
          37,
          37,
          40,
          34
         ],
         "yaxis": "y"
        }
       ],
       "layout": {
        "legend": {
         "tracegroupgap": 0
        },
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             }
            },
            "type": "barpolar"
           }
          ],
          "box": [
           {
            "boxpoints": "outliers",
            "notched": false,
            "type": "box"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "#C8D4E3",
             "linecolor": "#C8D4E3",
             "minorgridcolor": "#C8D4E3",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "#C8D4E3",
             "linecolor": "#C8D4E3",
             "minorgridcolor": "#C8D4E3",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             },
             "line": {
              "width": 1
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             },
             "opacity": 0.8
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             },
             "opacity": 0.8
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "white",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "#C8D4E3"
          },
          "height": 400,
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "margin": {
           "b": 2,
           "l": 2,
           "r": 2,
           "t": 30
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "white",
          "polar": {
           "angularaxis": {
            "gridcolor": "#EBF0F8",
            "linecolor": "#EBF0F8",
            "ticks": ""
           },
           "bgcolor": "white",
           "radialaxis": {
            "gridcolor": "#EBF0F8",
            "linecolor": "#EBF0F8",
            "ticks": ""
           }
          },
          "scene": {
           "camera": {
            "eye": {
             "x": 1.5,
             "y": 1.5,
             "z": 0.1
            }
           },
           "xaxis": {
            "backgroundcolor": "white",
            "gridcolor": "#DFE8F3",
            "gridwidth": 2,
            "linecolor": "#EBF0F8",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "#EBF0F8"
           },
           "yaxis": {
            "backgroundcolor": "white",
            "gridcolor": "#DFE8F3",
            "gridwidth": 2,
            "linecolor": "#EBF0F8",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "#EBF0F8"
           },
           "zaxis": {
            "backgroundcolor": "white",
            "gridcolor": "#DFE8F3",
            "gridwidth": 2,
            "linecolor": "#EBF0F8",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "#EBF0F8"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "#DFE8F3",
            "linecolor": "#A2B1C6",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "#DFE8F3",
            "linecolor": "#A2B1C6",
            "ticks": ""
           },
           "bgcolor": "white",
           "caxis": {
            "gridcolor": "#DFE8F3",
            "linecolor": "#A2B1C6",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.5
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "#EBF0F8",
           "linecolor": "#EBF0F8",
           "linewidth": 2,
           "showline": true,
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "#EBF0F8",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "#EBF0F8",
           "linecolor": "#EBF0F8",
           "linewidth": 2,
           "showline": true,
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "#EBF0F8",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Temperatures over time (°F)"
        },
        "xaxis": {
         "anchor": "y",
         "domain": [
          0,
          1
         ],
         "title": {
          "text": "year"
         }
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0,
          1
         ],
         "title": {
          "text": "min temperature"
         }
        }
       }
      }
     },
     "metadata": {}
    }
   ],
   "source": [
    "from datetime import datetime\n",
    "import pandas\n",
    "import graphing # custom graphing code. See our GitHub repository\n",
    "\n",
    "# Load a file containing weather data for Seattle\n",
    "data = pandas.read_csv('Data/seattleWeather_1948-2017.csv', parse_dates=['date'])\n",
    "\n",
    "# Remove all dates after July 1st because we have to to plant onions before summer begins\n",
    "data = data[[d.month == 1 for d in data.date]].copy()\n",
    "\n",
    "\n",
    "# Convert the dates into numbers so we can use it in our models\n",
    "# We make a year column which can contain fractions. For example\n",
    "# 1948.5 is half way through the year 1948\n",
    "data[\"year\"] = [(d.year + d.timetuple().tm_yday / 365.25) for d in data.date]\n",
    "\n",
    "# Let's take a quick look at our data\n",
    "print(\"Visual Check:\")\n",
    "graphing.scatter_2D(data, \n",
    "                    label_x=\"year\", \n",
    "                    label_y=\"min_temperature\",\n",
    "                    title=\"Temperatures over time (°F)\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fitting a model automatically\n",
    "\n",
    "Let's fit a line to this data well using an existing library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "C:\\Users\\leere\\Anaconda3\\envs\\ml_crash_course\\lib\\site-packages\\statsmodels\\tools\\_testing.py:19: FutureWarning:\n",
      "\n",
      "pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.\n",
      "\n",
      "The model is: y = 0.088 * X + -138.867\n"
     ]
    }
   ],
   "source": [
    "import statsmodels.formula.api as smf\n",
    "\n",
    "# Perform linear regression to fit a line to our data\n",
    "# NB OLS uses the sum or mean of squared differences as a cost function\n",
    "# which we are familiar with from our last exercise \n",
    "model = smf.ols(formula = \"min_temperature ~ year\", data = data).fit()\n",
    "\n",
    "# Print the model\n",
    "intercept = model.params[0]\n",
    "slope = model.params[1]\n",
    "\n",
    "print(f\"The model is: y = {slope:0.3f} * X + {intercept:0.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ooh, some math! Don't let that bother you. It's quite common for labels and features to be referred to as `y` and `X`, respectively.\n",
    "Here:\n",
    "* `y` is temperature (°F)\n",
    "* `X` is year\n",
    "* -83 is a _model parameter_ that acts as the line offset  \n",
    "* 0.063 is a _model parameter_ that defines the line slope (in °F per year)\n",
    "\n",
    "So this little equation states that the model estimates temperature by multiplying the year by `0.063`, then subtracting `83`.\n",
    "\n",
    "How did the library calculate these values? Let's go through the process.\n",
    "\n",
    "## Model Selection\n",
    "\n",
    "The first step is always selecting a model. Let's re-use the model we used in previous exercises."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Model class ready\n"
     ]
    }
   ],
   "source": [
    "class MyModel:\n",
    "\n",
    "    def __init__(self):\n",
    "        '''\n",
    "        Creates a new MyModel\n",
    "        '''\n",
    "        # Straight lines described by two parameters:\n",
    "        # The slop is the angle of the line\n",
    "        self.slope = 0\n",
    "        # The intercept moves the line up or down\n",
    "        self.intercept = 0\n",
    "\n",
    "    def predict(self, date):\n",
    "        '''\n",
    "        Estimates the temperature from the date\n",
    "        '''\n",
    "        return date * self.slope + self.intercept\n",
    "\n",
    "    def get_summary(self):\n",
    "        '''\n",
    "        Returns a string that summarises the model\n",
    "        '''\n",
    "        return f\"y = {self.slope} * x + {self.intercept}\"\n",
    "\n",
    "print(\"Model class ready\")"
   ]
  },
  {
   "source": [
    "## Fitting our model with gradient descent\n",
    "\n",
    "The automatic method used the _ordinary least squares_ (OLS) method, which is the standard way to fit a line. OLS uses the mean (or sum) of square differences as a cost function (recall our experimentation with the sum of squared differences in the last exercise). Let's replicate the above line fitting, and break down each step so we can watch it in action.\n",
    "\n",
    "Recall that for each iteration, our training conducts three steps: \n",
    "\n",
    "1) Estimation of `y` (temperature) from `X` (year)\n",
    "\n",
    "2) Calculation of the cost function and its slope\n",
    "\n",
    "3) Adjustment of our model according to this slope\n",
    "\n",
    "Let's implement this now to watch it in action. Note that, **to keep things simple, we'll focus on estimating one parameter (line slope) for now**.\n",
    "\n",
    "### Visualising the error function\n",
    "\n",
    "First, lets look at the error function for this data. Normally we don't know this in advance, but for learning purposes, let's calculate it now for different potential models"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "output_type": "display_data",
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "type": "surface",
         "x": [
          -100,
          -96.66666666666667,
          -93.33333333333333,
          -90,
          -86.66666666666667,
          -83.33333333333333,
          -80,
          -76.66666666666666,
          -73.33333333333333,
          -70
         ],
         "y": [
          0.06,
          0.06111111111111111,
          0.06222222222222222,
          0.06333333333333334,
          0.06444444444444444,
          0.06555555555555556,
          0.06666666666666667,
          0.06777777777777778,
          0.06888888888888889,
          0.07
         ],
         "z": [
          [
           315.569208432957,
           248.77807708093135,
           191.69284249871447,
           144.31350468630603,
           106.64006364370688,
           78.67251937091602,
           60.410871867934205,
           51.85512113476093,
           53.005267171396426,
           63.86130997784074
          ],
          [
           218.30613671998142,
           166.20051503916017,
           123.80079012814768,
           91.10696198694373,
           68.11903061554885,
           54.836996013962455,
           51.260858182184926,
           57.39061712021611,
           73.22627282805593,
           98.76782530570469
          ],
          [
           143.2652872292277,
           105.84517521961094,
           78.13095997980288,
           60.122641509803465,
           51.82021980961294,
           53.22369487923109,
           64.33306671865792,
           85.14833532789363,
           115.66950070693778,
           155.89656285579113
          ],
          [
           90.44665996069658,
           67.71205762228416,
           54.68335205368049,
           51.36054325488554,
           57.7436312258993,
           73.83261596672195,
           99.62749747735303,
           135.12827575779326,
           180.3349508080417,
           235.24752262809952
          ],
          [
           59.85025491438758,
           51.80116224717955,
           53.45796634978024,
           64.82066722218978,
           85.88926486440783,
           116.66375927643493,
           157.1441504582703,
           207.33043840991502,
           267.22262313136775,
           336.82070462263
          ],
          [
           51.47607209030074,
           58.11248909429716,
           74.4548028681023,
           100.50301341171637,
           136.2571207251388,
           181.71712480837044,
           236.8830256614102,
           301.75482328425943,
           376.3325176769165,
           460.61610883938334
          ],
          [
           65.32411148843615,
           86.64603816363694,
           117.67386160864648,
           158.40758182346502,
           208.84719880809172,
           268.99271256252786,
           338.8441230867719,
           418.4014303808256,
           507.6646344446869,
           606.6337352783582
          ],
          [
           101.39437310879391,
           137.40180945519919,
           183.11514257141314,
           238.53437245743626,
           303.65949911326726,
           378.4905225389079,
           463.0274427343563,
           557.2702596996146,
           661.2189734346803,
           774.8735839395562
          ],
          [
           159.68685695137373,
           210.3798029689834,
           270.7786457564018,
           340.88338531362933,
           420.69402164066463,
           510.2105547375097,
           609.4329846041625,
           718.3613112406251,
           836.9955346468952,
           965.3356548229755
          ],
          [
           240.20156301617575,
           305.5800187049897,
           380.6643711636125,
           465.4546203920445,
           559.9507663902841,
           664.1528091583338,
           778.0607486961908,
           901.6745850038579,
           1034.994318081332,
           1178.019947928617
          ]
         ]
        }
       ],
       "layout": {
        "scene": {
         "xaxis": {
          "title": {
           "text": "Model intercept"
          }
         },
         "yaxis": {
          "title": {
           "text": "Model slope"
          }
         },
         "zaxis": {
          "title": {
           "text": "Cost"
          }
         }
        },
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             }
            },
            "type": "barpolar"
           }
          ],
          "box": [
           {
            "boxpoints": "outliers",
            "notched": false,
            "type": "box"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "#C8D4E3",
             "linecolor": "#C8D4E3",
             "minorgridcolor": "#C8D4E3",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "#C8D4E3",
             "linecolor": "#C8D4E3",
             "minorgridcolor": "#C8D4E3",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             },
             "line": {
              "width": 1
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             },
             "opacity": 0.8
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             },
             "opacity": 0.8
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "white",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "#C8D4E3"
          },
          "height": 400,
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "margin": {
           "b": 2,
           "l": 2,
           "r": 2,
           "t": 30
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "white",
          "polar": {
           "angularaxis": {
            "gridcolor": "#EBF0F8",
            "linecolor": "#EBF0F8",
            "ticks": ""
           },
           "bgcolor": "white",
           "radialaxis": {
            "gridcolor": "#EBF0F8",
            "linecolor": "#EBF0F8",
            "ticks": ""
           }
          },
          "scene": {
           "camera": {
            "eye": {
             "x": 1.5,
             "y": 1.5,
             "z": 0.1
            }
           },
           "xaxis": {
            "backgroundcolor": "white",
            "gridcolor": "#DFE8F3",
            "gridwidth": 2,
            "linecolor": "#EBF0F8",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "#EBF0F8"
           },
           "yaxis": {
            "backgroundcolor": "white",
            "gridcolor": "#DFE8F3",
            "gridwidth": 2,
            "linecolor": "#EBF0F8",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "#EBF0F8"
           },
           "zaxis": {
            "backgroundcolor": "white",
            "gridcolor": "#DFE8F3",
            "gridwidth": 2,
            "linecolor": "#EBF0F8",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "#EBF0F8"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "#DFE8F3",
            "linecolor": "#A2B1C6",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "#DFE8F3",
            "linecolor": "#A2B1C6",
            "ticks": ""
           },
           "bgcolor": "white",
           "caxis": {
            "gridcolor": "#DFE8F3",
            "linecolor": "#A2B1C6",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.5
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "#EBF0F8",
           "linecolor": "#EBF0F8",
           "linewidth": 2,
           "showline": true,
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "#EBF0F8",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "#EBF0F8",
           "linecolor": "#EBF0F8",
           "linewidth": 2,
           "showline": true,
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "#EBF0F8",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Cost for Different Model Parameters"
        }
       }
      }
     },
     "metadata": {}
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "x = data.year\n",
    "temperature_true = data.min_temperature\n",
    "\n",
    "# We will use a pre-built method to show a 3D plot\n",
    "# This requires a range of x values, a range of y values,\n",
    "# and a way to calculate z\n",
    "# Here, we set:\n",
    "#   x to a range of potential model intercepts\n",
    "#   y to a range of potential model slopes\n",
    "#   z is the cost for that combination of model parameters   \n",
    "\n",
    "# Choose a range of intercepts and slopes values\n",
    "intercepts = np.linspace(-100,-70,10)\n",
    "slopes = np.linspace(0.060,0.07,10)\n",
    "\n",
    "\n",
    "# Set a cost function. This will be the mean of squared differences\n",
    "def cost_function(temperature_estimate):\n",
    "    \"\"\"\n",
    "    Calculates cost for a given temperature estimate\n",
    "    Our cost function is the mean of squared differences (aka mean squared error)\n",
    "    \"\"\"\n",
    "    # Note that with numpy to square each value we use **\n",
    "    return np.mean((temperature_true - temperature_estimate) ** 2)\n",
    "\n",
    "def predict_and_calc_cost(intercept, slope):\n",
    "    '''\n",
    "    Uses the model to make a prediction, then calculates the cost \n",
    "    '''\n",
    "\n",
    "    # Predict temperature using these model parameters\n",
    "    temperature_estimate = x * slope + intercept\n",
    "\n",
    "    # Calculate cost\n",
    "    return cost_function(temperature_estimate)\n",
    "\n",
    "# Call the graphing method. This will use our cost function\n",
    "# which is above. If you would like to view this code in detail\n",
    "# then see this project's GitHub repository\n",
    "graphing.surface(x_values=intercepts, \n",
    "                y_values=slopes, \n",
    "                calc_z=predict_and_calc_cost, \n",
    "                title=\"Cost for Different Model Parameters\",\n",
    "                axis_title_x=\"Model intercept\",\n",
    "                axis_title_y=\"Model slope\",\n",
    "                axis_title_z=\"Cost\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The graph above is interactive. Try clicking and dragging the mouse to rotate it.\n",
    "\n",
    "Notice how cost changes with both intercept and with line slope. This is because our model has a slope and an intercept, which both will affect how well the line fits the data. A consequence is that the gradient of the cost function must also be described by two numbers - one for intercept, and one for line slope.\n",
    "\n",
    "Our lowest point on the graph is the location of the best line equation for our data: a slope of 0.063 and intercept of -83. Let's try to train a model to find this point.\n",
    "\n",
    "### Implementing Gradient Descent\n",
    "\n",
    "To implement gradient descent, we need a method that can calculate the gradient of the above curve. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Function is ready!\n"
     ]
    }
   ],
   "source": [
    "def calculate_gradient(temperature_estimate):\n",
    "    \"\"\"\n",
    "    This calculates gradient for a linear regession \n",
    "    using the Mean Squared Error cost function\n",
    "    \"\"\"\n",
    "\n",
    "    # The partial derivatives of MSE are as follows\n",
    "    # You don't need to be able to do this just yet but\n",
    "    # it is important to note these give you the two gradients\n",
    "    # that we need to train our model\n",
    "    error = temperature_estimate - temperature_true\n",
    "    grad_intercept = np.mean(error) * 2\n",
    "    grad_slope = (x * error).mean() * 2\n",
    "\n",
    "    return grad_intercept, grad_slope\n",
    "\n",
    "print(\"Function is ready!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now all we need is a starting guess, and a loop that will update this guess with each iteration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Next estimate: y = 0.0004687797653851128 * x + -83 Cost: 13808.558670566295\n",
      "Next estimate: y = 0.03275951794075497 * x + -83 Cost: 2889.024411091647\n",
      "Next estimate: y = 0.04742499360470808 * x + -83 Cost: 636.6505644658864\n",
      "Next estimate: y = 0.054085608648004095 * x + -83 Cost: 172.05309863882115\n",
      "Next estimate: y = 0.057110658347825645 * x + -83 Cost: 76.22051490664224\n",
      "Next estimate: y = 0.05848454433686817 * x + -83 Cost: 56.45311456485186\n",
      "Next estimate: y = 0.05910852175813128 * x + -83 Cost: 52.37569015391621\n",
      "Next estimate: y = 0.05939191341169742 * x + -83 Cost: 51.53463925499022\n",
      "Next estimate: y = 0.059520621320949006 * x + -83 Cost: 51.36115556938533\n",
      "Next estimate: y = 0.0595790765566465 * x + -83 Cost: 51.32537106548731\n",
      "Final estimate: y = 0.05960545010956096 * x + -83\n"
     ]
    }
   ],
   "source": [
    "def gradient_descent(learning_rate, number_of_iterations):\n",
    "    \"\"\"\n",
    "    Performs gradient descent for a one-variable function. \n",
    "\n",
    "    learning_rate: Larger numbers follow the gradient more aggressively\n",
    "    number_of_iterations: The maximum number of iterations to perform\n",
    "    \"\"\"\n",
    "\n",
    "    # Our starting guess is y = 0 * x - 83\n",
    "    # We are going to start with the correct intercept so that \n",
    "    # only the line's slope is estimated. This is just to keep\n",
    "    # things simple for this exercise\n",
    "    model = MyModel()\n",
    "    model.intercept = -83\n",
    "    model.slope = 0\n",
    "\n",
    "    for i in range(number_of_iterations):\n",
    "        # Calculate the predicted values\n",
    "        predicted_temperature = model.predict(x)\n",
    "\n",
    "        # == OPTIMISER ===\n",
    "        # Calculate the gradient\n",
    "        _, grad_slope = calculate_gradient(predicted_temperature)\n",
    "        # Update the estimation of the line\n",
    "        model.slope -= learning_rate * grad_slope\n",
    "\n",
    "        # Print the current estimation and cost every 100 iterations\n",
    "        if( i % 100 == 0):\n",
    "            estimate = model.predict(x)\n",
    "            cost = cost_function(estimate)\n",
    "            print(\"Next estimate:\", model.get_summary(), f\"Cost: {cost}\")\n",
    "\n",
    "    # Print the final model\n",
    "    print(f\"Final estimate:\", model.get_summary())\n",
    "\n",
    "# Run gradient descent\n",
    "gradient_descent(learning_rate=1E-9, number_of_iterations=1000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our model found the correct answer, but it took a number of steps. Looking at the print out we can see how it progressively stepped toward the correct solution.\n",
    "\n",
    "Now, what happens if we make the learning rate faster? This means taking larger steps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Next estimate: y = 0.004687797653851128 * x + -83 Cost: 11916.2876489164\n",
      "Next estimate: y = 0.059612444340874586 * x + -83 Cost: 51.31698829963089\n",
      "Final estimate: y = 0.0596277108755386 * x + -83\n"
     ]
    }
   ],
   "source": [
    "gradient_descent(learning_rate=1E-8, number_of_iterations=200)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It appears to have found the solution faster. If we increase the rate even faster, however, things don't go so well:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Next estimate: y = 0.2343898826925564 * x + -83 Cost: 120107.77635612259\n",
      "Next estimate: y = 8.757346630024033e+45 * x + -83 Cost: 3.014642180154946e+98\n",
      "Next estimate: y = 4.388313627755413e+92 * x + -83 Cost: 7.5698279399796e+191\n",
      "Next estimate: y = 2.1989875825539867e+139 * x + -83 Cost: 1.9007992198248367e+285\n",
      "Next estimate: y = 1.1019144934497266e+186 * x + -83 Cost: inf\n",
      "Final estimate: y = -1.8839690443755677e+232 * x + -83\n"
     ]
    }
   ],
   "source": [
    "gradient_descent(learning_rate=5E-7, number_of_iterations=500)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice how the cost is getting worse each time.\n",
    "\n",
    "This is because the steps it was taking were too large. While it would step towards the correct solution, it would step too far and actually get worse with each attempt.\n",
    "\n",
    "For each model, there is an ideal learning rate. This is something that requires experimentation.\n",
    "\n",
    "## Fitting multiple variables simultaneously\n",
    "\n",
    "We've just fit one variable here to keep things simple. Expanding this to fit multiple variables requires only a few small code changes:\n",
    "\n",
    "1) We need to update more than one variable in the gradient descent loop\n",
    "\n",
    "2) We need to do some pre-processing of the data, which we alluded to in an earlier exercise. We will cover how and why in later learning material. \n",
    "\n",
    "## Summary\n",
    "\n",
    "Well done! In this unit we have:\n",
    "\n",
    "1) watched gradient descent in action\n",
    "\n",
    "2) seen how changing the learning rate can improve a model's training speed\n",
    "\n",
    "3) learned that changing the learning rate can also result in unstable models \n",
    "\n",
    "You might have noticed that where the cost function stopped and the optimiser began became a little blurred here. Don't let that bother you. This is happens commonly, simply because, while they are conceptually separate, their mathematics sometimes can become intertwined."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "name": "python3710jvsc74a57bd0a970c7e51787c6ef7f3c805442fbb5a30ae2745635e1d2514e62d936ba85c55c",
   "display_name": "Python 3.7.10 64-bit ('ml_crash_course': conda)"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}