linear-regression.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "b2180960",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from sklearn.linear_model import LinearRegression\n",
    "\n",
    "import statsmodels.api as sm\n",
    "from statsmodels.stats.diagnostic import het_white"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "131dfbdf",
   "metadata": {},
   "source": [
    "Notes: \n",
    "- predictor = independent variable = explanatory variable = factor = feature\n",
    "- dependent variable = output = $Y$ = variable that *depends* on others"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6af5ce83",
   "metadata": {},
   "source": [
    "- What does that mean when residuals are gaussian/sub gaussian/other?\n",
    "- Can we have no residuals?\n",
    "- When do we have a unique solution? How to detect such cases?\n",
    "- When do we have collinearity? What is the problem with this? Can we illustrate the problem?\n",
    "- How to assess the accuracy of linear regression? is $R^2$ enough?\n",
    "- How to assess whether a WLS \"fit\" well?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09681401",
   "metadata": {},
   "source": [
    "[Generate data](#Generate-data-(univariate))\n",
    "\n",
    "[Univariate / multivariate](#Univariate-/-multivariate)\n",
    "\n",
    "[Estimator](#Estimator)\n",
    "\n",
    "[Coefficient of determination](#Coefficient-of-determination)\n",
    "\n",
    "[Intercept](#Intercept)\n",
    "\n",
    "[Assumptions](#Assumptions)\n",
    "\n",
    "- [Homoscedasticity](#Assumption-1:-errors-have-constant-variance-(homoscedasticity))\n",
    "\n",
    "- [Exogeneity](#Assumption-2:-no-correlation-between-the-error-and-the-exploratory-variables-(exogeneity))\n",
    "\n",
    "- [Error autocorrelation](#Assumption-3:-the-errors-are-uncorrelated)\n",
    "\n",
    "- Predictors are linearly independent\n",
    "\n",
    "- Errors are normally distributed\n",
    "\n",
    "[Predictions](#Predictions)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c91494f",
   "metadata": {},
   "source": [
    "### Generate data (univariate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "728b59c1",
   "metadata": {},
   "outputs": [],
   "source": [
    "n_observations = 100\n",
    "\n",
    "sigma_1 = 1\n",
    "sigma_2 = 0.3\n",
    "\n",
    "mu_1 = 3\n",
    "mu_2 = 1\n",
    "\n",
    "x = np.random.normal(mu_1, sigma_1, n_observations)\n",
    "y = np.random.normal(mu_2, sigma_2, n_observations)\n",
    "\n",
    "data = np.array([x, y]).T"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "261c1a59",
   "metadata": {},
   "source": [
    "We now rotate the data using a rotation matrix as explained here https://scipython.com/book/chapter-6-numpy/examples/creating-a-rotation-matrix-in-numpy/:\n",
    "\n",
    "$$R = \\begin{pmatrix}\n",
    "cos \\theta & -sin \\theta\\\\\n",
    "sin \\theta & cos \\theta\n",
    "\\end{pmatrix}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "83117035",
   "metadata": {},
   "outputs": [],
   "source": [
    "angle = 45\n",
    "theta = np.radians(angle)\n",
    "c, s = np.cos(theta), np.sin(theta)\n",
    "R = np.array(((c,-s),(s,c)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "bf91e623",
   "metadata": {},
   "outputs": [],
   "source": [
    "data_rotated = (R @ data.T).T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "eb9ad228",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAbbElEQVR4nO3df4xc1XUH8O/Z2Vk8xsRDlA3EgwlIjYZCHLzFCqT+o2MXBScQsnGaAkqiSIm0ldpEsUWdrEXUQEWFK9QkFY3UWkkaVbGI3QAbgtO6RGaKgmISzI4xjllAIdgeCBDh57LZxZ6ZPf1j9+3OzL735r15d37d+X4kFHb3zX33ZpYzd887915RVRARkT0GOt0BIiIyi4GdiMgyDOxERJZhYCcisgwDOxGRZRjYiYgsM2iiERH5LYC3AFQAlFV1g4l2iYgoOiOBfcEmVf29wfaIiKgJTMUQEVlGTKw8FZGXAJwGoAD+TVV3e1wzBmAMAFasWHHNpZdeGvu+3Wpubg4DA3Z+Zp48eRKqClvfP5vfO4Dj63XPP//871V1uNF1pgL7GlV9RUTeDeBRAF9S1cf9rs9mszo1NRX7vt0qn88jl8t1uhstkcvl4DgOCoVCp7vSEja/dwDH1+tE5HCYZ5hGPtpU9ZWF/30dwEMAPmiiXSIiii52YBeR80XkAvffAXwYwLNx2yUiouaYmLFfBODnInIEwC8B7FfV/zbQLhFRKBOTRWzcdRBHi2ewcddBTEwWO92ljopd7qiqvwFwtYG+EBFFNjFZxM4Hj2K2VAHWAkVnFjsfPAoAGB3JdLh3nWHv42Mi6gv3HpiaD+pVZksV3HvA3gKNRhjYiainveLMRvp+PzC58pSIqO3WpFMoegTxNemU72smJou498AUXnFmsSadwo4bslalbThjJ6KetuOGLFLJRM33UskEdtyQ9bzezckXnVkolnLyNj1wZWAnop42OpLBPVvXIbMwQ8+kU7hn6zrfGXg/5OSZiiGinjc6ksHoSAb5fB5f+nQu8Np+yMlzxk5EfcUv9x6Uk+81DOxE1Fei5uR7EVMxRNRX3Ny7zVUxDOxE1HfcnLytmIohIrIMAzsRkWUY2ImILMPATkRkGQZ2IiLLMLATEVmGgZ2IyDIM7ERElmFgJyKyDFeeElFXsv0wjFZiYCeirlNzQDVqD6gG7N7nxQQGdiLqOn6HYdz58DGcLc95BnwG9yXGcuwikhCRSRF5xFSbRNSf/A69cGZL1p9+ZILJh6dfBnDcYHtE1KeiHnrhdZh1PzMS2EXkEgA3AviOifaIqL/5HYZx4cqk5/UCWHUYdVymcuzfAvAVABf4XSAiYwDGAGB4eBj5fN7QrbvP9PS0teNzHAeVSsXa8dn83gG9M740gHv+NIHXzpRwrjKHocQALlo9BAA4+WbZ8zWvTT2NwVXSE+NrtdiBXURuAvC6qh4WkZzfdaq6G8BuAMhms5rL+V7a8/L5PGwdXzqdhuM41o7P5vcOaO/4WlWueNn4fs/vC4B/33K+1e9fWCZSMRsB3CwivwXwQwCbReQHBtoloh7llisWnVkolqpXTKRLMn1wGHVcsWfsqroTwE4AWJix/62qfiZuu0TUvRrNxv3KFe89MBV51l5/r01XDOOBw8Wa9hcPoz7zQryBWYJbChBRJGFm437lin7fj3KvBw4X8clrMsikUxDMz+Dv2bqOdexVjC5QUtU8gLzJNomou4SZja9JpzxLEKOmS/zu9dhzb+CJ8c0Re94/OGMnokjCzMb9yhV33JA1fi9ajlsKEHVYr212FWY27vY/7rhMzfz7DQM7UQcFbXbVrcF9xw3Zmj4D3rPx0ZFM7DGEvRfVYmAn6qBG+epunM2bmo13271swsBO1AFuwPbb4+QVZzb0bD5M8HevuXXtW7hj18HYwdHEbLwb72ULBnaiNqsP2F7WpFOhqk/CBP+aa9YuXfPUy2/isefe4EzYQqyKIWozr4Bdzc0hh6kICQr+ja7Zc+hES1aGUucxsBO1WVCpXvViG7/Kj+rvhwn+ftdo3dfc19weDOxEbeYXsDPpFJ4Y37yYDglTC+7X1oDI4uw77bPVrRfWh9uBgZ2ozcIu3hkdyeCeresCl857tQUAFVXsfPAovjZxFNNve29z6yVuffjEZBEbdx3E5eP7sXHXQaZ2OoQPT4naLEoJX6OKEPdnt+87gorWJldmSxXc/+TJZd8HgFRyAIAYrQ/vxZp8WzGwE3WAyRK+0ZEMtu8teP7MK6gDwNulOXzzlvVG68P9HtLevu/IYj+pPRjYiTqo2QVI1a9bnUpCBPCK4QkRz+C+Jp0yXh/ul59300IAg3u7MMdOFJLp/HGzh1HUv86ZLWHOI6inkgncdu1aI5txhRGUn2fFTXsxsBOF0IoTgcLUoId9Xb0BAc4bHMCeQyewIjmAdCoJATCUGKh5AGvyw8rvQa6LFTftw8BOFEKzQThIs1vShgmQczo/k1cAp2dKOFuez6lnL75g2YrU6g+r7XsLuKzJIO9W8SREPH/OHRnbh4GdKIRW7AseZgFSMz/34vUh5PVh5WZ0mv2LZHQkg3/6y6vblv4hbwzsRCH4BVMFmk5hRDmMwk2ZXDa+H6+eae7DpP5DqNGHUrN/kYSpv6fWYlUMUQhe+4K73BTGUy+/ibtH1wEIV+0Stp59YrKIHT86glJlfj5d/6A0lRzAimQCzkwJa9Ip/OFsGc5saVk/6z+c/A6xqNbsXyTckbGzGNiJQqgOwl7BUAHsOXQCG977TgBYtlBn294C7vrJMXz9Y1fVBLygANhoa1/XeYMJrBwahDMzH8xvuvo9eOBw0Xvx0ZkXFr8X9GHlYl68NzEVQ32vujJk6ndv+aZVRkcyeGJ8M7wfDc4H9zsfPobb9x3xDJanZ0qh89bVDzYbcWZLNQ9AHzhcxCevyTRMhVSnTAAsGxfz4r2LM3bqGa04Tah+Gfy5ylzDxTRBKQyvFEg1r/3UvcYUpqQx6B6PPfcGnhjf3PDa6r8YuvG0JmpO7MAuIisAPA7gvIX2fqSqX4/bLlG1Vu1DEuYwi3o7bshi+97Csm1vw3Lz1kFjilvz3czrmRe3h4lUzFkAm1X1agDrAWwRkesMtEu0qBV15EBzZYyjIxl8+rpLfVMyjbh5a78xbdtbwIBPLXjUe1B/ij1jV1UFML3wZXLhn2YnM0Se4tSRB6UY/NIqjQLj3aPrsOG976xpd+ZcGadnglMxgvmZ+cZdBwPz536bd4XB3DiJxvgFWmxEJAHgMIA/AvBtVf2qxzVjAMYAYHh4+Jp9+/bFvm+3mp6exqpVqzrdjZbYtm0bKpUK7rvvvrbed+p3b+FcZW7Z94cSA8hefIHv65zZEoqnZzFX9Xs+IILMhSmkU8llP78oBbzx9tLPg9p97czbOFeZw1BiABetXgEAy+4lIhgQoOK1mUsIAoFCF//Xz+CAoDyni33x67vNv5uA/ePbtGnTYVXd0Og6I4F9sTGRNICHAHxJVZ/1uy6bzerUlL0bAuXzeeRyuU53oyVyuRwcx0GhUGjrfb0OgE4lEw0XvvjNjN3Tity23Zn3+Po5XJT9k8A2g/oCeNelN5qhB/nWLesD27hwZRKTf/fhUG3Z/LsJ2D8+EQkV2I1WxaiqIyJ5AFsA+AZ2oqiiHE5RLUwKp/qhYT6fR65Bm0H5/uqj7cL0I4ztewvYtreAdCqJZEIWFyoB8x8oX//YVU23TXYyURUzDKC0ENRTAK4H8I+xe0ZUp5mqDb8c+oAILh/f31RZXzP5fr9++O2XXs39qTNbQnJAcOHK5OIq0+q+s1yRXCaqYt4D4DEReQbArwA8qqqPGGiXKLagM0Gb3X63mc27/PaF8dovPUhpTrFyaBAv7bqx5q+DVmwrTL0rdmBX1WdUdURVP6Cq71fVvzfRMSIT6jek8tpSNmrZZJTNu7z6gYV+uAuJqleJ+m15W83rL4M7Hz7mmR666yfHQoyIbMOVp9TzGqUgqlM4l4/v92wjSg682Xy/+/P6RUkPHC4uPgT2ejBbr/4vg4nJou+K19MzJUxMFpmS6TMM7NTTGq1IrQ/6qxdKHOu5wdKZLWHjroMNA3azqzT9ZtbuStf6zcYEtYtCquvgq7cfCBK0ipbsxMBOPa3RitT6oJ9MCJIDglJVXXlyQDBzrozLxvfj9nVlFJ3BxetNHsIcNLMuOrM1D3PrSzHrg3yU7Qd4JF3/4e6O1NOCKlS8gn6poli1YnAxp51OJQGB74pRk4cwN2rH66Gnu6NkJp1atjzJ7VsrTlyi3sbATj0tqELFL+g7MyU8Mb4ZL+26EeefN1hTF+7F1Iw3bDteHyZBH2BBh0hze4H+xMBOPWtisoiZc+Vl33eDWZiyxDDB1tSMN0o79f0KGotXxQ3AI+n6GXPs1JP8qkfSqSTuvPkqzwoUYPkMttHxcFFnvEEVOl4nFtU/HK3uVzWv11b3jVvuUjXO2Kkn+R1Ecf55g4sBLsyhyo2CdpQZb6NFQm5/qjfoWjmUWPYfYXJAlvWLB0RTFJyxU08Ks6w/7IHSdz58zLNaJbOQ5ggr7KEdZ8tLu1T+4dzyD6fSnOKpl9/07CsDOYXBGTv1pEb58yhL7O+8+arIK0m9hPmwCXvk3Z5DJ7gdADWNgZ16UqNl/VFOXKp/+NhsmsPUw1pgPu9uqsyS+g8DO/UkNxhfuHIpX33e4NKvc9QdGN168XWZ1b5b7zYSZg+ZKJUxze7fTsTATj3t7dJSvtqZLS2mW5rZgTGu+gec6VQSK5ID2L63gI27DmJishhYc14vzIZgRF4Y2KlnBaVbmtmB0QR35v/NW9bjbHkOp2dKNTl+AA13m3TFOfeU+hurYqhnBaVbmt2B0RX10Ir662fOlUOdsjQxWcT2vQXPWvZMC/+6ILsxsFPPCjodyd2qtplceaMdI8Nc76f+w2h0JIOnXn4Tew6dqAnu3AqA4mAqhnpW0OlIzZweNDFZxNTv3sK2vYXQFTVA+BJGwDvHf/foOnzzlvVcfETGcMZOPcsNfLfvO7IsH+21MCiIO+v+6yvm4Dffaeas02pBs3AuPiKTOGOnnjY6ksGcz0NGd49ztyIlSJhZd9RKm3QqyVk4dQRn7BTaxGQRkyccvPu8cs0JPp0WtJFXfUWKX38bzbqDZtt+G3RVb0ZG1E6csVMobqribLn2gWI3LHsPUxve6MCMoPr2RrNtbtBF3YYzdgol7AZXzYhSWhh0rft9v+pvd1bu1YY76waW9ndPJROhAzRz5NRNYgd2EVkL4D8AXAxgDsBuVf3nuO1Sd4n74NCPV6ngjv88grt+cgzOTKkmeDcqQ3QD68ZdBz1TM2vSKd827tm6DvdsXYfXpp6GLFzbLakmoqhMzNjLAG5X1adF5AIAh0XkUVX9tYG2qUv45bHjLtH3PJd0ThfPIK0O3mH/agg6lCKojSfGNyN/5gW8tCvX9HiiLmwiaoXYOXZVfVVVn17497cAHAfA32TLtGqJfpgZvxt4w/7VEJTzbtVfHkC0rYKJWslojl1ELgMwAuBJk+1S57mzzs/tSQAoI2NoNtroaDqXOwNutNK0ur9efWvURjpa92u08jkEURSihjYaEpFVAP4XwD+o6oMePx8DMAYAw8PD1+zbt8/IfbvR9PQ0Vq1a1elutMS2bdtQqVRw3333GWnPmS2heHrWtxbdNZQYwEWrV/heOyCCzIWpmmPnot5vQASZVYL0Oy6INogFR4tnfH+2LrO6qTZNs/l3E7B/fJs2bTqsqhsaXWcksItIEsAjAA6o6jcaXZ/NZnVqyt5DBPL5PHK5XKe70RK5XA6O46BQKDTdRn0eetMVw3jkyKuex9MBtdUpE5NFz5WmwHzK5YnxzaHu79fGzvVz+KtbP+bb16C/Uvwe2obtVzvY/LsJ2D8+EQkV2GPn2EVEAHwXwPEwQZ36m1ceeu8vT+IP58o117mb2Vbnx782cdQ3IAPh8+RBq1XPVZb2d4+aM+/UVsFE9UwsUNoI4LMANotIYeGfjxpolyzkVwVTqtQGWsXSTNcN6j84dCJwj/IoFTp+1w4llv6TiHK8HsCFStQ9Yj88VdWfY2mCRRQoSvVJ9bX3P3ky8NqoM2O/ksiLVg817GvQGLhQiboBV55SW4WtgnGvdQXN1L0qdBrlxv0O4kifeaFhX1t5vB6RCQzs1FZeM+XkgACCmnRM/Qw8IeIZ3BMiyx5Mhj0ow2t2nc8vBfaghU7N4OIlahduAkZt5ZWHvvdTV+Pev7g6MDd927VrPdvz+n7U3HiUvjabM+fiJWonztip7fzy0EEB8+7RdQDmc+0VVSREcNu1axe/X83k6lJTOXMuXqJ2YmCnnnH36DrPQF4vKDdenQ5Jr0xCFTgzu7TZWLoF/QZat4kakRemYsg6fvXkm64YrkmHnJ4pwZkt1aRG/BZJxRX19CWiOBjYqSkTk0Vs3HUw9NFz7eSXG3/suTcCj7+bLVXw2pm3W9InLl6idmIqhiIrz2moqpNO8sqNb99baPi66pWnpvsDLC+v7Jb/v8guDOwU2bnyHM7FfBBYnetenUpCBMsO1jAtTA199cpT07h4idqFqRiKzG/juLAPAr82cRTb9xYWc93ObAmnZ0otLwNsdDbq/MrTFcbvS9RunLFTZPP7vi0X5kHgxGQRew6d8D2XFGiuDDDM4p/6dIhnVUzVylOiXsXATpENDQ4gkUz4rshsdOB0mI2io5QBhl1p6n4d9IFRvfKUqFcxFUORDQ6I74rMRisswwbsKGWAplaaEtmCM3Zqit/Mt9EKyzAPMKOWAXLxD1EtztjJqEZB1u8B5vlDiab3Y+HiH6JanLGTUY22um1FPbfpXRiJeh0DOxkVJsiarufm4h+iWgzsZFSngiwX/xAtYWCnpgSVNDLIEnUWAztF1gt7xRD1M1bFUGTnynOsGyfqYgzsFFncvWKIqLWMBHYR+Z6IvC4iz5poj7pbnL1iiKj1TM3Yvw9gi6G2qMsNDQ7w0AiiLmYksKvq4wDeNNEWdb+gvWKIqPNYFUNNYUkjUfdqW2AXkTEAYwAwPDyMfD7frlu33fT0tLXjcxwHlUrF2vHZ/N4BHF+/aFtgV9XdAHYDQDab1Vwu165bt10+n4et40un03Acx9rx2fzeARxfv2C5IxGRZUyVO94P4BcAsiJySkS+YKJdIiKKzkgqRlVvM9EOERHFx1QMEZFlGNiJiCzDwE5EZBkGdiIiyzCwExFZhoGdiMgyDOxERJZhYCcisgwDOxGRZRjYiYgsw8BORGQZBnYiIsswsBMRWYaBnYjIMgzsRESWYWAnIrIMAzsRkWUY2ImILMPATkRkGQZ2IiLLMLATEVmGgZ2IyDIM7EREljES2EVki4hMiciLIjJuok0iImpO7MAuIgkA3wbwEQBXArhNRK6M2y4RETVn0EAbHwTwoqr+BgBE5IcAPg7g134vOHnyJHK5nIFbdyfHcZBOpzvdjZYoFAool8vWvn82v3cAx9cvTAT2DICTVV+fAnBt/UUiMgZgDACSySQcxzFw6+5UqVSsHV+5XIaqWjs+m987gOPrFyYCu3h8T5d9Q3U3gN0AkM1mtVAoGLh1d8rn89bOaHO5HBzHga3vn83vHcDx9ToRr3C7nImHp6cArK36+hIArxhol4iImmAisP8KwPtE5HIRGQJwK4CHDbRLRERNiJ2KUdWyiHwRwAEACQDfU9VjsXtGRERNMZFjh6r+FMBPTbRFRETxcOUpEZFlGNiJiCzDwE5EZBkGdiIiyzCwExFZhoGdiMgyDOxERJZhYCcisgwDOxGRZRjYiYgsw8BORGQZBnYiIsswsBMRWYaBnYjIMgzsRESWYWAnIrIMAzsRkWUY2ImILMPATkRkGQZ2IiLLMLATEVmGgZ2IyDKxAruIfEpEjonInIhsMNUpIiJqXtwZ+7MAtgJ43EBfiIjIgME4L1bV4wAgImZ6Q0REscUK7FGIyBiAMQAYHh5GPp9v163bbnp62trxOY6DSqVi7fhsfu8Ajq9fNAzsIvIzABd7/OgOVf1x2Bup6m4AuwEgm81qLpcL+9Kek8/nYev40uk0HMexdnw2v3cAx9cvGgZ2Vb2+HR0hIiIzWO5IRGSZuOWOnxCRUwA+BGC/iBww0y0iImpW3KqYhwA8ZKgvRERkAFMxRESWYWAnIrIMAzsRkWUY2ImILMPATkRkGQZ2IiLLMLATEVmGgZ2IyDIM7ERElmFgJyKyDAM7EZFlGNiJiCzDwE5EZBkGdiIiyzCwExFZhoGdiMgyDOxERJZhYCcisgwDOxGRZRjYiYgsw8BORGQZBnYiIsvECuwicq+IPCciz4jIQyKSNtQvIiJqUtwZ+6MA3q+qHwDwPICd8btERERxxArsqvo/qlpe+PIQgEvid4mIiOIYNNjW5wHs9fuhiIwBGFv48qyIPGvw3t3mXQB+3+lOtNC7RMTW8Vn/3oHj62XZMBeJqgZfIPIzABd7/OgOVf3xwjV3ANgAYKs2anD++qdUdUOYDvYijq932Tw2gOPrdWHH13DGrqrXN7jR5wDcBODPwwR1IiJqrVipGBHZAuCrAP5MVWfMdImIiOKIWxXzLwAuAPCoiBRE5F9Dvm53zPt2O46vd9k8NoDj63Whxtcwx05ERL2FK0+JiCzDwE5EZJmOBXabtyMQkU+JyDERmRMRa0qvRGSLiEyJyIsiMt7p/pgkIt8TkddtXV8hImtF5DEROb7wu/nlTvfJFBFZISK/FJEjC2O7q9N9agURSYjIpIg80ujaTs7Ybd6O4FkAWwE83umOmCIiCQDfBvARAFcCuE1Eruxsr4z6PoAtne5EC5UB3K6qfwzgOgB/Y9H7dxbAZlW9GsB6AFtE5LrOdqklvgzgeJgLOxbYbd6OQFWPq+pUp/th2AcBvKiqv1HVcwB+CODjHe6TMar6OIA3O92PVlHVV1X16YV/fwvzASLT2V6ZofOmF75MLvxjVVWIiFwC4EYA3wlzfbfk2D8P4L863QkKlAFwsurrU7AkMPQbEbkMwAiAJzvcFWMW0hQFAK8DeFRVrRnbgm8B+AqAuTAXm9wrZpkI2xGUAexpZV9MCzM2y4jH96yaFfUDEVkF4AEA21T1/zrdH1NUtQJg/cKzuodE5P2qasXzEhG5CcDrqnpYRHJhXtPSwG7zdgSNxmahUwDWVn19CYBXOtQXaoKIJDEf1Peo6oOd7k8rqKojInnMPy+xIrAD2AjgZhH5KIAVAN4hIj9Q1c/4vaCTVTHudgQ3czuCnvArAO8TkctFZAjArQAe7nCfKCQREQDfBXBcVb/R6f6YJCLDblWdiKQAXA/guY52yiBV3amql6jqZZj/7+5gUFAHOptjb3Y7gq4nIp8QkVMAPgRgv4gc6HSf4lp40P1FAAcw/+Btn6oe62yvzBGR+wH8AkBWRE6JyBc63SfDNgL4LIDNC/+9FRZmgDZ4D4DHROQZzE9AHlXVhiWBNuOWAkRElumWqhgiIjKEgZ2IyDIM7ERElmFgJyKyDAM7EZFlGNiJiCzDwE5EZJn/B/2aKtp8RcmeAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(data_rotated[:,0], data_rotated[:,1])\n",
    "plt.xlim([-2, 4])\n",
    "plt.ylim([-2, 5])\n",
    "plt.axhline(y=0, color='k')\n",
    "plt.axvline(x=0, color='k')\n",
    "plt.grid()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "c317c09b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Data for univariate regression\n",
    "X = data_rotated[:,0].reshape(-1, 1)\n",
    "y = data_rotated[:,1]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a9430b37",
   "metadata": {},
   "source": [
    "### Univariate / multivariate"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bba0ee1b",
   "metadata": {},
   "source": [
    "--> Univariate regression or \"simple\" regression is when there is only one predictor.\n",
    "\n",
    "--> Multivariate regression or \"multiple\" regression is when there is more than one predictors."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90fb23c7",
   "metadata": {},
   "source": [
    "### Estimator"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4cf32000",
   "metadata": {},
   "source": [
    "$$\\hat \\theta = (X^TX)^{-1}X^Ty$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "a5c80bb4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "**** Scikit-learn ****\n",
      "Intercept: 1.5070946888954442\n",
      "Coef: 0.9059593050419618\n",
      "**** Manual ****\n",
      "Intercept: 1.5070946888954406\n",
      "Coef: 0.9059593050419643\n"
     ]
    }
   ],
   "source": [
    "# estimator (scikit-learn)\n",
    "print('**** Scikit-learn ****')\n",
    "reg = LinearRegression().fit(X,y)\n",
    "print('Intercept: {}'.format(reg.intercept_))\n",
    "print('Coef: {}'.format(reg.coef_[0]))\n",
    "\n",
    "# estimator (manual)\n",
    "print('**** Manual ****')\n",
    "# add a new column for intercept\n",
    "X_ = np.zeros((n_observations, 2))\n",
    "X_[:,0] = X.flatten()\n",
    "X_[:,1] = 1\n",
    "X_gram = np.linalg.inv(np.dot(X_.T, X_))\n",
    "theta = np.dot(X_gram, np.dot(X_.T, y)) \n",
    "print('Intercept: {}'.format(theta[1]))\n",
    "print('Coef: {}'.format(theta[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "964ea956",
   "metadata": {},
   "source": [
    "### Coefficient of determination"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a7707861",
   "metadata": {},
   "source": [
    "$$R^2 = 1-\\frac{\\sum (y_i - \\hat y_i)^2}{\\sum (y_i - \\overline y)^2}$$\n",
    "\n",
    "We note that: \n",
    "\n",
    "$$R^2 = \\frac{\\sum (y_i - \\overline y)^2 - \\sum (y_i - \\hat y_i)^2}{\\sum (y_i - \\overline y)^2} = \\frac{\\text{variance explained by the regression}}{\\text{variance of }y}$$\n",
    "\n",
    "We also have: $$R^2 = r_{y,\\hat y}$$\n",
    "\n",
    "Since the coefficient of determination also applies in the multivariate case, the metric is actually a more general use of the correlation coefficient."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "275542db",
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = reg.intercept_ + reg.coef_*X.flatten()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "f6b2f6fe",
   "metadata": {},
   "outputs": [],
   "source": [
    "RSS = ((y - y_pred)**2).sum()\n",
    "TSS = ((y - y.mean())**2).sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "7547c7d3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7159013201761292"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "R_2 = 1 - RSS/TSS\n",
    "R_2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "ba89af04",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7159013201761294"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.corrcoef(y_pred, y)[0][1]**2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "05883de4",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7159013201761292"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reg.score(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "860fc55b",
   "metadata": {},
   "source": [
    "Problems with $R2$:\n",
    "\n",
    "- It's a performance metric assessed on the *training data*. Thus it doesn't tell whether the algorithm generalizes well.\n",
    "- $R2$ tends to drop when the variance is high. However, even if the variance is high, there can still be a strong linear relationship between the data.\n",
    "- More generally, high $R2$ is not necessarily good and low $R2$ is not necessarily bad. Below is an example where $R2$ is very high for a non-linear relationship."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "ce08e7f2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "R2: 0.25\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD4CAYAAADlwTGnAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAaxUlEQVR4nO3df4zc9X3n8ecrm7VqXFqTePODsYl9qWVkZGC5PcPVUVqiA9tUnDckEua4RG0jWb4r1wP1rJi7ih8XJBxxUtKTSJGFqBQ1xNDEtjbFwUR1pahwpB7HBmOCI59DYHdzYvmxSSjbsjbv+2NmzXg9s/uZ9cx+vzPf10Na7c73+/nsfmb89bzn+/nx/igiMDOz4vlA1g0wM7NsOACYmRWUA4CZWUE5AJiZFZQDgJlZQX0w6wbUs2TJkli+fHnWzTAz6xiHDh16PSL6mqmTywCwfPlyyuVy1s0wM+sYkn7ebB13AZmZFVRSAJC0QdJxSSckbZ+h3L+RdFrS55uta2Zm82vWACCpB3gQ2AisBm6RtLpBua8C+5uta2Zm8y/lDmAtcCIiTkbEu8AuYFOdcv8F+C7w2hzqmpnZPEsJACXg1ZrHw9VjZ0gqAZ8FHmq2bs3v2CKpLKk8NjaW0CwzMzsfKbOAVOfY9AxyXwe+HBGnpbOKp9StHIzYCewEGBgYaDpD3d7DIzyw/zij4xNcvHgh29avYrC/bqwxMzPSAsAwsKzm8VJgdFqZAWBX9c1/CXCDpFOJdc/b3sMjbPub55h8rxI3RsYn2PY3zwE4CJiZNZDSBXQQWClphaQFwGZgqLZARKyIiOURsRz4DvCfI2JvSt1WuGfo2Jk3/ymT7wX3DB1r9Z8yM+sas94BRMQpSbdRmd3TAzwSEcckba2en97vP2vd1jT9feMTk00dNzOzxJXAEbEP2DftWN03/oj4w9nqmplZ9rpiJfBFF/Q2ddzMzLokANx942X09pw94ai3R9x942UZtcjMLP+6IgAM9pd44PNXUFq8EFH55L9owQe547EjrNtxgL2HR7JuoplZ7nRFAIBKEHh6+2f42s1X8s+T7zE+MUlQmRJ65+6jDgJmZtN0TQCY8sD+40xMnj7r2MTkaR7YfzyjFpmZ5VPXBYDR8YmmjpuZFVXXBYCLFy+se/y3F3pGkJlZra4LANvWr6L3A+emIPqnd095HMDMrEbXBYDB/hK/+Rvnrm+bPB0eBzAzq9F1AQBg/J36KSBGPA5gZnZGVwaARuMAAH++9+g8tsTMLL+6MgBsW7+q4blv/+jVhufMzIqkKwPATHsAnI6m95oxM+tKXRkAAHpUbzOyxsfNzIqmawPALVcva+q4mVnRJO0H0InuG1wDVPr8T0fQI3HL1cvOHDczKzpFDvvEBwYGolwuZ90MM7OOIelQRAw0UyepC0jSBknHJZ2QtL3O+U2Snpd0RFJZ0qdqzr0s6ejUuWYaZ2Zm7TNrF5CkHuBB4DpgGDgoaSgiXqwp9nfAUESEpMuBx4FLa85fGxGvt7DdZmZdY+/hER7Yf5zR8QkuXryQbetXzTibsVVSxgDWAici4iSApF3AJuBMAIiIt2vKLwLy169kZpZDew+PcOfuo2fS2E/tYQIzT2lvhZQuoBJQu3pquHrsLJI+K+kl4Angj2tOBfCUpEOStjT6I5K2VLuPymNjY2mtNzPrcFnuYZJyB1Bv4vw5n/AjYg+wR9Knga8A/656al1EjEr6CPADSS9FxA/r1N8J7ITKIHDqE2hWVrdaZmb1ZLmHScodwDBQO3l+KTDaqHD1zf2TkpZUH49Wv78G7KHSpZSJqVutkfEJbxdpZrnQKHfZTDnNWiUlABwEVkpaIWkBsBkYqi0g6XekyhJbSVcBC4A3JC2SdGH1+CLgeuCFVj6BZni7SDPLm23rV7Gwt+esYwt7e2bMadYqs3YBRcQpSbcB+4Ee4JGIOCZpa/X8Q8DngC9KmgQmgJurM4I+SqVbaOpvPRoRT7bpucyq0S2V00SbWVamuqCz6Jou1EKwdTsO1H2zF/C1m6/0WICZday2LQTrFtvWr2o4ou1uIDMrmkIFgMH+UsMFCvMx4m5mlieFCgAApQxH3M3M8qRwASDLEXczszzp2nTQjWQ54m5mlieFCwBQCQJ+wzezoitkAJjO6SHMrIgKHwD2Hh5h23eeY/J0ZX7QyPgE277zHND+THxmZlkq3CDwdPd+79iZN/8pk6eDe793LKMWmZnNj8IHgLfemWzquJlZtyh8AJiJs4SaWTcrfABYvLC34TmnhzCzblb4AHDPv7+s4TmnhzCzblb4ADDYX+KiC+rfBTg9hJl1s8IHAIC7b7zM6SHMrHAKvw4AnB7CzIrJAaDK6SHMrGiSuoAkbZB0XNIJSdvrnN8k6XlJRySVJX0qta6ZmWVj1gAgqQd4ENgIrAZukbR6WrG/A66IiCuBPwYebqKumZllIOUOYC1wIiJORsS7wC5gU22BiHg73t9ceBGc2Xhr1rpmZpaNlABQAl6teTxcPXYWSZ+V9BLwBJW7gOS61fpbqt1H5bGxsZS2m5nZeUgJAI32UT/7QMSeiLgUGAS+0kzdav2dETEQEQN9fX0JzWq/vYdHWLfjACu2P8G6HQecGsLMukrKLKBhYFnN46XAaKPCEfFDSZ+UtKTZunmy9/AId+4+ysTkaaCSJvrO3UcBp4k2s+6QcgdwEFgpaYWkBcBmYKi2gKTfkaTqz1cBC4A3Uurm1QP7j595858yMXna+YHMrGvMegcQEack3QbsB3qARyLimKSt1fMPAZ8DvihpEpgAbq4OCtet26bn0lKN8gCNOD+QmXWJpIVgEbEP2Dft2EM1P38V+Gpq3U5w8eKFdd/sRaV7yN1AZtbpnAuogW3rVzUcwXY3kJl1AweABgb7S/WnK+E00WbWHRwAZlBqkA7aaaLNrBs4AMxg2/pVThNtZl3LAWAGg/0l7r9pDaXFCxGV7SN/o/cD3PHYES8MM7OO5wAwi8H+Ek9v/wxfu/lK/uXUe7z1ziTB+wvDHATMrFM5ACRqtDDsnqGOWNZgZnYOB4BEjWb+jE9M+i7AzDqSA0CimWb+eF2AmXUiB4BEM8388boAM+tEDgCJBvtLXHRBb91zXhdgZp3IAaAJd994mdcFmFnXSEoGZxVTCeAe2H+c0fEJLl68kG3rVzkxnJl1JAeAJg32l/yGb2ZdwV1AZmYF5QBgZlZQSQFA0gZJxyWdkLS9zvlbJT1f/XpG0hU1516WdFTSEUnlVjbezMzmbtYxAEk9wIPAdVQ2eT8oaSgiXqwp9jPg9yLiLUkbgZ3A1TXnr42I11vYbjMzO08pg8BrgRMRcRJA0i5gE3AmAETEMzXlnwWWtrKRnWDv4RHPDjKzjpLSBVQCXq15PFw91siXgO/XPA7gKUmHJG1pvon5t/fwCHfuPsrI+IQzhZpZx0gJAI22xj23oHQtlQDw5ZrD6yLiKmAj8CeSPt2g7hZJZUnlsbGxhGblR6NMoc4RZGZ5lhIAhoFlNY+XAqPTC0m6HHgY2BQRb0wdj4jR6vfXgD1UupTOERE7I2IgIgb6+vrSn0EONMoF5BxBZpZnKQHgILBS0gpJC4DNwFBtAUmXALuBL0TET2uOL5J04dTPwPXAC61qfF40ygXkHEFmlmezBoCIOAXcBuwHfgI8HhHHJG2VtLVa7C7gw8A3pk33/CjwD5KeA/4ReCIinmz5s8iY9w42s06kiLrd+ZkaGBiIcrmzlgx4FpCZZUnSoYgYaKaOcwG1yPQcQXsPj7BuxwEHBDPLLQeANpiaFjo1M2hqWijgIGBmueFcQG3gaaFm1gkcANrA00LNrBM4ALSBp4WaWSdwAGgDTws1s07gQeA28NaRZtYJvA5gHniNgJm1m9cB5JCnhJpZXnkMoM08JdTM8soBoM08JdTM8soBoM08JdTM8soBoM08JdTM8sqDwG3mKaFm+VbkWXoOAPNgeqZQM8uHos/ScxeQmRVW0WfpOQCYWWEVfZZeUgCQtEHScUknJG2vc/5WSc9Xv56RdEVqXTOzrBR9lt6sAUBSD/AgsBFYDdwiafW0Yj8Dfi8iLge+Auxsoq6ZWSaKPksvZRB4LXAiIk4CSNoFbAJenCoQEc/UlH8WWJpat+iKPAPBLGtFn6WXEgBKwKs1j4eBq2co/yXg+3OsWyhFn4FglgdFnqWXMgagOsfqphCVdC2VAPDlOdTdIqksqTw2NpbQrM5X9BkIZpatlAAwDCyrebwUGJ1eSNLlwMPApoh4o5m6ABGxMyIGImKgr68vpe0dr+gzEMwsWykB4CCwUtIKSQuAzcBQbQFJlwC7gS9ExE+bqVtkjWYaLL6gd55bYmZFNGsAiIhTwG3AfuAnwOMRcUzSVklbq8XuAj4MfEPSEUnlmeq24Xl0pG3rV9Hbc24v2dv/fIq9h0cyaJGZFYl3BMvYlfc+xfjE5DnHFy/s5cjd12fQIjPrRHPZEcwrgTP2yzpv/gDjE5O+CzCztnIAyNhMKw49G8jM2skBIGMzrTj0bCAzaycHgIwN9pe4qMGsn6LkIzGzbDgA5MDdN15W6HwkZpYNbwiTA/XykVx7aR8P7D/OHY8dKVx+EjObHw4AOVGbj8Q5gsxsPrgLKIecI8jM5oMDQA41mv0zMj7htQFm1jIOADk00+yfO3cfdRAws5ZwAMihersUTZmYPM09Q06nZGbnzwEghwb7S9x/05qG550mwsxawQEgpwb7S5ScJsLM2sgBIMecJsLM2skBIMecJsLM2skBIOecJsLM2sUrgXOuXpoIp4Uws1ZICgCSNgB/AfQAD0fEjmnnLwX+CrgK+B8R8b9qzr0M/Bo4DZxqdscaOztNBFRSRazbccABwczOy6wBQFIP8CBwHTAMHJQ0FBEv1hR7E/hTYLDBr7k2Il4/z7Ya9fME3fHYEco/f5P7BhtPHTUzmy5lDGAtcCIiTkbEu8AuYFNtgYh4LSIOAvX3N7SWqZcnKIBvPfuK1waYWVNSAkAJeLXm8XD1WKoAnpJ0SNKWRoUkbZFUllQeGxtr4tcXS6Ppn4HXBphZc1ICgOociyb+xrqIuArYCPyJpE/XKxQROyNiICIG+vr6mvj1xTLT9E+vDTCzZqQEgGFgWc3jpcBo6h+IiNHq99eAPVS6lGyOtq1fVTcig9cGmFlzUgLAQWClpBWSFgCbgaGUXy5pkaQLp34GrgdemGtjrTIj6NZrLjknCHhtgJk1a9ZZQBFxStJtwH4q00AfiYhjkrZWzz8k6WNAGfgt4D1JtwOrgSXAHklTf+vRiHiyLc+kQO4bXMPAJz7ktQFmdl4U0Ux3/vwYGBiIcrmcdTPMzDqGpEPNrrNyKggzs4JyADAzKygHADOzgnIAMDMrKGcD7VJ7D494lpCZzcgBoAvVSxh35+6jAA4CZnaGu4C6UL2EcROTp50ryMzO4gDQhRrlBHKuIDOr5QDQhRrlBArgz/cend/GmFluOQB0oW3rV52zj/CUv372FQcBMwMcALrSYH+J+29qvDvYX3vzGDPDAaBrzTbb587dRx0EzArOAaCL9ajRzgGVWUH3DB2bx9aYWd44AHSxW65eNuP58YlJ3wWYFZgDQBe7b3AN//GaS2Ys47UBZsXlANDl7htcw9dvvrLh+ZHxCT555z7PDDIrIAeAAhjsL3HRBb0Nz5+O8PRQswJKCgCSNkg6LumEpO11zl8q6f9I+hdJ/62ZujY/7r7xsoZrA6Y8+qNX5qk1ZpYHsyaDk9QDPAhcBwwDByUNRcSLNcXeBP4UGJxDXZsHU9NCH9h/nJEGKSHey9/uoGbWRil3AGuBExFxMiLeBXYBm2oLRMRrEXEQmGy2rs2fwf4ST2//TNbNMLOcSAkAJeDVmsfD1WMpkutK2iKpLKk8NjaW+OttLhb21v9nb3TczLpTyv/4equJUjsLkutGxM6IGIiIgb6+vsRfb3Nx/02Xn/MP/4HqcTMrjpQNYYaB2hVFS4HRxN9/PnWtTWrHA2p3DANYt+OAdxEzK4iUAHAQWClpBTACbAb+Q+LvP5+61kaD/aWz3tzr7SJ2x2NHKP/8Te4bbJxYzsw616wBICJOSboN2A/0AI9ExDFJW6vnH5L0MaAM/BbwnqTbgdUR8at6ddv0XOw81NtFLIBvPfsKA5/4kO8EzLpQ0p7AEbEP2Dft2EM1P/8/Kt07SXUtfxrtFhZUgoMDgFn38bQPAxrvIgbeStKsWzkAGFDZRaxR8uigMjjszKFm3cUBwIDKoPCt11zSMAhMDQo7X5BZ93AAsDPuG1zD126+ktIMm8p/y9tJmnUNBwA7y1S6iJm6g/7s8eccBMy6gAOA1TXToPDpCO8pbNYFHACsrpkGhaGyp7B3EzPrbA4AVtdsg8Lg6aFmnS5pIZgV032Daxj4xIf4s8ef43Scm8Nvpm4iK4a9h0fOySnlRYOdwwHAZjT1n7k2TxDAwt6eMwnkrJjq5Y+6c3dlmrCDQGdwALBZOXuo1VMvf9TU2JCvg87gAGBJUrKH+tNfsTQaA/LYUOfwILDNyUyf/qwYGo0BeWyoczgA2Jz4059tW7+Khb09Zx3z2FBncQCwOfGnPxvsL3H/TWsoLV6IgNLihdx/0xp3AXYQRZ3pfVkbGBiIcrmcdTNsBtPHAKDy6e9z/7rE37805oFhs3km6VBEDDRTx4PANif1ZgZde2kf3z004oFhsw6RFAAkbQD+gsq2jg9HxI5p51U9fwPwDvCHEfHj6rmXgV8Dp4FTzUYoy6/pM4PW7TjgaYFmHWTWACCpB3gQuA4YBg5KGoqIF2uKbQRWVr+uBv6y+n3KtRHxestabbnkgWGzzpIyCLwWOBERJyPiXWAXsGlamU3AN6PiWWCxpI+3uK2Wcx4YNussKQGgBLxa83i4eiy1TABPSTokaUujPyJpi6SypPLY2FhCsyxvPC3QrLOkjAHUSwg5ferQTGXWRcSopI8AP5D0UkT88JzCETuBnVCZBZTQLsuZRikj3P9vlk8pAWAYWFbzeCkwmlomIqa+vyZpD5UupXMCgHWH6QPDZpZfKV1AB4GVklZIWgBsBoamlRkCvqiKa4BfRsQvJC2SdCGApEXA9cALLWy/mZnN0ax3ABFxStJtwH4q00AfiYhjkrZWzz8E7KMyBfQElWmgf1St/lFgT2WWKB8EHo2IJ1v+LMzMrGleCWxm1gXmshLYuYDMzArKqSAsF7y1oNn8cwCwzHlzmfZzgLV63AVkmfPmMu01FWBHxicI3g+wew+PZN00y5gDgGWuUa6gkfEJVmx/gnU7DvjN6jw4wFojDgCWuZlyBU19Yr39sSNcee9TDgRz4CR91ogDgGWuXg6hesYnJrn9sSP0/08HgmY4SZ814gBgmZu+teBs3npn0n3YTXCSPmvEAcByYbC/xNPbP8PPdvwBpYRPpu7DTue9e60RTwO13Nm2ftU5+w3X4z7sdE7SZ/U4AFjuTL1R3fu9Y7z1zmTDcu7DNjs/7gKyXBrsL3H4ruv5+s1Xsnhh7znn3Ydtdv58B2C5NtV10cxKVq96NUvjAGAdIbUP22klzNK5C8i6ile9mqVzALCu4lWvZukcAKyreNWrWbqkACBpg6Tjkk5I2l7nvCT97+r55yVdlVrXrJW86tUs3awBQFIP8CCwEVgN3CJp9bRiG4GV1a8twF82UdesZbzq1SxdyiygtcCJiDgJIGkXsAl4sabMJuCbUdlg+FlJiyV9HFieUNespbzq1SxNShdQCXi15vFw9VhKmZS6AEjaIqksqTw2NpbQLDMzOx8pAaBegsZILJNSt3IwYmdEDETEQF9fX0KzzMzsfKR0AQ0Dy2oeLwVGE8ssSKhrZmYZSLkDOAislLRC0gJgMzA0rcwQ8MXqbKBrgF9GxC8S65qZWQZmvQOIiFOSbgP2Az3AIxFxTNLW6vmHgH3ADcAJ4B3gj2aq25ZnYmZmTVFl4k6+SBoDfj6HqkuA11vcnFbKc/vctrnLc/vctrnJc9ugfvs+ERFNDaDmMgDMlaRyRAxk3Y5G8tw+t23u8tw+t21u8tw2aF37nArCzKygHADMzAqq2wLAzqwbMIs8t89tm7s8t89tm5s8tw1a1L6uGgMwM7N03XYHYGZmiRwAzMwKqmMCQJ73JEho263VNj0v6RlJV9Sce1nSUUlHJJUzaNvvS/pl9e8fkXRXat15at+2mra9IOm0pA9Vz7X7tXtE0muSXmhwPstrbra2ZXnNzda2zK65hLZleb0tk/T3kn4i6Zik/1qnTGuvuYjI/ReVVcT/F/hXVPILPQesnlbmBuD7VBLQXQP8KLXuPLTtd4GLqj9vnGpb9fHLwJIMX7ffB/52LnXno33Tyt8IHJiP1676+z8NXAW80OB8JtdcYtsyueYS25blNTdj2zK+3j4OXFX9+ULgp+1+n+uUO4AzexJExLvA1L4Ctc7sSRARzwJTexKk1G1r2yLimYh4q/rwWSpJ8ebD+Tz3dr9uc/kbtwDfbnEbGoqIHwJvzlAkq2tu1rZleM2lvG6NZP66TTPf19svIuLH1Z9/DfyEc9Pnt/Sa65QAMC97ErSxbbW+RCWCTwngKUmHJG1pYbuaadu/lfScpO9LuqzJuvPRPiRdAGwAvltzuJ2vXYqsrrlmzec1lyqray5J1tebpOVAP/Cjaadaes2lpIPOg3nZk2COkn+/pGup/Gf8VM3hdRExKukjwA8kvVT9lDJfbfsxlRwib0u6AdhLZWvPdr9uNPk3bgSejojaT2/tfO1SZHXNJcvgmkuR5TWXKrPrTdJvUgk8t0fEr6afrlNlztdcp9wBnM+eBCl12902JF0OPAxsiog3po5HxGj1+2vAHiq3cvPWtoj4VUS8Xf15H9AraUlK3floX43NTLsdb/NrlyKray5JRtfcrDK+5lJlcr1J6qXy5v+tiNhdp0hrr7l2DWi08ovKncpJYAXvD3BcNq3MH3D24Mg/ptadh7ZdQiVV9u9OO74IuLDm52eADfPcto/x/oLAtcAr1dewra9bM/82wG9T6bddNF+vXc3fWU7jwcxMrrnEtmVyzSW2LbNrbra2ZXm9VV+DbwJfn6FMS6+5jugCihzvSZDYtruADwPfkARwKiqZ/D4K7Kke+yDwaEQ8Oc9t+zzwnySdAiaAzVG5otq+l0Ni+wA+CzwVEf9UU72trx2ApG9TmbGyRNIwcDfQW9O2zPbBSGhbJtdcYtsyu+YS2gYZXW/AOuALwFFJR6rH/juVYN6Wa86pIMzMCqpTxgDMzKzFHADMzArKAcDMrKAcAMzMCsoBwMysoBwAzMwKygHAzKyg/j812CmZB+lRkAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x2 = np.random.exponential(.4,50)\n",
    "y2 = ((x2-1)**2)*np.random.uniform(low=0.3, high=1.2)\n",
    "plt.scatter(x2, y2)\n",
    "\n",
    "reg2 = LinearRegression().fit(x2.reshape(-1, 1),y2)\n",
    "print('R2: {}'.format(round(reg2.score(x2.reshape(-1, 1),y2),2)))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "01b3b6e7",
   "metadata": {},
   "source": [
    "### Intercept"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "9da7078c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.3692197187214836"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.mean(X.flatten())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8346d668",
   "metadata": {},
   "source": [
    "$X$ is not centered."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "d6f0c3db",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAft0lEQVR4nO3dfXAc5X0H8O9Pp5N9xsZHG9XEwhRm2lyaYLADDSSeCWeXKW5JqOqUBqYv6ZQZhWmTYuqQiHEmIR23dsc0CaVpEk9K0k5cYpcXheC0rql90JKQBCM5xgVBhjTYcngbfIplCet0+vWP1Un3sru3d/fc7e1z38+MB2u12nsWyd979NvnRVQVRERkj66wG0BERGYx2ImILMNgJyKyDIOdiMgyDHYiIssw2ImILNNt4iIi8n8ATgPIA5hR1StMXJeIiGpnJNjnrFfV1w1ej4iI6sBSDBGRZcTEzFMR+QmAUwAUwFdUdZfLOQMABgBg8eLFl1944YUNv267mp2dRVeXne+Zx48fh6rC1u+fzd87gPcXdc8///zrqtpb7TxTwb5SVU+KyC8BOADgY6r6uNf5qVRKR0dHG37ddpXJZJBOp8NuRlOk02lks1mMjIyE3ZSmsPl7B/D+ok5EDgd5hmnkrU1VT87991UADwF4t4nrEhFR7RoOdhE5R0SWFf4O4DcBPNPodYmIqD4meuwrAPyPiBwB8AMA+1T1Pwxcl4gokKHhMazbcRBHx8axbsdBDA2Phd2kUDU83FFVXwRwmYG2EBHVbGh4DHc8eBRTuTywChjLTuGOB48CAPrX9oXcunDY+/iYiDrCzv2jTqgXmcrlsXO/vQM0qmGwE1GkncxO1XS8E5iceUpE1HIrkwmMuYT4ymTC82uGhsewc/8oTmansDKZwO3Xpqwq27DHTkSRdvu1KSTisZJjiXgMt1+bcj2/UJMfy05BsVCTt+mBK4OdiCKtf20ftm9ajb65HnpfMoHtm1Z79sA7oSbPUgwRRV7/2j70r+1DJpPBx/4g7XtuJ9Tk2WMnoo7iVXv3q8lHDYOdiDpKrTX5KGIphog6SqH2bvOoGAY7EXWcQk3eVizFEBFZhsFORGQZBjsRkWUY7ERElmGwExFZhsFORGQZBjsRkWUY7ERElmGwExFZhjNPiagt2b4ZRjMx2Imo7ZRsUI3SDaoBu9d5MYHBTkRtx2szjDsfPoazM7Ougc9wX2Csxi4iMREZFpFHTF2TiDqT16YX2amc9bsfmWDy4emtAJ41eD0i6lC1bnrhtpl1JzMS7CJyAYDrAHzVxPWIqLN5bYZx3pK46/kCWLUZdaNM1di/AOATAJZ5nSAiAwAGAKC3txeZTMbQS7efiYkJa+8vm80in89be382f++A6NxfEsD298bwyngO0/lZ9MS6sGJ5DwDg+Bszrl/zyujT6F4qkbi/Zms42EXk/QBeVdXDIpL2Ok9VdwHYBQCpVErTac9TIy+TycDW+0smk8hms9ben83fO6C199es4YoXDe5zPS4AvrbxHKu/f0GZKMWsA3C9iPwfgG8C2CAi3zBwXSKKqMJwxbHsFBQLo1dMlEv6OmAz6kY13GNX1TsA3AEAcz32j6vqHzZ6XSJqX9V6417DFXfuH625117+Wuvf3osHDo+VXH9+M+rxFxq7MUtwSQEiqkmQ3rjXcEWv47W81gOHx/DBy/vQl0xA4PTgt29azXHsRYxOUFLVDICMyWsSUXsJ0htfmUy4DkGstVzi9VqHnnsNTwxuqLHlnYM9diKqSZDeuNdwxduvTRl/LarEJQWIQha1xa6C9MYL7W/0vkz1/DsNg50oRH6LXbVruN9+baqkzYB7b7x/bV/D9xD0tagUg50oRNXq1e3YmzfVG2+317IJg50oBIXA9lrj5GR2KnBvPkj4F865cdVpbN1xsOFwNNEbb8fXsgWDnajFygPbzcpkItDokyDhX3LOqoVznvrpGzj03GvsCVuIo2KIWswtsIsVashBRoT4hX+1c3Y/+VJTZoZS+BjsRC3mN1SveLKN18iP4uNBwt/rHC37mOua24PBTtRiXoHdl0zgicEN8+WQIGPBva7VJTLf+056LHXrhuPD7cBgJ2qxoJN3+tf2Yfum1b5T592uBQB5Vdzx4FF8augoJt50X+bWTaPjw4eGx7Bux0FcPLgP63YcZGknJHx4StRitQzhqzYipPC5LXuPIK+lxZWpXB73ff94xXEASMS7AIjR8eFRHJNvKwY7UQhMDuHrX9uH2/aMuH7OLdQB4M3cLD7/oTVGx4d7PaTdsvfIfDupNRjsRCGqdwJS8dctT8QhArhleEzENdxXJhPGx4d71ecLZSGA4d4qrLETBWS6flzvZhTlX5edymHWJdQT8RhuunKVkcW4gvCrz3PETWsx2IkCaMaOQEHGoAf9unJdAizq7sLuJ1/C4ngXkok4BEBPrKvkAazJNyuvB7kFHHHTOgx2ogDqDWE/9S5JGyQgZ9XpySuAU5M5nJ1xauqp85dVzEgtfrO6bc8ILqoz5AujeGIirp/nioytw2AnCqAZ64IHmYBUz+fduL0Jub1ZFSo69f5G0r+2D3/3+5e1rPxD7hjsRAF4hakCdZcwatmMolAyuWhwH342Xt+bSfmbULU3pXp/Iwky/p6ai6NiiAJwWxe8oFDCeOqnb2Bb/2oAwUa7BB3PPjQ8htvvP4Jc3ulPlz8oTcS7sDgeQ3Yyh5XJBM6cnUF2KlfRzvI3J69NLIrV+xsJV2QMF4OdKIDiEHYLQwWw+8mXcMUv/wIAVEzU2bxnBJ/99jF85gPvLAk8vwCstrRvwaLuGJb0dCM76YT5+y97Kx44POY++Wj8hfljfm9WBayLRxNLMdTxikeGjL582rOs0r+2D08MboD7o0En3O98+Bi27D3iGpanJnOB69bFDzaryU7lSh6APnB4DB+8vK9qKaS4ZAKg4r5YF48u9tgpMpqxm1D5NPjp/GzVyTR+JQy3Ekgxt/XU3e4pyJBGv9c49NxreGJwQ9Vzi39jaMfdmqg+DQe7iCwG8DiARXPXu19VP9PodYmKNWsdkiCbWZS7/doUbtszUrHsbVCFurXfPTU65ruer2dd3B4mSjFnAWxQ1csArAGwUUSuMnBdonnNGEcO1DeMsX9tH/7gqgs9SzLVFOrWXve0ec8IujzGgtf6GtSZGu6xq6oCmJj7MD73p97ODJGrRsaR+5UYvMoq1YJxW/9qXPHLv1By3cnpGZya9C/FCJye+bodB33r516LdwXB2jiJNvADNH8RkRiAwwB+BcAXVfWTLucMABgAgN7e3sv37t3b8Ou2q4mJCSxdujTsZjTF5s2bkc/ncc8997T0dUdfPo3p/GzF8Z5YF1LnL/P8uuxUDmOnpjBb9HPeJYK+8xJIJuIVn1+RAF57c+Hzftd9ZfxNTOdn0RPrworliwGg4rVEBF0C5N0WcwlAIFDo/H+9dHcJZmZ1vi1ebbf5ZxOw//7Wr19/WFWvqHaekWCfv5hIEsBDAD6mqs94nZdKpXR01N4FgTKZDNLpdNjNaIp0Oo1sNouRkZGWvq7bBtCJeKzqxBevnnFht6LCtQs978E1s1iRepfvNf3aAriPS6/WQ/fzhQ+t8b3GeUviGP70bwa6ls0/m4D99ycigYLd6KgYVc2KSAbARgCewU5Uq1o2pygWpIRT/NAwk8kgXeWafvX+4q3tgrQjiNv2jGDznhEkE3HEYzI/UQlw3lA+84F31n3tjjU+DnziE8CnPw302ffA2MSomF4AublQTwC4BsDfNtwyojL1jNrwqqF3ieDiwX11Deurp97v1Q6v9dKLFT6bncoh3iU4b0l8fpZpcds5XDGA06eB974XeGau37lsGXDXXeG2qQlMjIp5K4BDIvIjAD8EcEBVHzFwXaKG+e0JWu/yu/Us3uW1Lozbeul+crOKJT3d+MmO60p+O2jGssJWmZgA3vUu4NxzF0J9cBDYuTPcdjVJw8Guqj9S1bWqeqmqXqKqf2WiYUQmlC9I5bakbK3DJmtZvMutHZhrR2EiUfEsUa8lb4u5/WZw58PHXMtDn/32sQB3ZLHJSeDKK52e+fCwc+zjHwdmZ4Ht24EGh5W2K848pcirVoIoLuFcPLjP9Rq11MDrrfcXPl8+KemBw2PzD4HdHsyWK//NYGh4zHPG66nJHIaGxzqvJDM1BVxzDfDd7y4c27wZ+NznrA3zYgx2irRqM1LLQ3/53BDHcoWwzE7lsG7HwaqBXe8sTa+edWGma/liY4LSSSHF4+CLlx/w4zeL1jZd09PA1VcDjz++cPCjHwX+/u87ItALGOwUadVmpJaHfjwmiHcJckXjyuNdgsnpGVw0uA9bVs9gLNs9f77JTZj9etZj2amSh7nlQzHLQ76W5Qc6Yku6n/8cWL4c7ys+dsstwD/+Y0cFegFXd6RI8xuh4hb6ubxi6eLu+Zp2MhEHBJ4zRk1uwlztOm4PPQsrSvYlExXTkwpta8aOS5ExMeEE9/LlC8duvhnI54EvfakjQx1gsFPE+Y1Q8Qr97GQOTwxuwE92XIdzFnWXjAt3Y6rHG/Q6bm8mfm9gfptIW7u8wPi4E9rLSmcdP/boo8BXvwp0dXa0dfbdU6QNDY9hcnqm4nghzIIMSwwStqZ6vLVcp7xdfvfiNuIGsHRLumzWCfRksvT42bOAKjQWfOiozVhjp0jyGj2STMRx5/XvdB2BAlT2YKttD1drj9dvhI7bjkXlD0eL21XM7WuL22b9krunTztj0MudOQMsWdL69rQ59tgpkrw2ojhnUfd8wAXZVLlaaNfS4602SajQnuIFupb0xCr+Eca7pKJdHbtB9JkzTg+9PNRPnQJUGeoe2GOnSAoyrT/ohtJ3PnzMdbRK31yZI6igm3acnVlYpfLMdOWbU25W8dRP33Btq/VBXjA5CZxzTuXx118HfvEXW9+eiGGPnSKpWv28lin2d17/zppnkroJ8mYTdMu73U++1JnLAbz5ptNDLw/1l192eugM9UAY7BRJ1ab117LjUvnDx3rLHKYe1gJO3d3UMMtImJ52Aj1R9v/wxRedQF+xIpx2RRSDnSKpEMbnLVmoVy/qXvhxrnUFxsJ48dV9yz2X3q0myBoytYyMqXf99kiZmXECfdGi0uPPPecE+sUXh9OuiGOwU6S9mVuoV2encvPllnpWYGxU+QPOZCKOxfEu3LZnBOt2HMTQ8JjvmPNyQRYEi6x83gn0eNlOT0ePOoGesnDsfQsx2Cmy/Mot9azAaEKh5//5D63B2ZlZnJrMldT4AVRdbbKgkX1P25aqE+jdZeM2Dh92PnfJJeG0yzIcFUOR5VduqXcFxoJaN60oP39yeibQLktDw2O4bc+I61j2vib+dtFyqu6zQb/7XeA972l9eyzHYKfI8tsdqbBUbT218morRgY530v5m1H/2j489dM3sPvJl0rC3ZqlALwCPZNxVmGkpmAphiLLb3ekenYPGhoew+jLp7F5z0jgETVA8CGMgHuNf1v/anz+Q2vsm3wkUhnq+/c7Yc9Qbyr22CmyCsG3Ze+Rinq028QgP4Ve95+9fRZe/Z169jot5tcLt2rykdtzg299C7j++ta3pUOxx06R1r+2D7MeDxkLa5wXRqT4CdLrrnWkTTIRt68X7kekMtTvusvpoTPUW4o9dgpsaHgMwy9l8UuLZkp28Amb30Je5SNSvNpbrdft19v2WqCreDEyq7n10LduBbZta31bCAB77BRQoVRxdqb0gWI7THsPMja82oYZfuPbq/W2O3aBLrce+q23Oj10hnqo2GOnQIIucFWPWoYW+p1bOO41+rvQK3e7RqHXDSys756IxwIHtFU18mrceug33+xscEFtoeFgF5FVAP4FwPkAZgHsUtW7G70utZdGHxx6cRsqePu/HcFnv30M2clcSXhXG4ZYCNZ1Ow66lmZWJhOe19i+aTW2b1qNV0afhsyd2y6lprbhFuhXX+0MXaS2YqLHPgNgi6o+LSLLABwWkQOq+r8Grk1twquO3egUfdd9SWd1fg/S4vAO+luD36YUftd4YnADMuMv4Cc70nXfT60TmyLBLdAvvRQ4cqT1baFAGq6xq+rPVPXpub+fBvAsgIj/JFO5Zk3RD9LjLwRv0N8a/GrezfrNA6htqeBIcKuhX3ihU0NnqLc1ozV2EbkIwFoA3zd5XQpfodf54d0xADPoM9QbrbY1XUGhB1xtpmlxe93aVu0aydqaX6KZzyFayq2Hfu65zgbSFAmihhYaEpGlAB4D8Neq+qDL5wcADABAb2/v5Xv37jXyuu1oYmICS5cuDbsZTbF582bk83ncc889Rq6Xncph7NSU51j0gp5YF1YsX+x5bpcI+s5LlGw7V+vrdYmgb6kgee6y2m5iztEx7+Bb3be8rmua5vezmV6/3vV45tChZjbJKJv/7QHA+vXrD6vqFdXOMxLsIhIH8AiA/ar6uWrnp1IpHR21dxOBTCaDdDoddjOaIp1OI5vNYmRkpO5rlNeh17+9F48c+Znr9nRA6eiUoeEx15mmgFNyeWJwQ6DX97rGHWtm8ZEbP+DZVr/fUrwe2gZtVyu4/mx6rTAZwdUlbf63BwAiEijYG66xi4gA+CcAzwYJdepsbnXoPT84jjPTMyXnFaKmuD7+qaGjnoEMBK+T+81Wnc4vrO9ea808rKWC6+ZWQwecQI9gqNMCExOU1gH4IwAbRGRk7s9vG7guWchrFEwuXxokioWebiHUv/HkS75rlNcyQsfr3J7Ywj+JWrbXAyI0UYmBbr2GH56q6v9goYNF5KuW0SfF5973/eO+59baM/YaErlieU/VtvrdQ1tPVBJB2u04w9w6nHlKLRV0FEzh3AK/nrrbCJ1qtXGvjTiS4y9UbWszt9drCotq6BQMg51ayq2nHO8SQFBSjinvgcdEXMM9JlLxYDLoRhluvetMZiHY/SY61aPlk5cY6B2Li4BRS7nVoXfecBl2/t5lvrXpm65c5Xo9t+O11sZraWu9NfOWTl7yqaFHaegi1Y89dmo5rzq0X2Bu618NwKm151URE8FNV66aP17M5OxSUzXzlkxeYg+d5jDYKTK29a92DfJyfrXx4nJIckkcqsD41MJiY8kmtBto3iJqABjoVIGlGLKO13jy9W/vLSmHnJrMITuVKymNeE2SalStuy8FwmGL5IHBTnUZGh7Duh0HA28910petfFDz73mu/3dVC6PV8bfbEqbjE5eYqBTFSzFUM1mZjXQqJMwudXGb9szUvXrimeemm4PUDm8sqb/Xyy5UEAMdqrZ9Mwspht8EFhc616eiEMEFRtrmBZkDH3xzFPT6n4Qy0CnGrEUQzXzWjgu6IPATw0dxW17RuZr3dmpHE5N5po+DLDa3qjOzNPFxl+3biy5UJ3YY6eaiUcPMsiDwKHhMex+8iXPfUmB+oYBBpn8U14OcR0VUzTzNDTsoVODGOxUs57uLsTiMc8ZmdU2nA4ST7UMAww607Twsd8bRvHM05ZjoJMhLMVQzbq7xHNGZrUZlkEDu5ZhgKZmmoaGJRcyjD12qotXz7faDMsgDzBrHQbY1Mk/zcQeOjUJe+xkVLWQ9XqAeU5PrO71WJoy+aeZ2EOnJmOPnYyqttStkfHcZUyvwtg07KFTizDYyaggIWt6M4pmvFkYxUCnFmOwk1FhhWxb7lzEQKeQMNipLn5DGtsyZFuJgU4hY7BTzaKwVkwoGOjUJjgqhmo2PTMb7XHjpnGUC7UZ9tipZo2uFWMN9tCpTRnpsYvIvSLyqog8Y+J61N4aWSvGCuyhU5szVYr5OoCNhq5Fba6nu8vcphFRwkCniDAS7Kr6OIA3TFyL2p/fWjFWYqBTxLDGTnXpiCGNXjX02VnvzxG1gZYFu4gMABgAgN7eXmQymVa9dMtNTExYe3/ZbBb5fN7a+5uYmMB0Mome8fGKzz326KPQWAx47LEQWmaGzT+bgP33F1TLgl1VdwHYBQCpVErT6XSrXrrlMpkMbL2/ZDKJbDZr5/1dcw3wX/9VeXxqCli8GFe3vkXG2fyzCdh/f0FxHDvRzTc7pZXyUB8fd2roi9touzyiAEwNd7wPwPcApETkhIjcbOK6RE31yU86gX7vvaXHX3vNCfRzzw2nXUQNMlKKUdWbTFyHqCXuvhvYvLny+IkTyLzwAtJveUvLm0RkEksx1Dn+9V+dHnp5qD//vNND77N8lA91DA53JPt95zvAdddVHj9yBLj00ta3h6jJGOxkr//+b+B976s8/r3vAVdd1fr2ELUISzFkn8OHnZJLeagfOOCUXBjqZDn22MkeL7wAvO1tlcfvvx/44Adb3x6ikLDHTtF38qTTQy8P9S99yemhM9SpwzDYKbqyWSfQy0ez/M3fOIF+yy2hNIsobCzFUPScOQMsXVp5fMcOZ9IRUYdjsFN05HJAT0/l8W3bgK1bW98eojbFYKf2l88D3S4/qn/xF84sUiIqwRo7tS9Vp4ZeHurbtjmfY6gTuWKPndqPKtDl0ucYHAS2b299e4gihsFO7cVtZ6JbbnGGLhJRIAx2ag9ugf6RjwBf/nLr20IUcQx2CpdboN94I3Dffa1vC5El+PCUwiFSGep/+ZdOfZ2hTtQQ9tiptdx66Fu2AHfd1fq2EFmKPXZqDbce+tatTg+doU5kFHvs1FxuPfSvfx348Idb3hSiTsEeOzWHWw/9K19xeugMdaKmYrCTWW6BfvfdTqAPDITTJqIOw1IMmeFWcvnnfwb++I9b3xaiDsceOzXGrYf+ta85PXSGOlEojAS7iGwUkVER+bGIDJq4JrW5++/3rqH/yZ+E0iQicjQc7CISA/BFAL8F4B0AbhKRdzR6XWpTp04heeQIcMMNC8dYQydqKyZq7O8G8GNVfREAROSbAH4HwP96fcHx48eRTqcNvHR7ymazSCaTYTejKUaOHgUApAHg138dWLIEePBB548FbP7eAby/TmEi2PsAHC/6+ASAK8tPEpEBAAMAEI/Hkc1mDbx0e8rn89beXz6RgKoi+7a3AdPTzh+L2Py9A3h/ncJEsLsMh4BWHFDdBWAXAKRSKR0ZGTHw0u0pk8lY+xtJOp1GNpuFrd8/m793AO8v6sRt9JkLEw9PTwBYVfTxBQBOGrguERHVwUSw/xDAr4rIxSLSA+BGAA8buC4REdWh4VKMqs6IyEcB7AcQA3Cvqh5ruGVERFQXIzNPVfU7AL5j4lpERNQYzjwlIrIMg52IyDIMdiIiyzDYiYgsw2AnIrIMg52IyDIMdiIiyzDYiYgsw2AnIrIMg52IyDIMdiIiyzDYiYgsw2AnIrIMg52IyDIMdiIiyzDYiYgsw2AnIrIMg52IyDIMdiIiyzDYiYgsw2AnIrIMg52IyDINBbuI3CAix0RkVkSuMNUoIiKqX6M99mcAbALwuIG2EBGRAd2NfLGqPgsAImKmNURE1LCGgr0WIjIAYAAAent7kclkWvXSLTcxMWHt/WWzWeTzeWvvz+bvHcD76xRVg11EHgVwvsuntqrqt4K+kKruArALAFKplKbT6aBfGjmZTAa23l8ymUQ2m7X2/mz+3gG8v05RNdhV9ZpWNISIiMzgcEciIss0Otzxd0XkBID3ANgnIvvNNIuIiOrV6KiYhwA8ZKgtRERkAEsxRESWYbATEVmGwU5EZBkGOxGRZRjsRESWYbATEVmGwU5EZBkGOxGRZRjsRESWYbATEVmGwU5EZBkGOxGRZRjsRESWYbATEVmGwU5EZBkGOxGRZRjsRESWYbATEVmGwU5EZBkGOxGRZRjsRESWYbATEVmmoWAXkZ0i8pyI/EhEHhKRpKF2ERFRnRrtsR8AcImqXgrgeQB3NN4kIiJqREPBrqr/qaozcx8+CeCCxptERESN6DZ4rT8FsMfrkyIyAGBg7sOzIvKMwdduN28B8HrYjWiit4iIrfdn/fcOvL8oSwU5SVTV/wSRRwGc7/Kprar6rblztgK4AsAmrXZB5/ynVPWKIA2MIt5fdNl8bwDvL+qC3l/VHruqXlPlhT4M4P0AfiNIqBMRUXM1VIoRkY0APgngalWdNNMkIiJqRKOjYv4BwDIAB0RkRES+HPDrdjX4uu2O9xddNt8bwPuLukD3V7XGTkRE0cKZp0RElmGwExFZJrRgt3k5AhG5QUSOicisiFgz9EpENorIqIj8WEQGw26PSSJyr4i8auv8ChFZJSKHROTZuZ/NW8NukykislhEfiAiR+bu7bNht6kZRCQmIsMi8ki1c8Pssdu8HMEzADYBeDzshpgiIjEAXwTwWwDeAeAmEXlHuK0y6usANobdiCaaAbBFVX8NwFUA/tyi799ZABtU9TIAawBsFJGrwm1SU9wK4NkgJ4YW7DYvR6Cqz6rqaNjtMOzdAH6sqi+q6jSAbwL4nZDbZIyqPg7gjbDb0Syq+jNVfXru76fhBERfuK0yQx0Tcx/G5/5YNSpERC4AcB2ArwY5v11q7H8K4N/DbgT56gNwvOjjE7AkGDqNiFwEYC2A74fcFGPmyhQjAF4FcEBVrbm3OV8A8AkAs0FONrlWTIUaliOYAbC7mW0xLci9WUZcjlnVK+oEIrIUwAMANqvqz8NujymqmgewZu5Z3UMicomqWvG8RETeD+BVVT0sIukgX9PUYLd5OYJq92ahEwBWFX18AYCTIbWF6iAicTihvltVHwy7Pc2gqlkRycB5XmJFsANYB+B6EfltAIsBnCsi31DVP/T6gjBHxRSWI7ieyxFEwg8B/KqIXCwiPQBuBPBwyG2igEREAPwTgGdV9XNht8ckEektjKoTkQSAawA8F2qjDFLVO1T1AlW9CM6/u4N+oQ6EW2OvdzmCticivysiJwC8B8A+EdkfdpsaNfeg+6MA9sN58LZXVY+F2ypzROQ+AN8DkBKREyJyc9htMmwdgD8CsGHu39vIXA/QBm8FcEhEfgSnA3JAVasOCbQZlxQgIrJMu4yKISIiQxjsRESWYbATEVmGwU5EZBkGOxGRZRjsRESWYbATEVnm/wFq141J5b9gWQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "y_pred = reg.coef_*X.flatten()\n",
    "plt.scatter(data_rotated[:,0], data_rotated[:,1])\n",
    "plt.plot(X.flatten(), y_pred, 'r')\n",
    "plt.xlim([-2, 4])\n",
    "plt.ylim([-2, 5])\n",
    "plt.axhline(y=0, color='k')\n",
    "plt.axvline(x=0, color='k')\n",
    "plt.grid()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "8dfce343",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-3.3761502190305483"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# R2\n",
    "RSS = ((y - y_pred)**2).sum()\n",
    "TSS = ((y - y.mean())**2).sum()\n",
    "1 - RSS/TSS"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78fa150a",
   "metadata": {},
   "source": [
    "Very bad $R2$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "921c9818",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAfSUlEQVR4nO3dfZRU5Z0n8O+vm0IKEUtOOhoKHJiTbJkYFjsyMTO9OVugEzDxpQfXTdwkkzmTTOfMjBxhWLRJmImexQUPozGjnknIy5icoIH1paNgwhqh4oQTNEK3QcRWJyrQGDGRy9qhle7q3/5RVd236r7Urapbb099P+d4Qt2+de9zU/Ctp3/3uc8jqgoiIjJHW70bQERE4WKwExEZhsFORGQYBjsRkWEY7EREhmGwExEZZkoYBxGRVwG8DSANYExVF4VxXCIiKl0owZ61WFV/F+LxiIioDCzFEBEZRsJ48lREXgFwAoAC+JaqbnbZpwdADwBMmzbt4vPPP7/i8zaq8fFxtLWZ+Z155MgRqCpM/fxM/uwAXl+ze/HFF3+nqh3F9gsr2Ger6jEReS+AxwGsUNUnvfZPJBI6ODhY8XkbVSqVQjKZrHczqiKZTMKyLAwMDNS7KVVh8mcH8PqanYjsC3IPM5SvNlU9lv3f4wAeBvDRMI5LRESlqzjYReRMETkr92cAnwDwXKXHJSKi8oTRYz8XwC9E5FkATwPYoao/DeG4RESB9PUPoWvjLhwYOomujbvQ1z9U7ybVVcXDHVX1NwAWhtAWIqKS9fUPYe1DBzAymgbmAkPWCNY+dAAA0N0Zr3Pr6sPc28dE1BI27RzMhLrNyGgam3aaO0CjGAY7ETW1Y9ZISdtbQZhPnhIR1dzsWBRDLiE+Oxb1fE9f/xA27RzEMWsEs2NRrFmaMKpswx47ETW1NUsTiEba87ZFI+1YszThun+uJj9kjUAxWZM36YYrg52Imlp3Zxwbli9APNtDj8ei2LB8gWcPvBVq8izFEFHT6+6Mo7szjlQqhRWfTfru2wo1efbYiaileNXe/WryzYbBTkQtpdSafDNiKYaIWkqu9m7yqBgGOxG1nFxN3lQsxRARGYbBTkSt6cUXgbGxereiKhjsRNRaNm0CRIBEAvipmRPRssZORK3hzjuBVasmX8+ZAyxdWrfmVBN77ERktrvvzvTQc6H+3vcCx48DR44AkUh921Yl7LETkZm++U3gb/928vWsWcDzzwPnnlu/NtUIg52IzPLtbwM9PZOvZ84EXngBeN/76temGmOwE5EZLrsMySeemHw9fXpm5Evc3PHqXlhjJ6LmdvnlmRp6LtSnTgUOHwb+8IeWDHWAPXYialBFF8O45BLg6afz3tN/553ovOGGGre08bDHTkQNx3cxDJHMf/ZQ/9nPAFWcXLiwbm1uJAx2Imo4bothHFp/Obo/Midv299d3YsPrvsJ+mZdUMvmNbzQSjEi0g7gGQBDqnpFWMclotZjX/Ti1duccXLfwmX4yrLrMy+yqx+ZPKlXqcKssd8A4BCAmSEek4ha0OxYFHvWXurY/uCFi7H6itWO7W6LWbeyUIJdROYA+BSAWwH8QxjHJKIWJYI9BZse+09/htX//R8xLdIGnBp1vgWZunysFu1rAmH12O8EcCOAs7x2EJEeAD0A0NHRgVQqFdKpG8/w8LCx12dZFtLptLHXZ/JnBzT29SUXL3ZsG571Hvzbv3wHU9vbsOHszKpHR95yn5HxjcH9mDJDGvb6aqniYBeRKwAcV9V9IpL02k9VNwPYDACJREKTSc9dm14qlYKp1xeLxWBZlrHXZ/JnB9T2+ooOV8wRcW6bORM4eRIzAKwo+NG83h2u5xMA/7bsTKM/v6DC6LF3AbhKRD4JYBqAmSLyQ1X9XAjHJqImlBuumBvZkhuuCEwuTeca6ACg6nvseCzqWlM3aTHqSlUc7Kq6FsBaAMj22P8nQ53IbMV6427DFUdyo1cKhixO8Aj0wnMtvqADD+4byjv+xGLUJ1+q/OIMwCdPiagkQXrjx1x61G7DFgH49tDdzvXgviFcc3Ecu1940/HFkkox2IGQg11VUwBSYR6TiBqLb288G+yzbeWScgK92Ll2v/Am9vQuKaP1rYE9diIqiVtvvHD7mqWJkksu5Z6LnDilAFGd9fUPoWvjLszv3YGujbsy86E0MK+blBPbRVxDvW//0ZJCPdC5yBWDnaiOfCe7alBrliYQjbTnbYtG2jNPirqNdFEFVMt65N/rXGuWJko+VithKYaojorVqwOPBa+h3Plz7Xqlghp6qedqlP8PGh2DnagOcoHtNcfJMWsk2FhwBHsQKLfPZ+a+ja9u3FVxOHZ3xkOpoQc+F4O8JAx2ohorDGw3s2PRQKNPgoR/3j5zJ/d55rW3XIcMFlXmg0VUO6yxE9WYW2Db5WrIQUaE+IV/sX227D1cWm0/t8BFoWwNnRoHg52oxvyG6sVjUWxYvgDdnfFAI0KChL/XPoVRXPiFMIGB3nQY7EQ15hXY8VgUe3qXTJRDgowI8TpWm8hE7zs2PRK4bXlfArNmMdCbFIOdqMaCDuHr7oxjw/IFiMeiEOT35v2OBQBpVax96ADW9R3A8Dvu09y6mR2LAn/8x5lAP3Ei/4cBAr3ZxuSbijdPiWqslCF8xUaE5H62etuzSBeE7shoGvc/dcSxHQCikTYAkld7/9U9n0fH8AnHvkF750FH8VD1MdiJ6iDMIXzdnXGs2jrg+jO3UAeAd0bH8fVPX4RNOwdx7z//FT7w+yPOnUost3jdpF297dmJdlJtMNiJ6qjcB5Ds7zs7GoGIew63i7iG++xYFN2LL0T3yZPON5VZP/e6SZsrCwEM91phsBMFFPZToOWWLgrfZ4041wAFMnX7ay6OO+Yuf/JbX8L51m+db6jwhuhsjwUwAOf4e6ou3jwlCqAac7oEGYMe9H2F2gQ4Y0obtuw9jGmRNsSiEfT94B+w4nPdjlDv2vAE5t+0veKbnV43cnM4I2PtsMdOFECQp0BLVe6UtEECclwne/L/+p3V+NiR5xz79O0/mun5Z483ZI1g1dYBrNw6gHgZv5H43cgFOCNjLTHYiQKoxrzgXqWLYgHoV/Kw++bDt2LZi790/iAbups27nJ8WeXiuNwRLbl9C6dM4IyMtcVSDFEAXmGrQNkljFKmpM2ND5/XuwOvn/QP9Tu2345Xb7vCEerzb9qO1O7dE6+LfSkFKQu5CTL+nqqLPXaiANYsTXhO3JUrYTzz2ltY370AQLAbrUHHs/f1D2HNA89iNJ3pT48XVDmikTZMi7Tjn3+wDpf+x68c7Zt346OACOIFX05Bev7l/kbCGRnri8FOFIA9hN3CUAFs2XsYi/5oFgA4Rrus3DqAWx49iK9deWFe4PkFYLGpfXN+dO9qLDx6yLE9F+iA7TeBk5OLPft9WeWwLt6cGOzU8uy9696LxmH1D/k+BTq/d4djAi0gE+43P3IQb78z5nrz8MSp0cB16yBT+25+aD0+8dJex/YPf2U7uv/kfMRdpuRNpSaDvfDLSpA/MRjr4s2LwU5NoxqrCRUG6On0eNHw9StheI0pz3GbT93tmvyGNN6+/XZcc3C3Y3ti9UN4d8pUIA3sfuFN7Old4tsWIP83hkZcrYnKU3Gwi8g0AE8COCN7vAdU9WuVHpfIrlrzkJQzjHHN0gRWbR1w7bUHkatb+12TW237ju23Y7lLoH945TYMnzHd9RylYF3cHGH02N8FsERVh0UkAuAXIvITVXX+jkhUpmqMIwfKG8bY3RnHM6+9hS17D5cV7rm6tdc1rdw6kDcVwC2P/yu+sH+H4zgXX/9D/P7MmO85qDVVHOyqqgCGsy8j2f84WTOFqpJx5H4lhnLHkq/vXoBFfzQr77inTo/hxCn/Uowg0zPv2rjL96ZoWhUr9tyP1b/Y4vjZf/nyd3A0dp7ne1kbJ9EQJswXkXYA+wC8H8A9qnqTyz49AHoAoKOj4+Jt27ZVfN5GNTw8jBkzZtS7GVWxcuVKpNNp3HXXXTU97+Bv38bp9Lhj+9T2NiTOO8vzfdbIKIZOjGDc9ve8TQTxc6KIRSOOn58bBd58Z/Lnfsd94+Q7OJ0ex9T2Npx79jQAcJxLRNAmQLpwjKKPBY8/huT3Nzu23/e/78Tvz5/n+p4pbYKxcZ1oi1fbTf67CZh/fYsXL96nqouK7RdKsE8cTCQG4GEAK1TV+QxzViKR0MHB0h98aBapVArJZLLezaiKZDIJy7IwMDBQ0/O6jRKJRtqLPvji1TPOrVaUO7Z9VMy5iY+UNAmXvS2A+7j0Yj10ALj64G58Y/vtju3//u0H8PEvXeN5jHOmR9D/T5/wPXaOyX83AfOvT0QCBXuoo2JU1RKRFIBlADyDnahUpSxOYRekhGO/aZhKpZAscky/er99absg7QCAP3t1APdtXefY/sVr/hFPvP8SyMuA9u5ALBpBpF0mHlQCMl8oX7vyQt/2UusJY1RMB4DRbKhHAVwG4LaKW0ZUoJxRG1419DYRzO/dUdawvnLq/W7tuPjo83hwy42OfVd/chUeXHDpxOtcjFsjo4i0Cc6ZHoF1atTRdg5XpJwweuzvA/D9bJ29DcA2Vd0ewnGJKub1dGVuxEk5wybLueFqb8eFb/wHdtx7g2Of/7XkS/jun3T7nnt0XDF96hRH6YXL0pFdGKNifg2gM4S2EIWusITT5rKiUKnDJt2+LIqNROnujOPMw6/gz7s/7vjZz6/5Ir6y6H/gmDXiueKRndtvBjc/ctC1PHTLowcZ7C2IT55S0ytWgrCXcOb3OseDA6U90FNyvf/114HZs/HnBZu3X/BxXH/1TZkbr9n3B5lKoPA3g77+Ic8nXk+cGkWfxxQJZC4GOzW1YiWIwtA/OzvEsVAuLK2RUXRt3FU0sAPV+y0LOOccx+Zfn/d+XPWFOyde239jKDZ/i30cvH36AT9ckq71MNipqRVbXq4w9CPtgkibYNQ2rjzSJjh1egzzendg9YIxDFlTJvYvq049MgJMn+7Y/M6s9+CCv7nX9S1D1kjezdzCoZiFIV9s+gE7LknXerjQBjU1vxEqbqE/mlbMmDZlYhGIWDQCCDyfGC1psYmxscw0uS6hDlVcumar79vd1lLt7oxjT+8SxGNRx+PcubYFWXGJWguDnZqaV2jNjkU9Q986NYo9vUvwysZP4cwzpuSNC3dTtMermgn0iMvTnqoTS9EF7Tm7fZn4fYH5LSLN6QVaE4OdmlZf/xBOnR5zbM+FmV/o5wQJW98erwjQ5vLPyBbogY5ToLBdftdiX4oOANqzi2twSbrWxRo7NSWv0SOxaAQ3XzW5SlGxYYnFlofz7PFmw9PRrv1HMzdrXR5+chsmWXhz1N4uu2JDLDnlLtmxx05NyWshijPPmDIRcEEWVS5WpnD0eEXcQ10VffuPYu1DBzBkjXjWyzcsX5A3Qdf0qe2Of4SRNnG0iwtEUynYY6emFOSx/qALSt/8yEHXIZDxbJkDgGcP3V5uCTpn/Ltjk7NU/uG088tpdFzxzGtvubaVQU5BsMdOTalY/TxXqvHqPdvdfNWFjpuPE2UOnx56YQ09yJeN35J3dlv2HnZtK1EQDHZqSm4jQew152Lj2+0Kbz7GY1EcWn85uj8yx3lil0DPCetmLZCpuwceZklUgKUYakq5ksQtjx6cGIN+xpTJfkqpMzBOlDlEsMJthwDrFgSZQ6bYzVq7oPsRFWKPnZraO6OT9WprZHSi3BKk95ynhJKLl8IbnLFoBNMibVi1dQBdG3ehr3/Id8x5oXavuj5REQx2alp+5ZZipZoJIQS6Xe5J0a9/+iK8OzaOE6dG82r8APLC3y+8i83ySOSFpRhqWn7llqIzMHoEamr3biSTyZIXrSjc/9TpsUCrLPX1D2HV1gHXsezxEh5oIrJjsFPT8lsdKTdVrSOMiw1bTKVKXrTCbX8vhV9G3Z1xPPPaW9iy93BeuHMqAKoESzHUtLzq1WlV59DGACWXvv4hDP72bazcOhB4RA0QfAgj4F7jX9+9AF//9EV8+IhCwx47Na1c8K3e9qz3qkhuQxYBR/081+v+uwvG4dXfKWetUzu/XjgfPqIwscdOTa27M45xl5uMr952BfasvdT5Bo+bokF63aWOtIlFI+yFU12wx06B9fUPof+whfeeMZa3gk+92Wvtr952hes+H1z3k0ywehyjWK/br7ftNX7dPhkZUS0x2CmQXKni3bE0cEYFqwtVwZqlCc+Sy7ybtmf+UGTBar8Hh+JFRsWUvAYqUZUx2CmQoBNclaOUoYWF++5Ze6lrL3wi0G1yvXK38+V63cDk/O7RSHvg8glr5NRIKg52EZkL4AcAzgMwDmCzqn6j0uNSY6n0xqEXt6GCa/7Ps7jl0YOwTo3mBb19X6+SC1TRtXEX4NKu2bGo51DGDcsXYMPyBXhjcD8kuy973dSswuixjwFYrar7ReQsAPtE5HFVfT6EY1OD8CpVVLqepuu6pOM6Mf+LveSzaecgDq2/3P1AthuifnO2+P3msad3CVInX8IrG5NlX0+pDzYRVUPFo2JU9XVV3Z/989sADgHg32TDBH5Ev0RBevwjo2l0f2SO6yiXeTdtx/yCsovfohTV+s0DKG2qYKJqCrXGLiLzAHQCeCrM41L95XqdX9jSDmCs6A3FoIrNduhVcrHX0O1Pmtrb69a2Yk+rxkpoe6Fq3ocgKoVoSBMNicgMAD8HcKuqPuTy8x4APQDQ0dFx8bZt20I5byMaHh7GjBkz6t2Mqli5ciXS6TTuuuuuUI5njYxi6MSIYyz6is91u+5/z5Yfu45bbxNB/Jxo3rJzpZxv4hgzBLGZZwW/AJsDQyc9f7YgfnZZxwybyX83AfOvb/HixftUdVGx/UIJdhGJANgOYKeq3lFs/0QioYOD5i4ikEqlkEwm692Mqkgmk7AsCwMDA2Ufo7AOvfiCDmx/9nVYI6OePfS+/UcnbqC6PWkKZEoue3qXBDq/1zHWXjSOL3/mSs+2+v2W0rVxl+tvA0HbVQsm/90EzL8+EQkU7BXX2EVEAHwXwKEgoU6tza0OvfXpIxi4ealrqHdteGIi1Nf1HfAMZCB4ndzraVUAOJ2enN+91Jp5te5DEJUqjBp7F4DPAzggIgPZbV9R1cdCODYZprAO7VdDt/d01/UdwA/3HvY9dikjdLxq7VPbJ/s6pdbM+aASNYqKg11VfwGAS71QILle9UubrkZk3Dk3i/2mqL0Hfv9TR3yPW2rP2GtI5LlnT3U9v53fbwZ8UIkaAZ88pZr6lyfuxpXP/NSx3e1JUXsP3G81IbcROsVq416969jJl/LOX42x+0TVxmCn2lixArj7blxZsHneTdsRaRNEBBhNT4Z3YQ+8XcQ13NtFHDcmgy6U4da7TqUmgz3I4tSl4MNLVCuctpeq68YbMwtc3H133uauDU9gfraOvunahdj03xb6TnF73SVzXQ/vtt2vNl4KvwedSsWHl6iW2GOn6rj1VmDdOuf2bK97j8tb/AJzffcCAJlae1oV7SK47pK5E9vtwny6NKyaOR9eolpisFO47rsP+OxnndtDeF5iffcC1yAv5Fcbt5dDYtMjUAVOjkxONharuJXuqjmVAVEhlmIoHA88kCm5FIa6x4pF1eQ1nnzxBR155ZATp0ZhjYzmlUaskdGqtKnU1ZeIKsFgp7L09Q+ha+Mu/M01/5QJ9Guvzd+hDoGe41Ub3/3Cm77L342MpvHGyXeq0iY+vES1xFIMlWxsXHHH93+OPd+oTsklDG618VVbB4q+z/7kadjtAfjwEtUGg51Kc/o0Zh08iCcP5If6xJOiAQ9jr3WfHY1ABI6FNcJWbCZJIP/J07Dx4SWqFQY7leapp9A2nunVfu2yL+P7F0+OTA96I3Bd3wFs2XsYub69va5dzbVU3cal2xU+eUrUrBjsVJqFC2H99jjm/eW3HD8KciOwr38oL9TdlDMMMMjDP4XlENdRMbYnT4maFYOdSjNzJsbG0ohG2j2fyPQL2U07B31DPaeUYYBBnzTNvfb7wrA/eUrUrDgqhko2pU08n8gs9oRl0MAuZRhgWE+aEpmCPXYqi1fPt9gTlkFuYJY6DJAP/xDlY4+dQlUsZN3GcwPAmVPby56PhQ//EOVjj51CVWyq22qM5w57FkaiZsdgp1AFCdmwx3Pz4R+ifAx2ClW9QpYP/xBNYrBTWfyGNDJkieqLwU4lGxvXwOPGiaj2OCqGSnZ6bJzjxokaGIOdSqYeMzhy3DhRYwgl2EXkeyJyXESeC+N41NhExHU7x40TNYaweuz3AlgW0rGowU2d0sZFI4gaWCjBrqpPAngrjGNR4/ObK4aI6o+jYqgsHNJI1LhqFuwi0gOgBwA6OjqQSqVqdeqaGx4eNvb6LMtCOp029vpM/uwAXl+rqFmwq+pmAJsBIJFIaDKZrNWpay6VSsHU64vFYrAsy9jrM/mzA3h9rYLDHYmIDBPWcMf7AfwSQEJEjorIF8M4LhERlS6UUoyqXhfGcYiIqHIsxRARGYbBTkRkGAY7EZFhGOxERIZhsBMRGYbBTkRkGAY7EZFhGOxERIZhsBMRGYbBTkRkGAY7EZFhGOxERIZhsBMRGYbBTkRkGAY7EZFhGOxERIZhsBMRGYbBTkRkGAY7EZFhGOxERIZhsBMRGYbBTkRkGAY7EZFhQgl2EVkmIoMi8rKI9IZxTCIiKk/FwS4i7QDuAXA5gA8BuE5EPlTpcYmIqDxTQjjGRwG8rKq/AQAR+RGAqwE87/WGI0eOIJlMhnDqxmRZFmKxWL2bURUDAwMYGxsz9vMz+bMDeH2tIoxgjwM4Ynt9FMAlhTuJSA+AHgCIRCKwLCuEUzemdDpt7PWNjY1BVY29PpM/O4DX1yrCCHZx2aaODaqbAWwGgEQioQMDAyGcujGlUilje7TJZBKWZcHUz8/kzw7g9TU7Ebe4dQrj5ulRAHNtr+cAOBbCcYmIqAxhBPuvAHxAROaLyFQAnwHwSAjHJSKiMlRcilHVMRG5HsBOAO0AvqeqBytuGRERlSWMGjtU9TEAj4VxLCIiqgyfPCUiMgyDnYjIMAx2IiLDMNiJiAzDYCciMgyDnYjIMAx2IiLDMNiJiAzDYCciMgyDnYjIMAx2IiLDMNiJiAzDYCciMgyDnYjIMAx2IiLDMNiJiAzDYCciMgyDnYjIMAx2IiLDMNiJiAzDYCciMgyDnYjIMBUFu4hcKyIHRWRcRBaF1SgiIipfpT325wAsB/BkCG0hIqIQTKnkzap6CABEJJzWEBFRxSoK9lKISA+AHgDo6OhAKpWq1alrbnh42NjrsywL6XTa2Osz+bMDeH2tomiwi8jPAJzn8qOvquqPg55IVTcD2AwAiURCk8lk0Lc2nVQqBVOvLxaLwbIsY6/P5M8O4PW1iqLBrqqX1aIhREQUDg53JCIyTKXDHf9CRI4C+FMAO0RkZzjNIiKiclU6KuZhAA+H1BYiIgoBSzFERIZhsBMRGYbBTkRkGAY7EZFhGOxERIZhsBMRGYbBTkRkGAY7EZFhGOxERIZhsBMRGYbBTkRkGAY7EZFhGOxERIZhsBMRGYbBTkRkGAY7EZFhGOxERIZhsBMRGYbBTkRkGAY7EZFhGOxERIZhsBMRGaaiYBeRTSLygoj8WkQeFpFYSO0iIqIyVdpjfxzAh1X1PwN4EcDayptERESVqCjYVfX/qupY9uVeAHMqbxIREVViSojH+msAW71+KCI9AHqyL98VkedCPHejeQ+A39W7EVX0HhEx9fqM/+zA62tmiSA7iar67yDyMwDnufzoq6r64+w+XwWwCMByLXbAzP7PqOqiIA1sRry+5mXytQG8vmYX9PqK9thV9bIiJ/oCgCsAXBok1ImIqLoqKsWIyDIANwH4r6p6KpwmERFRJSodFXM3gLMAPC4iAyLyzYDv21zheRsdr695mXxtAK+v2QW6vqI1diIiai588pSIyDAMdiIiw9Qt2E2ejkBErhWRgyIyLiLGDL0SkWUiMigiL4tIb73bEyYR+Z6IHDf1+QoRmSsiu0XkUPbv5g31blNYRGSaiDwtIs9mr+2WerepGkSkXUT6RWR7sX3r2WM3eTqC5wAsB/BkvRsSFhFpB3APgMsBfAjAdSLyofq2KlT3AlhW70ZU0RiA1ar6QQAfA/D3Bn1+7wJYoqoLAVwEYJmIfKy+TaqKGwAcCrJj3YLd5OkIVPWQqg7Wux0h+yiAl1X1N6p6GsCPAFxd5zaFRlWfBPBWvdtRLar6uqruz/75bWQCIl7fVoVDM4azLyPZ/4waFSIicwB8CsB3guzfKDX2vwbwk3o3gnzFARyxvT4KQ4Kh1YjIPACdAJ6qc1NCky1TDAA4DuBxVTXm2rLuBHAjgPEgO4c5V4xDCdMRjAHYUs22hC3ItRlGXLYZ1StqBSIyA8CDAFaq6v+rd3vCoqppABdl79U9LCIfVlUj7peIyBUAjqvqPhFJBnlPVYPd5OkIil2bgY4CmGt7PQfAsTq1hcogIhFkQn2Lqj5U7/ZUg6paIpJC5n6JEcEOoAvAVSLySQDTAMwUkR+q6ue83lDPUTG56Qiu4nQETeFXAD4gIvNFZCqAzwB4pM5tooBERAB8F8AhVb2j3u0Jk4h05EbViUgUwGUAXqhro0KkqmtVdY6qzkPm390uv1AH6ltjL3c6goYnIn8hIkcB/CmAHSKys95tqlT2Rvf1AHYic+Ntm6oerG+rwiMi9wP4JYCEiBwVkS/Wu00h6wLweQBLsv/eBrI9QBO8D8BuEfk1Mh2Qx1W16JBAk3FKASIiwzTKqBgiIgoJg52IyDAMdiIiwzDYiYgMw2AnIjIMg52IyDAMdiIiw/x/PYcLBYBBFhsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "y_pred = reg.intercept_ + reg.coef_*X.flatten()\n",
    "plt.scatter(data_rotated[:,0], data_rotated[:,1])\n",
    "plt.plot(X.flatten(), y_pred, 'r')\n",
    "plt.xlim([-2, 4])\n",
    "plt.ylim([-2, 5])\n",
    "plt.axhline(y=0, color='k')\n",
    "plt.axvline(x=0, color='k')\n",
    "plt.grid()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "2f1ecef5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7159013201761292"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# R2\n",
    "RSS = ((y - y_pred)**2).sum()\n",
    "TSS = ((y - y.mean())**2).sum()\n",
    "1 - RSS/TSS"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "81d7691a",
   "metadata": {},
   "source": [
    "Better $R^2$ when centering the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "c70e517a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAgU0lEQVR4nO3df5RT9Z038PcnmTATxMfAcVplgIVWn9BFq6OU7R6621BtcVtrR+x2+2u32/Upu+dZ95Gu5QHWtuBue8Cy27W17emy1rPtPraFCo4I9YCKcZ9yHlqFGToijNVqhWD9UQg6TpyZZD7PH5mE/Lg39yb3ZpJ85/06h6O5ubn5XmZ455vP/X6/V1QVRERkjkCjG0BERP5isBMRGYbBTkRkGAY7EZFhGOxERIZhsBMRGcZzsItIh4j8QkQOi8gREbnNj4YREVFtxOs4dhERAOeo6pCIhAD8DMDNqnrAjwYSEVF12rweQLOfDEMTD0MTfzjriYioQTwHOwCISBDAQQAXAfi2qv7cYp+VAFYCQEdHx5Xz5s3z462b0vj4OAIBcy9fmHp+x48fh6qCv5uty/Tze/rpp19V1U6n/TyXYooOJhIBcB+Av1PVJ+32i0ajOjg46Nv7Npt4PI5YLNboZtSNqecXi8WQTCbR39/f6KbUjak/uxzTz09EDqrqYqf9fP1oU9UkgDiAa/w8LhERuefHqJjOiZ46RCQM4GoAx7wel4iIauNHjf1CAN+fqLMHAGxT1V0+HJeIiGrgx6iYXwLo9qEtRETkA3MvHxMRTVEMdiIiwzDYiYgMw2AnIjIMg52IyDAMdiIiwzDYiYgMw2AnIjIMg52IyDAMdiIiwzDYiYgMw2AnIjIMg52IyDAMdiIiwzDYiYgMw2AnIjIMg52IyDAMdiIiwzDYiYgMw2AnIjIMg52IyDAMdiIiwzDYiYgMw2AnIjIMg52IyDAMdiIiwzDYiYgM0+b1ACIyF8APAFwAYBzAFlX9htfjEpG5evsS2LxnECeTKcyOhLF6eRQ93V2NbpYxPAc7gDSAW1T1kIicC+CgiDykqk/5cGwiMkxvXwLrdgwgNZYBACSSKazbMQAADHefeC7FqOqLqnpo4v9fB3AUAH86RGRp857BfKjnpMYy2LxnsEEtMo+vNXYRmQ+gG8DP/TwuEZnjZDJV1Xaqnh+lGACAiMwAsB3AKlV9zeL5lQBWAkBnZyfi8bhfb910hoaGeH4tKJlMIpPJGHluOc3ws1t7+ThGM+Nl26cFA5ZtS6bG8NKZNzGaGce0YABvPa8DkXDI8tjNcH7NwJdgF5EQsqF+j6rusNpHVbcA2AIA0WhUY7GYH2/dlOLxOHh+rScSiSCZTBp5bjnN8LNLltTYASAcCmLjiksRK6mx9/YlsO6RAaTGAsgVGMKhDDau+H3LenwznF8z8FyKEREB8D0AR1X1696bREQm6+nuwsYVl6IrEoYA6IqEsXHFpZZBzXp8bfzosS8F8OcABkSkf2LbP6jqT304NhEZqKe7y9UIGNbja+M52FX1ZwDEh7YQERWZHQkjYRHisyPhBrSmdXDmKRE1rdXLowiHgkXbwqEgVi+PNqhFrcG3UTFERH7LlWs4S7U6DHYiampu6/F0FksxRESGYbATERmGwU5EzS+TAcbGGt2KlsFgJ6LmpQp89rNAWxswbVqjW9MyePGUiJrT/fcDPT1nH3+Dt3lwi8FORM3lxAlg7tyzj9/+dmBgAAhzUpJbLMUQUXPIZICrrioO9SefBJ55hqFeJQY7ETXed7+braPv25d9/J3vZOvrixY1tl0tiqUYImqcp54qDu9YDHj4YSAYtH0JOWOwE9HkO3MGiESKt73wQnEZhmrGUgwRTS6R4lDfvj1bdmGo+4bBTkST45//ORvqhcbHgRUrGtMeg7EUQ0Se9fYl7FdgPHgQWLy4+AXPPgu87W2T39Apgj12IvKkd+IepolkCgogkUxh3Y4B9B46ke2hF4T6kZvWZssuDPW6YrATkSdW9yU9+pU/Qc+VxTXz+Wt24aOR96K3LzGZzZuSGOxE5Enh/Uc/3fdTPH/7tUXP//HKf8f8NbsA8EbUk4U1diLyZHYkjFMvn8bRf/1o0fa9F78bK1d8sWx/q3uYkr8Y7ETkyf51V5Vte8cXH0RHKAAMly+1K8jW5XlXpPphKYaIarNkSdnwxXfe/GMs3fgINq64FOs/vAhi8TIFWI6pM/bYiag6zz4LXHRR8babbgLuvBO/LNl11dZ+y0OcZDmmrhjsRATAYSx6TukEIyA7fNFGVyRsWVOfHeFqjfXEUgwR2Y9Fzw1NFCkP9UymYqgDwOrlUYRDxQt6hUNBrF4e9bH1VIrBTmS43r4Elm7ahwVrd2Pwt69bjiO3GoueGsvgZ9/4fnmg//CH2UAPOMdHT3cXNq64FF2RMATZHvzGFZfywmmdsRRDZLBcTzwX2qOZcazbMQAAReFaVvNWxfNf+3D5AR166Ln3LC3p7F/7vtpPgqrGYCcymF1PfPOewaJgn11QCy+dYATAVaAD5R8kuZIOAPbSJ5EvpRgRuVtEXhaRJ/04HhH5w270Sen21cuj+Nreb5WF+r6tD7kOdaDyBwlNHr967P8B4FsAfuDT8YjIB7PdjEp57TX0XDGn6PmTMy/ALx55oupettsPEqovX3rsqvpfAE75cSyiZlZ4IXLppn1Nv6CV46gUEeC884pfpIrZp16sqXRiN4yRwxsnl2gVX7MqHkhkPoBdqnqJzfMrAawEgM7Oziu3bdvmy/s2o6GhIcyYMaPRzagbU89v1apVyGQyuPPOOy2fT6bGkDidwnjBv5mACLpmhhEJhyarmVVLpsbw0pk3MZoZx4XTgbaO6ej54AfK9tu/fTvGZs3y/F6N/Dsy9XczZ9myZQdVdbHTfpN28VRVtwDYAgDRaFRjsdhkvfWki8fj4Pm1nkgkgmQyaXtuSzftQyJZfpPlrkgQ+9fG3E3wabDH77oL7/rc54o3XnIJMDCApT69RyP/Hkz93awWR8UQudDbl7BdlfBkMtUao0FE8K7SbT59Yy/U093VPOc8RTHYiRzkQtvO7EjY9bBCN71Z33u8VssApNNAsPzbB5nBr+GOPwLw/wBEReSEiNzox3GJmoFVaOfkLkS6GQ3iOG3fYZ+qL9xa3Dz6ub/8y2wvnaFuNF967Kr6CT+OQ9SMKg3Vy02P37xn0HFYoZtevd0+G3YewUh63F2px266vyp+E49jgf2pkiG4VgyRA7uhel2RcD5U3Sx25aZXb7dPMjXmbuKPSHmoq9allk7Ni8FO5MBNaLtZ7MruAyIgki+rVDveO/9BYHHTCzzwAAN9iuLFUyIHhWWSShc0nUaDrF4eLRo5k5NRzZdVli3sxD0HXkBhHIdDQXSEAjhtcZu5izrGq14jPacVhmdSbRjsRC74MYQv9/pbth1GpiR4U2MZ3PbAEbw5Nl4U6gLghiu7sPj3ZpV9KHCxLrLDUgzRJOrp7iqalVno9HB5HV0BPHrslaJSz/O3X1se6seP+7JY1y3bDjf9MgnkjMFONMlqraP3vPwk9q+7qnwHVWDOnPLtLo5ZKlcWYri3NpZiiFyqtSZd+LrzwiGMZcbL9gmHgmhvCyCZKq+jz46Ea66j27Fb9RGwnlhFrYXBTsby8+JgrTXp0tdZBXckHMKG6xYBgLs6+ugoEPK2oJbdhdwcLrPb2hjsZCS/Lw66XTLAzetKiZwdcROZHkJ7WwBr7vs6PnF4T9F+yYWX4kOfuQMnv7TX8wdVpQu5AJfZbXUMdjJSrUFsp9YbSLjp+Z4eHssPZTz9xqjlvUZ7D53IflBNHM+PUSy515X23EvH6FPr4cVTMpLfd/Kp9QYS1fR8n7/92vJQn5g1avdBtWprv6cbfriZWEWthz12MpKrW8JVwaomXaln29uXwG0PHLGcVFTKqo5+21Wfw/qHt+QfV/pA8tp75zK75mGPnYxktQwAAAyPpst6t7lVEw/8+ncYHs1Y9n6r6dn29iWw+t7DlqEeDgUwc3oIAmBR25uWoT5/zS7svfrjRducPpB4w2gqxB47GSkXuBt2HikaiXJ6eAyrtvZjw84jliNRtGB6f7VLBuTc9sARjGWshyLOOqcd+9e+z3L44vw1uwBYfxNwGsUCcCQLncVgJ2PlltO1GmKYTI1h3Y4BdIQCtrXrzXsGqx6rbjc2PGf/uquAdcXb7rzjXvw4NRPisA4NgIrvwZEslMNgp5ZTzfj0Sr3Y1FimYg+41rHqVm58vBdf2ndX2fb5a3Yh/GoQG1c4f4DkvjFYvR9HslAhBju1lGrHp1eaYelG4RBJuw8Up7HqdnV0q/dww+1qkzR1MdipLuq1JGy149Pd1KadON2s2u5bgVWgR/9+O0ZC7ZbvUQ2OZKFKGOzku3ouCVvt+PTc+7kdemil0s2qV23tR1CkaPam5TIAKO6lW70HkV843JF8V6lX7VUtE4V6urvQ9+UP4I4/u7xouOLM6c7rrTjdrBrA2VBXtS27FIZ66XgY1sfJb+yxk++8zvqsVMapdqJQodLyheVFTxFEwiGcSY0hMj0EVeDzW/sRKOmVl3Kqoxf61Lvn4dFjr7A+TnXDYCffeZn16VTGqeXCod0HRemx2tuCaG9T9K//QFk77ELdsuwyezaW/t1/AhZ/BzOnh/CVnksd/x6IvGCwk++89KrdXByt5sKhmw+K3LFiBzYjmUzatqPQglMJPPrvf122fenGR7B6eRSrYb241voPL3LVbiIvWGMn33lZWMrvxbtqrfdXer/nb7+2LNRzdfTCDw4urkWNwh471UWtw/HsyjgBESxYu7vqmnStHxRW7bAqu6y69hb0LlpWtC03WqarQlvrNRyUCGCPnZqM3eJdGVUozpZS3C5TW+tyu4Xt+PLDW2wvjpaGeiG7tubKQ4lkqqZzInLCHjs1ldILmlajUaqZqVlrvT9fz7+i/CbRSzc+gpPJFLoiYbwxkrZci6ZSW/2+CQhRKV+CXUSuAfANAEEAd6nqJj+OS1NTYRlnwdrdlvu4rbnXPP1eBD0lm96/fif+9iNXYL/TkEmHttotcZCYmOHKcCevPAe7iAQBfBvA+wGcAPC4iOxU1ae8HpvMU21t2Y8bZlRT748cPmy/pO6bKJtBW+2qi719CQgAuxHxfs3QpalNtMKkC1cHEPlDABtUdfnE43UAoKob7V4zffp0XbJkiaf3bWbJZBKRSKTRzaibWs/v1aER/PqVNzBe8DsXEMHbOs/B+TPa8/scP5XCSDqD9rYgItNDeOX1EcvXACjad+6scP44teh/7DEAwOUF2w7MvaRsv/a2ILrnRWo6v74XkhhJV163xu74fuDvZmt77LHHDqrqYqf9/CjFdAE4XvD4BIA/KN1JRFYCWAkAoVAoP17YRJlMhudnYXQ0g67ppR0JxejwEJLpFNLjipH0ON7SrkA7AKSB8QzmnytIj2dvgiEiaAsAo8NDUFW8pR35fUeGh/Dq6DDaAuU97koihw+XbUtedhneGEljrmXfOo2TL/8u355pbQG0BQRtAN52nmA0rch2mATAOEYL2vWW9vREeytJ1+33h7+bU4MfwW71r6jsX4OqbgGwBQCi0aj29/f78NbNKR6PIxaLNboZdVPr+S1Yu9syJgVA/6YPYemmfZbljLdEwtm7DuFsTXvUpqZduK+jkpJLbOK/cVX09iXw+a39tu0t3B4MBfGVgjHqVnX33D5ubsbRVc05VIm/m61NLMqEVvwY7ngCwNyCx3MAnPThuGQYp6GHbsacO80IdXVR9emnLevoeO97kbzssvz72BUpS7eXTniqNOrFbjhnDhcEIz/4EeyPA7hYRBaIyDQAHwew04fjkmGsQq0wyNyMOXfq7TpeVBUBoiXBqZr9U6Dama6F+1f6gCqdlTtzegiRcIizU8lXnksxqpoWkZsA7EF2uOPdqnrEc8vIKLnRMKmxTH798tKZmW7GnJeufV6oYm/Xqof+zW+i9z0rsHnTPpxMpnDqhSTmzsge2240jt37F36gOI3k4U0yqN58mXmqqj9V1f+uqm9X1a/6cUwyR+FMSyA7izQXwoUB52aNmUpL51r2dm+5xTrUVdH7nhVFM0BH0hmMpMfR25ew/XbxiT+YW/FbB+D8zYSo3jjzlOqumpmWTr3ZLpvecFckXP46m0Cv1C6oYvOewfzFyw07j+RnlnaEAvn/5l4XCYew4bpFZR9QueNzLRhqBAY71Z2fKza6WiLAKtBPnQJmzqy6XSPp8fz/nx4ew/858ELRvoXPF2K5hRqJwU5152b2qNsZqRV7w3ZDwWzKN07tchqBA2S/eXx+W39R24gajcFOdefUy6725teWvWGHsovbdkEk3y633yhUgdU/OWzbXqLJxmV7qe4KL4oC2ZEluRp74WiZQq5vfi1SHuoWwxed2iXAxK3xAvlwrmY9mrFx9eVm3UR+YLDTpOjp7sqPFsmNbMn1zO3GplfsMVsFekeHq0Avbdf+te/Dc5s+hO55kaLlCJwmE1XVXqJJxGCnSWPXMw/a1MYte8wvvmhfdknVHqy9fQn0vZDEGyNpLN20L798bunwy0qq6eET1RNr7DRp7Hq0uXHtjjfDqKGO7kauxj+SzgDt1je8zun+x704PWx9Yw2OU6dmwR47TRq7Hm1uIpLtxCSrssu2bUWh3tuXwNJN+7Bg7e58j9tO6b63PXDEdY1//YcXIRQs/4D59Lvn8cIpNQ322GnSWI5CAfDGSBoAylc0vPtu4MYbyw9U0kuvZlSN1b52rL5hcPIRtQIGO02aXPjd9sCRonJGMjVWHsQuyi65ETVW4Ww3s9XN2PQcu28YnHxEzY6lGJpUPd1dmD6tvD+RL31YlV3OnLHtpVfb43Y7coVru1ArY4+dJp1VuB765icxK/Va+c42F0fd9Lytetx2s00j4RBOtwUBpMtWnSRqNeyx06QrDdznb7+2PNQdJhk59bztetx2Ky9uuG4RuudFcE57G/avfR9DnVoae+yGcrv2SiPkLqIe/cqflD03f80uhENBbJwYR27HrucNoGKPu9LFzztqOx2ipsNgN1C1a69Mtp4r5qCnZNux838P19z4bQD2Fz4L2a0/4+YORLz4SaZjsBuomvXPq9Xbl8BLv30dn127u/pvAqdPA7NmlW2ev2ZX2bbCUkulbx/N+q2EqJEY7Abyc/3zQrlvAv9z4TgUgfw3gSd+cwqPHnulLGALA/m5268tP6Aqlm7aB1RYOtfp2weDnKgcg91AbtY/r4XdN4F7DryA3GXOwrDffjBhWUfHQw8BV18NwHlJ33p++wCKvw0U3vOUqJVxVIyB6nXPTbsef2kUpsYyOLZjr2WoL934SD7UAef7nNbr2wdQPBa+9J6nRK2MPXYD1av+XGkkSqHnLcouuTq62EzTt2tbvb59AJXvecoSD7UyBruh6lF/zpVNgHR+m+Bsj90q0Bf+/b14M9SRfxwQyS+JW817FgawAFi2sLOGMyhWz28DRI3EUgy5liubTAsG8mWTT717Hm7++U/KQn3ovFl4xxcfLAp1ILtE77odA67LHT3dXbjhyi4ULjKgALYfTHgumdj1+rmuOrU69tipIquhhtELzsVzm2LZHSwW6+o9dCL7IdCXwC3bDufvmJRT7cXPR4+9YlnHLz1GtZOynO55StSqGOxky26o4Zeu1IqrL/ZMvHbDziNloZ5TTbnDTcmklklZpdcisvc8VdbXqeWxFEO2rC4urt/5r/jk9R8s2nb71f8DvYdO5B/39iWw+ieHkUxZ32kIqK7c4aZkUusNsSvd85SoVbHHTrYKe8TT0mN4+l+uL9snN9qlq6AssnnPIMbG7ceDVzv00mmse2lb7c6BaKrwFOwi8qcANgB4B4AlqvqEH42i5pAbalhp+GJOYYA6hamb9VwKuRm+Wc9hkUStxmuP/UkAKwD8mw9toSbzH09tw8X/Wfyj/aO/vgsf+6PzgYHifQsD1GnlRatQd7rw6TR8002vnmiq8BTsqnoUAMTqQhq1rldfBTo7cXHBph9dthzf+vj/xrKFnQgMP1e0e2mArl4exeqfHC4rx4SC1iNO/FiN0u9JWc287DGRE9EKNzNwfRCROIAvVCrFiMhKACsBoLOz88pt27Z5ft9mNTQ0hBkzZjS6GTWJLVtWti3+6KNFj5OvvY6XhoHRzDimBQN463kdiIRDxfukxnAymUJmItzbAoILI+Gy/QBg8LevYzQzXrZ9WjCA6AXnejmdqqxatQqZTAb/9LWvI3E6hfGCfxsBEXTNtG5/K2nl3003TD+/ZcuWHVTVxU77OfbYReRhABdYPHWrqt7vtkGqugXAFgCIRqMai8XcvrTlxONxtNz5vetdwBMln8tjY0BbG2Ilu8bjcfRcV7q1dp9duxtqMUBLgLPj5SdBJBJBMpnE5sNBJJLBsue7IkHsXzt57amHlvzdrILp5+eWY7Cr6tVO+1ALe/ZZ4KKLirdt3Qp87GOT1oRmu/DJETbU6jjccSqrMMloMtld+Fy2sBNLN+3DyWQKkekhqAJnUmN1r3k32wcNUbU8TVASketF5ASAPwSwW0T2+NMsqqvLLy8PdYebR9eT1dK9N1zZhe0HE/kldU8PjyGZGoPi7MXVei2vW69lj4kmi9dRMfcBuM+ntlAd9fYl8Mh3fow77/pC8RO/+lV5KaYBSoczLt20r3xJ3QJ+3mzDqi0Ab7tHrYulmCmg99AJLFl2BXpeeyW/bVv3NZj2vbvQc1FzhpWbenY9a9687R61Mga76e65Bz2f/nTRJqtlACopHdP9twtH8rXvevVm3dzUgzVvImsMdpM9+CAwEeoDb307rv/zf0E6ePZH7qbH+8XegbJ7mv7ujTQSyfH842onE7lhuaRuAda8iewx2E22cCFwww342PyP4Bdts8qedurx9vYlikLdTi31bjdLCABn69yTOSqGqNUx2E22YAFw7734ZF8CAzWso7J5z6BjqOdUU+92u4QA69xEtWGwTwGVRnlU6jlXE9bV1LsrrZ3OICfyjsE+RVj1fp16zm4uYALV17s5s5OovngHpSnM6a5DVhN1BMCM9raiyUTVrq/Om0gT1Rd77FOYU8/ZroQTOfMr7P9orOb35drpRPXFYJ/C3KyJYlXCicd/5el9ObOTqL4Y7FNYI3vOHPFCVD8M9imMPWciMzHYpzj2nInMw1ExRESGYY99CuCNmYmmFga74dxO3ycic7AUYzinSUhEZB4Gu+E4fZ9o6mGwG47T94mmHga74XhjZqKphxdPDcdJSERTD4N9CuAkJKKphaUYIiLDMNiJiAzDYCciMgyDnYjIMAx2IiLDeAp2EdksIsdE5Jcicp+IRHxqFxER1chrj/0hAJeo6jsBPA1gnfcmERGRF56CXVX3qmp64uEBAHO8N4mIiLzws8b+VwAe9PF4RERUA8eZpyLyMIALLJ66VVXvn9jnVgBpAPdUOM5KACsBoLOzE/F4vJb2toShoSGeXwtKJpPIZDJGnluOqT+7HNPPzy1RVW8HEPkMgL8BcJWqDrt5TTQa1cFBc9cDj8fjiMVijW5G3Zh6frFYDMlkEv39/Y1uSt2Y+rPLMf38ROSgqi522s/TWjEicg2ANQDe6zbUiYiovrzW2L8F4FwAD4lIv4h814c2ERGRB5567Kp6kV8NISIif3DmKRGRYRjsRESGYbATERmGwU5EZBgGOxGRYRjsRESGYbATERmGwU5EZBgGOxGRYRjsRESGYbATERmGwU5EZBgGOxGRYRjsRESGYbATERmGwU5EZBgGOxGRYRjsRESGYbATERmGwU5EZBgGOxGRYRjsRESGYbATERmGwU5EZBgGOxGRYRjsRESGYbATERmGwU5EZBhPwS4i/yQivxSRfhHZKyKz/WoYERHVxmuPfbOqvlNVLwewC8CXvTeJiIi88BTsqvpawcNzAKi35hARkVdtXg8gIl8F8BcAzgBYVmG/lQBWTjwcEZEnvb53EzsfwKuNbkQdmXx+54uIqecGmP2zA8w/v6ibnUS1cidbRB4GcIHFU7eq6v0F+60D0KGq6x3fVOQJVV3spoGtiOfXukw+N4Dn1+rcnp9jj11Vr3b5nj8EsBuAY7ATEVH9eB0Vc3HBw+sAHPPWHCIi8sprjX2TiEQBjAP4DYC/cfm6LR7ft9nx/FqXyecG8Pxanavzc6yxExFRa+HMUyIiwzDYiYgM07BgN3k5AhHZLCLHJs7vPhGJNLpNfhKRPxWRIyIyLiLGDC0TkWtEZFBEnhGRtY1uj59E5G4RednU+SMiMldEHhWRoxO/mzc3uk1+EZEOEfmFiByeOLfbHF/TqBq7iPy33MxVEflfAH5fVd1efG1qIvIBAPtUNS0itwOAqq5pcLN8IyLvQPaC+b8B+IKqPtHgJnkmIkEATwN4P4ATAB4H8AlVfaqhDfOJiPwxgCEAP1DVSxrdHr+JyIUALlTVQyJyLoCDAHpM+PmJiAA4R1WHRCQE4GcAblbVA3avaViP3eTlCFR1r6qmJx4eADCnke3xm6oeVdXBRrfDZ0sAPKOqv1bVUQA/BvCRBrfJN6r6XwBONbod9aKqL6rqoYn/fx3AUQBdjW2VPzRraOJhaOJPxbxsaI1dRL4qIscBfArmLiD2VwAebHQjyFEXgOMFj0/AkGCYakRkPoBuAD9vcFN8IyJBEekH8DKAh1S14rnVNdhF5GERedLiz0cAQFVvVdW5AO4BcFM92+I3p3Ob2OdWAGlkz6+luDk/w4jFNmO+RU4VIjIDwHYAq0qqAi1NVTMTq+jOAbBERCqW0zwvAubQGGOXI3A6NxH5DIBrAVylLThZoIqfnSlOAJhb8HgOgJMNagvVYKL+vB3APaq6o9HtqQdVTYpIHMA1AGwvhDdyVIyxyxGIyDUA1gC4TlWHG90ecuVxABeLyAIRmQbg4wB2NrhN5NLEBcbvATiqql9vdHv8JCKduZF1IhIGcDUc8rKRo2K2I7sEZX45AlVNNKQxPhORZwC0A/jdxKYDpoz4AQARuR7AnQA6ASQB9Kvq8oY2ygci8kEAdwAIArhbVb/a2Bb5R0R+BCCG7LK2LwFYr6rfa2ijfCQi7wHwfwEMIJspAPAPqvrTxrXKHyLyTgDfR/b3MgBgm6r+Y8XXtGCVgIiIKuDMUyIiwzDYiYgMw2AnIjIMg52IyDAMdiIiwzDYiYgMw2AnIjLM/wcI/qiwthNJ7gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "X_centered = X.flatten()-X.mean()\n",
    "y_centered = y - y.mean()\n",
    "y_pred = reg.coef_*X_centered\n",
    "plt.scatter(X_centered, y_centered)\n",
    "plt.plot(X_centered.flatten(), y_pred, 'r')\n",
    "plt.xlim([-3, 3])\n",
    "plt.ylim([-3, 3])\n",
    "plt.axhline(y=0, color='k')\n",
    "plt.axvline(x=0, color='k')\n",
    "plt.grid()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "75b1bf83",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7159013201761293"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# R2\n",
    "RSS = ((y_centered - y_pred)**2).sum()\n",
    "TSS = ((y_centered - y_centered.mean())**2).sum()\n",
    "1 - RSS/TSS"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c7857e7",
   "metadata": {},
   "source": [
    "Centered data without intercept <=> uncentered data with intercept\n",
    "\n",
    "**If data are not centered, we need to use intercept.**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91fc6009",
   "metadata": {},
   "source": [
    "### Predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "b6d053c7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[3.29439706 3.12104591 2.42787468 2.66025231 2.40743574 2.2100448\n",
      " 2.15548195 2.70688728 3.10653862 2.36010903]\n",
      "[3.29439706 3.12104591 2.42787468 2.66025231 2.40743574 2.2100448\n",
      " 2.15548195 2.70688728 3.10653862 2.36010903]\n"
     ]
    }
   ],
   "source": [
    "# Scikit learn\n",
    "print(reg.predict(X)[:10])\n",
    "# Manual\n",
    "print(reg.intercept_ + np.dot(X, reg.coef_)[:10])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "213b83d0",
   "metadata": {},
   "source": [
    "### Assumptions"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d85d9728",
   "metadata": {},
   "source": [
    "*Gauss-Markov theorem*:\n",
    "\n",
    "OLS estimator is BLUE (Best Linear Unbiased Estimator = unbiased and lowest variance) if:\n",
    "\n",
    "Errors need to:\n",
    "\n",
    "- be uncorrelated\n",
    "\n",
    "- have constant variance (homoscedasticity)\n",
    "\n",
    "- have an expectation value of zero"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8aeed730",
   "metadata": {},
   "source": [
    "$$\\epsilon = y - \\hat y$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "1c3312aa",
   "metadata": {},
   "outputs": [],
   "source": [
    "res = y - (theta[1] + X.T*theta[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "ce4515af",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAQwklEQVR4nO3df6zddX3H8edr/PhjSIKul1+FUrI0ZGhGJTdFQ2ZAhbWFDDW6tVmUOZKqgUQT/1g3E132F8uiSxQm64SAiYIuipJQfo2YIIk/KKRIWWF0pI7ahhbZQIIJqb73x/02uV7Ouffc8z23Fz59PpKT8/1+Pp/v9/M535z76rff8/2RqkKS1K7fW+4BSJKWlkEvSY0z6CWpcQa9JDXOoJekxh2/3AMYZMWKFbV69erlHoYkvWk8+uijL1TV1KC6N2TQr169mh07diz3MCTpTSPJz4fVeehGkhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNW7BoE9ydpIfJNmd5Mkkn+7K35bkgSTPdO9vHbL8+iRPJ9mTZOukP4AkaX6j7NEfBj5bVX8EvAu4Nsn5wFbgwapaAzzYzf+OJMcBNwIbgPOBzd2ykqSjZMGgr6oDVfVYN/0rYDewErgKuK1rdhvwgQGLrwP2VNWzVfUacEe3nCTpKFnUlbFJVgPvBH4CnFZVB2DmH4Mkpw5YZCXw3Kz5fcBFQ9a9BdgCsGrVqsUMSzpqVm+9e1n63Xv9FcvSr9ow8o+xSd4CfAf4TFW9POpiA8oGPtKqqrZV1XRVTU9NDbxdgyRpDCMFfZITmAn5b1TVd7vi55Oc0dWfARwcsOg+4OxZ82cB+8cfriRpsUY56ybAzcDuqvrSrKq7gKu76auB7w9Y/BFgTZJzk5wIbOqWkyQdJaPs0V8MfBR4b5Kd3WsjcD1wWZJngMu6eZKcmWQ7QFUdBq4D7mPmR9xvV9WTS/A5JElDLPhjbFU9zOBj7QDvG9B+P7Bx1vx2YPu4A5Qk9eOVsZLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcYt6Zqw023I9P1XS4rhHL0mNW3CPPsktwJXAwap6R1f2LeC8rskpwP9V1doBy+4FfgX8BjhcVdMTGbUkaWSjHLq5FbgB+PqRgqr6iyPTSb4IvDTP8pdW1QvjDlCS1M8ojxJ8KMnqQXXdg8P/HHjvhMclSZqQvsfo/wR4vqqeGVJfwP1JHk2ypWdfkqQx9D3rZjNw+zz1F1fV/iSnAg8keaqqHhrUsPuHYAvAqlWreg5LknTE2Hv0SY4HPgR8a1ibqtrfvR8E7gTWzdN2W1VNV9X01NTUuMOSJM3R59DN+4GnqmrfoMokJyU5+cg0cDmwq0d/kqQxLBj0SW4HfgScl2Rfkmu6qk3MOWyT5Mwk27vZ04CHkzwO/BS4u6rundzQJUmjGOWsm81Dyv9qQNl+YGM3/SxwQc/xSZJ68spYSWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJatwojxK8JcnBJLtmlf19kl8k2dm9Ng5Zdn2Sp5PsSbJ1kgOXJI1mlD36W4H1A8r/uarWdq/tcyuTHAfcCGwAzgc2Jzm/z2AlSYu3YNBX1UPAi2Osex2wp6qerarXgDuAq8ZYjySphwUfDj6P65J8DNgBfLaq/ndO/UrguVnz+4CLhq0syRZgC8CqVat6DEtqz+qtdy9b33uvv2LZ+tZkjPtj7FeBPwTWAgeALw5okwFlNWyFVbWtqqaranpqamrMYUmS5hor6Kvq+ar6TVX9Fvg3Zg7TzLUPOHvW/FnA/nH6kySNb6ygT3LGrNkPArsGNHsEWJPk3CQnApuAu8bpT5I0vgWP0Se5HbgEWJFkH/AF4JIka5k5FLMX+ETX9kzga1W1saoOJ7kOuA84Drilqp5cig8hSRpuwaCvqs0Dim8e0nY/sHHW/HbgdadeSpKOHq+MlaTGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYtGPRJbklyMMmuWWX/lOSpJD9LcmeSU4YsuzfJE0l2JtkxwXFLkkY0yh79rcD6OWUPAO+oqj8G/gv423mWv7Sq1lbV9HhDlCT1sWDQV9VDwItzyu6vqsPd7I+Bs5ZgbJKkCZjEMfq/Bu4ZUlfA/UkeTbJlvpUk2ZJkR5Idhw4dmsCwJEnQM+iTfA44DHxjSJOLq+pCYANwbZL3DFtXVW2rqumqmp6amuozLEnSLGMHfZKrgSuBv6yqGtSmqvZ37weBO4F14/YnSRrPWEGfZD3wN8CfVdWrQ9qclOTkI9PA5cCuQW0lSUtnlNMrbwd+BJyXZF+Sa4AbgJOBB7pTJ2/q2p6ZZHu36GnAw0keB34K3F1V9y7Jp5AkDXX8Qg2qavOA4puHtN0PbOymnwUu6DU6SVJvXhkrSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjRvlUYK3JDmYZNessrcleSDJM937W4csuz7J00n2JNk6yYFLkkYzyh79rcD6OWVbgQerag3wYDf/O5IcB9wIbADOBzYnOb/XaCVJi7Zg0FfVQ8CLc4qvAm7rpm8DPjBg0XXAnqp6tqpeA+7olpMkHUULPhx8iNOq6gBAVR1IcuqANiuB52bN7wMuGrbCJFuALQCrVq0ac1iSJm311ruXpd+911+xLP22aCl/jM2AshrWuKq2VdV0VU1PTU0t4bAk6dgybtA/n+QMgO794IA2+4CzZ82fBewfsz9J0pjGDfq7gKu76auB7w9o8wiwJsm5SU4ENnXLSZKOolFOr7wd+BFwXpJ9Sa4BrgcuS/IMcFk3T5Izk2wHqKrDwHXAfcBu4NtV9eTSfAxJ0jAL/hhbVZuHVL1vQNv9wMZZ89uB7WOPTpLUm1fGSlLjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1LhxHw4uSUtquR5KDu09mNw9eklq3NhBn+S8JDtnvV5O8pk5bS5J8tKsNp/vPWJJ0qKMfeimqp4G1gIkOQ74BXDngKY/rKorx+1HktTPpA7dvA/476r6+YTWJ0makEkF/Sbg9iF1707yeJJ7krx92AqSbEmyI8mOQ4cOTWhYkqTeQZ/kRODPgH8fUP0YcE5VXQB8BfjesPVU1baqmq6q6ampqb7DkiR1JrFHvwF4rKqen1tRVS9X1Svd9HbghCQrJtCnJGlEkwj6zQw5bJPk9CTpptd1/f1yAn1KkkbU64KpJL8PXAZ8YlbZJwGq6ibgw8CnkhwGfg1sqqrq06ckaXF6BX1VvQr8wZyym2ZN3wDc0KcPSVI/XhkrSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjesV9En2Jnkiyc4kOwbUJ8mXk+xJ8rMkF/bpT5K0eL0eJdi5tKpeGFK3AVjTvS4Cvtq9S5KOkqU+dHMV8PWa8WPglCRnLHGfkqRZ+u7RF3B/kgL+taq2zalfCTw3a35fV3Zg7oqSbAG2AKxatarnsI4tq7fevdxDkPQG1neP/uKqupCZQzTXJnnPnPoMWKYGraiqtlXVdFVNT01N9RyWJOmIXkFfVfu794PAncC6OU32AWfPmj8L2N+nT0nS4owd9ElOSnLykWngcmDXnGZ3AR/rzr55F/BSVb3usI0kaen0OUZ/GnBnkiPr+WZV3ZvkkwBVdROwHdgI7AFeBT7eb7iSpMUaO+ir6lngggHlN82aLuDacfuQJPXnlbGS1DiDXpIaZ9BLUuMMeklq3CTudSO8OlXSG5d79JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqXJ9nxp6d5AdJdid5MsmnB7S5JMlLSXZ2r8/3G64kabH63L3yMPDZqnqse0j4o0keqKr/nNPuh1V1ZY9+JEk9jL1HX1UHquqxbvpXwG5g5aQGJkmajIkco0+yGngn8JMB1e9O8niSe5K8fZ51bEmyI8mOQ4cOTWJYkiQmEPRJ3gJ8B/hMVb08p/ox4JyqugD4CvC9Yeupqm1VNV1V01NTU32HJUnq9Ar6JCcwE/LfqKrvzq2vqper6pVuejtwQpIVffqUJC1On7NuAtwM7K6qLw1pc3rXjiTruv5+OW6fkqTF63PWzcXAR4Enkuzsyv4OWAVQVTcBHwY+leQw8GtgU1VVjz4lSYs0dtBX1cNAFmhzA3DDuH1IkvrzylhJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWpcn1sgvCGt3nr3cg9B0pvccuXI3uuvWJL1ukcvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1Lj+j4cfH2Sp5PsSbJ1QH2SfLmr/1mSC/v0J0lavD4PBz8OuBHYAJwPbE5y/pxmG4A13WsL8NVx+5MkjafPHv06YE9VPVtVrwF3AFfNaXMV8PWa8WPglCRn9OhTkrRIfW6BsBJ4btb8PuCiEdqsBA7MXVmSLczs9QO8kuTpHmMb1wrghWXo983C7bMwt9H83D7zyD/22j7nDKvoE/QZUFZjtJkprNoGbOsxnt6S7Kiq6eUcwxuZ22dhbqP5uX3mt1Tbp8+hm33A2bPmzwL2j9FGkrSE+gT9I8CaJOcmORHYBNw1p81dwMe6s2/eBbxUVa87bCNJWjpjH7qpqsNJrgPuA44DbqmqJ5N8squ/CdgObAT2AK8CH+8/5CW1rIeO3gTcPgtzG83P7TO/Jdk+qRp4yFyS1AivjJWkxhn0ktS4Yzrok3wkyZNJfptk6ClNC93qoVVJ3pbkgSTPdO9vHdJub5InkuxMsuNoj/No89Yf8xth+1yS5KXu+7IzyeeXY5zLJcktSQ4m2TWkfuLfn2M66IFdwIeAh4Y1GPFWD63aCjxYVWuAB7v5YS6tqrWtnyPtrT/mt4i/lx9235e1VfUPR3WQy+9WYP089RP//hzTQV9Vu6tqoStwR7nVQ6uuAm7rpm8DPrB8Q3nD8NYf8zuW/15GUlUPAS/O02Ti359jOuhHNOw2DseC045c99C9nzqkXQH3J3m0u5VFy0b5PhzL35lRP/u7kzye5J4kbz86Q3vTmPj3p88tEN4UkvwHcPqAqs9V1fdHWcWAsmbOSZ1v+yxiNRdX1f4kpwIPJHmq22tp0URv/dGgUT77Y8A5VfVKko3A95g5TKEZE//+NB/0VfX+nqto+jYO822fJM8nOaOqDnT/dTw4ZB37u/eDSe5k5r/vrQa9t/6Y34KfvapenjW9Pcm/JFlRVd7sbMbEvz8eulnYKLd6aNVdwNXd9NXA6/4HlOSkJCcfmQYuZ+ZH7lZ564/5Lbh9kpyeJN30OmZy6JdHfaRvXBP//jS/Rz+fJB8EvgJMAXcn2VlVf5rkTOBrVbVx2K0elnHYR9P1wLeTXAP8D/ARgNnbBzgNuLP7uz0e+GZV3btM411yjd76Y2JG3D4fBj6V5DDwa2BTHUOX6Ce5HbgEWJFkH/AF4ARYuu+Pt0CQpMZ56EaSGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMb9PzBe+ovytdyrAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(res.flatten())\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73af3053",
   "metadata": {},
   "source": [
    "##### Assumption 1: errors have constant variance (homoscedasticity)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "5c01d103",
   "metadata": {},
   "outputs": [],
   "source": [
    "np.random.seed(2)\n",
    "df_data_heterosc = pd.DataFrame()\n",
    "df_data_heterosc['constant'] = np.ones(data_rotated.shape[0])\n",
    "df_data_heterosc['x'] = data_rotated[:,0]\n",
    "df_data_heterosc['y'] = data_rotated[:,1]\n",
    "df_data_heterosc = df_data_heterosc.sort_values(by='x').reset_index()\n",
    "# df_data_heterosc['y_2'] = df_data_heterosc\\\n",
    "#                     .apply(lambda x: (x['y']**6)/100**7 if x.name%2==0 else -(x['y']**6)/100**7, axis=1)\n",
    "\n",
    "df_data_heterosc['y_2'] = df_data_heterosc\\\n",
    "                    .apply(lambda x: x['y']+x.name**1.1/100*np.random.normal(0,1), axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "51e3ebc7",
   "metadata": {},
   "source": [
    "White’s Lagrange Multiplier Test for Heteroscedasticity: \n",
    "\n",
    "$$\\mathcal{H}_0: \\text{there is homoscedasticity}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "d3789b0a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAcsklEQVR4nO3df4xc9Xnv8ffDZlM2oe228kqYNcZURb630ItNRmBk6YqiSCaGJlbDVYhuEjX/WOQmVdImrkz+CCRSVUu+SkJKG8u9iXpRopC0uK5LiKxIpjc/rqDZxTbga3xFSxK8RmWhWYjL3sY/nvvHzNjj2XPmnJlzzpzvOefzklbanTkz83CA53zn+T7f7zF3R0REqu+ysgMQEZF8KKGLiNSEErqISE0ooYuI1IQSuohITbylrA9etWqVr1u3rqyPFxGppPn5+VfdfSbqudIS+rp165ibmyvr40VEKsnMfhL3nEouIiI1oYQuIlITSugiIjWhhC4iUhNK6CIiNVFal4uISOj2H15g98ETnFpa5qrpKXZsWc+2jbNlhxVLCV1EJML+wwvct+9Zls+cA2BhaZn79j0LEGxSV8lFRCTC7oMnLiTzruUz59h98ERJESVTQhcRiXBqaXmox0OghC4iEuGq6amhHg+BErqISIQdW9YzNTlxyWNTkxPs2LK+pIiSaVJURCRCd+Izzy6XortmlNBFRGJs2zibW8IdR9eMSi4iImMwjq4ZJXQRkTEYR9eMErqIyBiMo2tGCV1EZAzG0TWjSVERkTEoomumnxK6iMiY5Nk1E0UlFxGRmkid0M1swswOm9ljEc/dZmavm9mRzs9n8g1TRESSDFNy+ThwHPiVmOe/7+53ZQ9JRERGkWqEbmZrgDuB/1FsOCIiMqq0JZcvAn8MnB9wzK1mdtTMvmNm10cdYGbbzWzOzOYWFxeHDFVERAZJTOhmdhfwirvPDzjsaeAad78R+DNgf9RB7r7X3Vvu3pqZmRklXhERiZFmhL4ZeLeZ/Rh4BLjdzL7We4C7v+Hupzu/Pw5MmtmqvIMVEZF4iQnd3e9z9zXuvg64Bzjk7h/oPcbMrjQz6/x+c+d9XysgXhERiTHywiIzuxfA3fcAdwMfMbOzwDJwj7t7PiGKiEgaVlbebbVaPjc3V8pni4hUlZnNu3sr6jmtFBURqQkldBGRmlBCFxGpCSV0EZGaUEIXEakJJXQRkZpQQhcRqQkldBGRmlBCFxGpCSV0EZGaUEIXEakJJXQRkZpQQhcRqYmRt88VEQnR/sML7D54glNLy1w1PcWOLevZtnG27LDGQgldRGpj/+EF7tv3LMtnzgGwsLTMffueBViR1OuY+FVyEZHa2H3wxIVk3rV85hy7D5645LFu4l9YWsa5mPj3H14YY7T5U0IXkdo4tbSc6vG0ib9qlNBFpDaump5K9XjaxF81qRO6mU2Y2WEzeyziOTOzL5nZC2b2jJndlG+YIiLJdmxZz9TkxCWPTU1OsGPL+kseS5v4q2aYEfrHgeMxz70LuK7zsx34csa4RESGtm3jLH/6e7/N7PQUBsxOT/Gnv/fbKyY7oxI/wJu/OFvpOnqqLhczWwPcCfwJ8EcRh7wHeNjbd5x+0symzWy1u7+cX6giIsm2bZxN7FbpPv/AgWMsLZ+58PjP3jwT2xVTBWlH6F8E/hg4H/P8LPBSz98nO49dwsy2m9mcmc0tLi4OE6eISK62bZzl7b+0ckxb5cnRxIRuZncBr7j7/KDDIh7zFQ+473X3lru3ZmZmhghTRCR/dZscTTNC3wy828x+DDwC3G5mX+s75iRwdc/fa4BTuUQoIlKQuk2OJiZ0d7/P3de4+zrgHuCQu3+g77ADwIc63S6bgNdVPxeR0KXtiqmKkZf+m9m9AO6+B3gc2Aq8ALwJfDiX6ESkFkJdZt+NIcTYRmHtxpTxa7VaPjc3V8pni8j49O+vAu1RcFQ7oSQzs3l3b0U9p5WiIlKoui6zD5F2WxSRVEYtm9StkyRkSugikmiYbWn7XTU9xUJE8h5XJ0mo9fsiqOQiIomylE3K7CSp6za5cZTQRSRRlrJJ2v1VitC0+r1KLiKSKGvZJM3+KkVoWv1eI3QRSVTVBTh1WwmaRAldRBKVWTbJoqoXolGp5CIiqZRVNsmibitBkyihSyM1qZWt6ap4IRqVEro0TpaeapGQKaFL4wxqZYtL6BrRSxUooUvjDNvKphG9VIUSujRC7wj7MjPORewyGtfKNsqIXqQMSuhSe/0j7KhkPqiVLevilCzlGpV6ZBhK6FJ7USNsgAkzzrsnJsosqySzlGtU6pFhKaFL7cWNpM+78+KuOxNfv2PL+sgbNKRZnJKlXDPMazWSF1BClwbIYx8SGG1xSpZyTdrXaiQvXYkJ3cwuB74H/FLn+L9x9/v7jrkN+Dvgxc5D+9z9c7lGKjKiLCPsrlEXp2S5mKR9rSZtpSvNXi7/Dtzu7jcCG4A7zGxTxHHfd/cNnR8lcwlGmfuQZNlLJO1rm7ajoMRLHKF7+y7Spzt/TnZ+yrmztMiIylr+naVck/a1Zd8RSMKRqoZuZhPAPPCbwJ+7+1MRh91qZkeBU8Cn3P1YfmGKVFf/xWT/4QU27zo0MEn3T3J+4X0bYi8CeZSU8qQJ2vKkSujufg7YYGbTwN+a2Q3u/lzPIU8D17j7aTPbCuwHrut/HzPbDmwHWLt2bcbQRaonzQTmsJOcIe0oqAnacplHLLIY+AKz+4F/c/f/PuCYHwMtd3817phWq+Vzc3NDfbZI1W3edSiyPDI7PcUPd96e+phQVTn2qjCzeXdvRT2XOClqZjOdkTlmNgW8E3i+75grzcw6v9/ced/XMsYtUjtpJjCrPMlZ5djrIE2Xy2rgCTN7BvgR8F13f8zM7jWzezvH3A0816mhfwm4x4cd+os0QJpbolX5tmlxMTrt0fv+wwvjDahhEhO6uz/j7hvd/T+5+w3dlkR33+Puezq/P+Tu17v7je6+yd3/d9GBi1RRmlbEKt82LSr2rm49XUm9OFopKpKjqA4PuHTC8r3vmOWJ5xdjJzBDmuQcVm/sUbV0LXgq1tCTonnRpKiUociWuv4OD4DJywwMzpy7+P/Z1OREJW6wnNW1O78duWDFINUeOhIt06SoSFbdvutrd3671DpqN+EuLC3j5F8CiFqCf+a8X5LM4eIote6qPBdQVUroUqiik+gwBu15kodhOjma0PVR5bmAqlINXQoV0sZRRbfUxS3Bjzu2K48yUIirM6s8F1BVSuhSqJD6kove8yRqCX5cDb07Ss1jZWXIqzPL2kOnqVRykUKFVEctugQQtavj7v9yI7vvvjF2p8c8ykBx7/HAgWNBzF3I+GiELoUKaeOocZQA4kakcZ+RxzeYuGOXls+wtHwGCGvULsVRQpdChVZHzbsEkLV2nUcZKG3tXj3g9aeELoWLSqIhTuINY//hBR44cOzCCBhGGwXn8Q0m6j3iNKG7psmU0GXsRp3EC+UiELWAqGvYUXAe32Ci3uPNX5zlZ2+eWXGsesDrTQldxi5uEu+T3zoKRCf1kDo5Pvv3xwaOhocdBedRBoq6iUYocxcyPkroMnZxCe+ce2ySLqufvf9bwe/8h5nIkW+vEEbBoc1dyHgoocvYDZrEi0vScReBhaVlrt357UISVtS3gq89+dOBr4kbBZdRLlIPePOoD10GKmIflkFbrMKlybv7+YO2kCtqS4GobwWD/NrbJiM33Qpp+wOpNyV0iVVUIuouwJlo3+RqhW7Jovfz00izIGeYC9QwtfDpqUnu/93r2X3wxIr3LnoPGZEulVwkVpF16+7rB03cDTtChsFJeNiJ1bT93VOTE9x14+rY9w5p+wOpN43QAxHKFrO9ik5EUUvle0sWcZ/TPTbKoAnJYUfKSaWhbiyXT17G1578aex7h7T9gdSbRugBCKklr1fRm1nB4Im7QZ8/yoKcYS9Q/XffMbiklt/deGtQ18uppWW+8L4NaiGUsUgcoZvZ5Wb2j2Z21MyOmdlnI44xM/uSmb1gZs+Y2U3FhFtPodZYy97PetDnJ43uo8RdiH51ajL229G2jbP8cOft/HjXnXzhfRsu+bwrLn/LiptXRH3mKLGKjCLxFnRmZsDb3f20mU0CPwA+7u5P9hyzFfgDYCtwC/Cgu98y6H11C7qLQr5VV9mrM/P8/LxvERf3723Y9xEZxqBb0CWWXLyd8U93/pzs/PT/d/we4OHOsU+a2bSZrXb3lzPE3RjjKG3AaMmx7F7mPD+/v4QyYcaZ8ytTctqJ30GTprNayCMlSDUpamYTZnYEeAX4rrs/1XfILPBSz98nO4/1v892M5szs7nFxcURQ66fcZQ2mtQLPWiCedvG2Qvn+9yAb6dpJn7j/r198X0b+OHO25XMZexSJXR3P+fuG4A1wM1mdkPfIVENxSv+b3H3ve7ecvfWzMzM0MHW1ThqrCHV6Yvs6Elz4UrTDpnm25Fq4xKaobpc3H3JzP4BuAN4ruepk8DVPX+vAU5ljq5Bii5thNILXXRHT5re+aR/5mG+HZVdkhLplabLZcbMpju/TwHvBJ7vO+wA8KFOt8sm4HXVz8MSSi900d8U0ly4Bv0zz05P8d53zEau+BQJXZqSy2rgCTN7BvgR7Rr6Y2Z2r5nd2znmceCfgReAvwT+WyHRysjKbkHsKvqbQpoL16Da944t63l0fqERcw1SP2m6XJ4BNkY8vqfndwc+mm9okqdQtlMtuqMnzYKjQedi865DsTdcLvvciSRJ7EMvivrQwzHOXvO4Gy/kOZmY5Z8nqbe8a9SYy+7rl+rL1Icu9TbubQfG8U0hy0RlkTdcDnWLB6kPJfSGK+NOQCF3hhR5w+Wy7rokzaGE3nChtDOGosgbLutcS9GU0BtuXNsOjFuWWnVRN1yefttk5IXBgc27DqmeLplpP/SGC6WdMU95b3OQx4rQ/YcXOP3/zsY+r/ZIyYNG6BWTd5dEKO2MeSqiVp217r/74InIjcB6qZ4uWSmhV8ioXRJJF4FhklUV2u5CrFWn/WzV0yULJfQKGWXkmbVVrjeB/+rUJP/2i7MX9g4Pte0urlZd5nbEadshqz53IeVSDb1CRhl5Ztk7pb8WvbR8ZsUdekK4s1KvuFr15IRdMi+QdcfHYev0ae5PWvW5CymfEnqFjLLBVpbyQ5ptZtO+17jE1arf/ta3XBg95zFpOuyFMmpi9QOb1mrrXcmVSi4VMsqNkbO0JaZN1CGVCeJifn35Ygkmj0nTUS6UIS+oknrQCL1CRmmfy9KWmCZRh1YmSPMtJo9J01C2IxbppYReMd270L+4685UtznL0kO9Y8v69k2UY4RYJkhzAcsjGdexf1+qTyWXCsjaKjjqV/1tG2f57N8fi+wYmZ2e4oc7bx/6PYuWpq9+lNLVKJ8jMm5K6IEre4e+pYhkDuOdCB32gpZ0AcsrGasmLqFRQg9c2Tv0lb3XS1EXNCVjqSPV0ANX9qrHsmvFRd+DVKROEkfoZnY18DBwJXAe2OvuD/Ydcxvwd8CLnYf2ufvnco20ocoeIZddK85yQYsr1VRh+wKRUaQpuZwFPunuT5vZLwPzZvZdd/8/fcd9393vyj/EZstjAi+rMssTo17Q4ko1cz/5Vx6dX9Bdg6SWEksu7v6yuz/d+f3nwHFA/+WPyahth1mXtodi1JJPXKnmG0+9FPn4J755pNLnSQSGnBQ1s3XARuCpiKdvNbOjwCngU+5+LOL124HtAGvXrh062KYadoRcdmdMnkYt+cSVZM4NuCl6lc+TCID5gP/ALznQ7ArgfwF/4u77+p77FeC8u582s63Ag+5+3aD3a7VaPjc3N2LYMsjmXYciyxSh9o4XIe4cTJgNTOrQrPMk1WNm8+7einouVZeLmU0CjwJf70/mAO7+hruf7vz+ODBpZqsyxCwZlN0ZE4K4Us37b7k6cdfDJp0nqZc0XS4GfAU47u6fjznmSuBf3N3N7GbaF4rXco1UUndnlN0ZE4JBpZrWNb/OJ755JPa1TTpPUi9pauibgQ8Cz5rZkc5jnwbWArj7HuBu4CNmdhZYBu7xtLUcSWWYungInTEhiJt72LZxlk9+62hs6aVp50nqIzGhu/sPgPgdmtrHPAQ8lFdQstIwK0bL7h0vWh595IPq6HU5T9I8WvpfEcPWxeu6tD2vDp7ZmLLUrMotUmFa+l8R2n+7La+tAMre0kCkCEroFdGUBJS0ICqvDp4s+8SLhEoll4oItS6e574oacopeXbw1LUsJc2lhF4hoSWgvFekppn4VQePSDwl9Aore9fAvPdqT1NOCfWbikgIlNArqoz9WvovIFGlDxh9pWXackpo31REQqFJ0Yoa940fuheQhaVlnPYFJG5xwqidN02Z+BUpikboFTXu/VqiLiBOe8VZ7xKdLAlY5RSRbJTQK2rc+7XEXSicdstfXglY5RSR0SmhV9S4uz3iLiDaalYkHJVK6GV3dYRkUHmiiPOkdkGR8FUmodfpLjx5iSpPFHWeVN8WCV9lEnrePc91VeR5Un1bJGyVaVvUXXjS0XkSaa7KjNBDvgtPSLX9Uc5TSPGLyOgqM0IPddFJ1IKb+/Y9u2KXwLTvNWinwTSGPU95xi8i5apMQg91u9OkFZtpk3ReiXXY8zTuFaciUpw0N4m+GngYuBI4D+x19wf7jjHgQWAr8Cbw++7+dN7BhjgpN6hmPUzHSZ6TmcOcpyrU3FUSEkknzQj9LPBJd/+PwCbgo2b2W33HvAu4rvOzHfhyrlEGbNCdhIYZ/ZaVWEO/E5JKQiLpJSZ0d3+5O9p2958Dx4H+4dF7gIe97Ulg2sxW5x5tgAbVrIdJ0mUl1lDnJrpUEhJJb6gaupmtAzYCT/U9NQu81PP3SVYmfcxsu5nNmdnc4uLikKGGaVDNepgkXVZiDXVuoqsKJSGRUKRuWzSzK4BHgU+4+xv9T0e8xFc84L4X2AvQarVWPF9VcTXrYZbLl7kSszf+br36D795JIh6dcjtqiKhSZXQzWySdjL/urvvizjkJHB1z99rgFPZw6u2YZN0/4Wh2yEzrgQf4vYK2kNGJL00XS4GfAU47u6fjznsAPAxM3sEuAV43d1fzi/M6hq1M6eM5Bri9graQ0YkvTQj9M3AB4FnzexI57FPA2sB3H0P8DjtlsUXaLctfjj3SBumjOQaar06xHZVkRAlJnR3/wHRNfLeYxz4aF5BhSipFzrvXukykmvWerX6xUXKVZm9XMqUVP7Iozyy//ACDxw4xtLyGQAuM/CIaeMiJwOz1KtDrL+LNE1llv6XKakXOmuv9P7DC+z466MXkjnA+YhkXvRkYJYWRvWLi5RPI/QUksofWcsjuw+e4ExUBgcmzDjvPrYSxqj16lDr7yJNooSeQlJtOWvteVDSO+/Oi7vuTBlpedQvLlI+lVxSSFrFmXWV56CkV5WEGPoWAiJNoISeQlJtOevy+R1b1jN52cpGoskJq0xCDH0LAZEmMI9qpRiDVqvlc3NzpXx2iPq7XH7tbZPc/7vXj7woSe2DIvVkZvPu3op6TjX0QOS1eKbp7YO6mEmTqeRSM01uH9Te6dJ0Sug10+T2wSZfzERACb12Qr8DUZGafDETASX02mly+2CTL2YioIReO01uH2zyxUwE1OVSS2k7ZurWEaK906Xp1IfeUP3tjdAezb73HbM88fyiEqJIoNSHLivEdYR8/cmfXrgZbNN62EWqTjX0horr/Oj/vqa2P5HqqM0IvW714KLF7Y4YRW1/ItWQOEI3s6+a2Stm9lzM87eZ2etmdqTz85n8wxxMKwSHF9UREnefQbX9iVRDmpLLXwF3JBzzfXff0Pn5XPawhhPCCsH9hxfYvOsQ1+78Npt3HQr+YhLV3vhfN61V259IhaW5SfT3zGzdGGIZ2ThXCEaVdoBKbogV1d7YuubXVboSqai8aui3mtlR4BTwKXc/FnWQmW0HtgOsXbs2p48ez91y+re3hYuJ+/LJy2K/IVQtGea166OIjF8eXS5PA9e4+43AnwH74w50973u3nL31szMTA4f3Vb0CsFujb43mXctnznHz95c+ThoMlFExivzCN3d3+j5/XEz+wszW+Xur2Z977SKXiEYVaNPo+jJRHX2iEivzAndzK4E/sXd3cxupj3qfy1zZEMqslSQNNKenprk38+eX7HqssjJxKbfyEJEVkpM6Gb2DeA2YJWZnQTuByYB3H0PcDfwETM7CywD93hZ+wmkNOzIdlDP9tTkBA+8+3pgvHuIDOrsUUIXaaY0XS7vT3j+IeCh3CIq2Cgj2x1b1q/Y9wRW3vdznIlUe3+LSL/arBRNa5SRbYi7+I2js0dEqqVxCX3UkW1o7XxR3xq0CEik2Rq3OVdd7mrT5BtZiEi0xo3Q6zSyDe1bg4iUq3EJPcR6uIhIHhqX0EEjWxGpp8bV0EVE6koJXUSkJpTQRURqQgldRKQmlNBFRGqi8l0u2kJWRKSt0gldW8iKiFxU6ZJLCDeHFhEJRaUTuraQFRG5qNIJvS4bbYmI5KHSCb3om0OLiFRJpSdFtdGWiMhFae4p+lXgLuAVd78h4nkDHgS2Am8Cv+/uT+cdaBxttCUi0pam5PJXwB0Dnn8XcF3nZzvw5exhiYjIsBITurt/D/jXAYe8B3jY254Eps1sdV4BiohIOnlMis4CL/X8fbLz2Apmtt3M5sxsbnFxMYePFhGRrjwSukU85lEHuvted2+5e2tmZiaHjxYRka48EvpJ4Oqev9cAp3J4XxERGUIebYsHgI+Z2SPALcDr7v5y0ovm5+dfNbOfjPiZq4BXR3xt2aocO1Q7/irHDtWOv8qxQ1jxXxP3RJq2xW8AtwGrzOwkcD8wCeDue4DHabcsvkC7bfHDaSJy95FrLmY25+6tUV9fpirHDtWOv8qxQ7Xjr3LsUJ34ExO6u78/4XkHPppbRCIiMpJKL/0XEZGLqprQ95YdQAZVjh2qHX+VY4dqx1/l2KEi8Vu7YiIiIlVX1RG6iIj0UUIXEamJoBO6md1hZifM7AUz2xnxvJnZlzrPP2NmN5URZ5QUsd9mZq+b2ZHOz2fKiDOKmX3VzF4xs+ding/5vCfFHux5BzCzq83sCTM7bmbHzOzjEccEef5Txh7s+Tezy83sH83saCf+z0YcE+S5v8Ddg/wBJoB/An4DeCtwFPitvmO2At+hvf3AJuCpsuMeIvbbgMfKjjUm/v8M3AQ8F/N8kOc9ZezBnvdOfKuBmzq//zLwfyv0332a2IM9/53zeUXn90ngKWBTFc599yfkEfrNwAvu/s/u/gvgEdo7O/YKdafHNLEHyyu8w2aK2IPm7i97534C7v5z4DgrN7sL8vynjD1YnfN5uvPnZOenv2skyHPfFXJCT7OLY+qdHscsbVy3dr7efcfMrh9PaLkI9bynVYnzbmbrgI20R4q9gj//A2KHgM+/mU2Y2RHgFeC77l6pcx/yLejS7OKYeqfHMUsT19PANe5+2sy2Avtp3ySkCkI972lU4ryb2RXAo8An3P2N/qcjXhLM+U+IPejz7+7ngA1mNg38rZnd4O698zFBn/uQR+hpdnEMdafHxLjc/Y3u1zt3fxyYNLNV4wsxk1DPe6IqnHczm6SdEL/u7vsiDgn2/CfFXoXzD+DuS8A/sPJubcGeewg7of8IuM7MrjWztwL30N7ZsdcB4EOdmedNpNzpcQwSYzezK83MOr/fTPvfxWtjj3Q0oZ73RKGf905sXwGOu/vnYw4L8vyniT3k829mM52ROWY2BbwTeL7vsCDPfVewJRd3P2tmHwMO0u4a+aq7HzOzezvPj7zTY9FSxn438BEzOwssA/d4Zxq9bFbQDpvjkCL2YM97x2bgg8CznVouwKeBtRD8+U8Te8jnfzXwP81sgvaF5lvu/lgVck6Xlv6LiNREyCUXEREZghK6iEhNKKGLiNSEErqISE0ooYuI1IQSuohITSihi4jUxP8HPoU6XiMZjmIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(df_data_heterosc['x'], df_data_heterosc['y'])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "b8906c78",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.12850679550432825"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = sm.OLS(df_data_heterosc['y'], df_data_heterosc[['constant','x']])\n",
    "results = model.fit()\n",
    "het_white(results.resid, df_data_heterosc[['constant','x']])[1] # no heteroscedasticity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "be7b1f18",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAYW0lEQVR4nO3dfYxcV3nH8d9je0LWvG0kb9uwwXWQUGhC1DhdRUGWUBJQkyYFohQJqErV/mO1aitKkSuHP5rwB7WlSLxUrdpaQAuCUgIJVpQAKZKTokYisMY2SXBSUV5KNqmytCyQeIm9ztM/dsZZj+/MPXfmvpxz7/cjWV7vzM4+e9fz3HOf85xzzd0FAIjXpqYDAACMR6IGgMiRqAEgciRqAIgciRoAIrelihfdtm2b79ixo4qXBoBWOnz48I/dfS7rsUoS9Y4dO7S4uFjFSwNAK5nZD0c9RukDACJHogaAyJGoASByJGoAiByJGgAiV0nXBwDE7uCRJd1x/xN6amVVr5qd0Z7rL9HNO+ebDisTiRpA5xw8sqRb735Eq6dOS5KWVlZ1692PSFKUyZrSB4DOueP+J84k6YHVU6d1x/1PNBTReCRqAJ3z1Mpqoc83jUQNoHNeNTtT6PNNI1ED6Jw911+imd7msz4309usPddf0lBE4zGZCKBzBhOGdH0AQMRu3jkfbWIeRukDACJHogaAyJGoASByJGoAiByJGgAiR6IGgMiRqAEgciRqAIgciRoAIkeiBoDIkagBIHJBidrMZs3sC2b2uJkdN7M3VB0YAGBd6KZMH5X0FXd/u5mdJ2lrhTEBADbITdRm9gpJb5T0B5Lk7iclnaw2LACpSOkmsakKKX28RtKypH8ysyNm9jEze+nwk8xst5ktmtni8vJy6YECiM/gJrFLK6tyvXiT2INHlpoOrVVCEvUWSVdK+nt33ynpOUl7h5/k7gfcfcHdF+bm5koOE0CMUrtJbKpCEvWTkp5094f7//6C1hM3gI5L7SaxqcpN1O7+P5J+ZGaDm4m9SdJ3Ko0KQBJSu0lsqkL7qP9M0mfM7NuSrpD015VFBCAZqd0kNlVB7XnuflTSQrWhAEhNajeJTRU3twUwlZRuEpsqlpADQORI1AAQORI1AESORA0AkSNRA0DkSNQAEDkSNQBEjj5qoAFsDYoiSNRAzQZbgw52nRtsDSqJZI1MJGqgZuO2BiVRF9OVKxMSNVAztgYtR5euTJhMBGrG1qDl6NJNC0jUQM3YGrQcXboyIVEDNbt557z23XK55mdnZJLmZ2e075bLW3e5XrUuXZlQowYaELo1aFcmyyax5/pLzqpRS+29MiFRA5Hq0mTZJLp00wISNRAp2vjydeWmBdSogUh1abIM45GogUh1abIM45GogUjRxocBatRApLo0WYbxghK1mf1A0s8lnZa05u4LVQYFYF1XJsswXpER9bXu/uPKIgEAZKJGDQCRCx1Ru6R/MzOX9I/ufmD4CWa2W9JuSdq+fXt5EQINY3UgmhaaqHe5+1Nm9kuSvmpmj7v71zY+oZ+8D0jSwsKClxwn0AhWByIGQaUPd3+q//czkr4o6aoqgwJi0aWtNBGv3ERtZi81s5cPPpb0m5IerTowIAasDkQMQkofvyzpi2Y2eP6/uPtXKo0KiMSrZme0lJGUWR2IOuWOqN39e+7+6/0/l7n7B+sIDIgBqwMRA1YmAmOwOhAxIFEDOVgdmJY2tlOSqAG0RlvbKVmZCKA12tpOSaIG0BptbackUQNojbbebIFEDZTk4JEl7dp/SBfvvU+79h/SwSNLTYfUOW1tp2QyEShBWyexUtPWdkoSNVAC7hgejza2U5KogSkMenazlplL6U9iIQ4kamBCw+WOLClMYrVxgUjbMJkITCir3DHsxMm1qCcVByebpZVVuV6srccccxeRqIEJhZQ1fnLiVNSJr60LRNqGRA1MKLSsEXPia+sCkbYhUQMTyurZHSXWxFdkgQh94s0hUQMTunnnvPbdcrnmZ2dkkuZnZ3TB1l7mc2OdVAxdIEItu1l0fQBTGO7ZzeoEiXllXOgCEfrEm0WiRmPa2BaW4sq4kAUi1LKbRaJGI8pech1T0m/jyjjuHdksatRoRJltYWXWT5kwy9bWzY5SwYi6o5oegZZ5KV1W/ZSNlUZLsaTTJiTqDoohIZV5KV1W0mfCbLw2lnRSEVz6MLPNZnbEzO6tMiBUL4bVaGVeSpe1WTwTZohVkRr1eyQdryoQ1CeGhJTVg7zvlssLj9gOHlnSc8+vnfP5SZJ+W+8OgvQFlT7M7CJJN0n6oKS/qDQiVC6WGfxpL6VH7V53wdaebnvLZYVfe8/1lyTVA43uCB1Rf0TSX0p6YdQTzGy3mS2a2eLy8nIZsaEibZnBH7V73dbzthSeRNy1/5De+7mjesmWTbpga2+qUT5QttwRtZn9tqRn3P2wmV0z6nnufkDSAUlaWFjwsgJE+doyg19GCWd4VL6yekozvc368DuuSO54oL1CSh+7JL3VzG6UdL6kV5jZp93996oNDVVqwwx+GSUcOj2QgtzSh7vf6u4XufsOSe+UdIgkjRiUUcKJYWIVyMPKRCSrjM4ROj2QgkILXtz9QUkPVhIJMIFpSzgpdno0vaoU9WNlIjottYnVGFaVon4kaiSjqpFkShOrsUx+MqqvF4kaSWAkuS6Gyc9UfhdtOpkwmYjKlLllaAz7k8QghsnPFH4Xbbt1GIkalSjyRglJ6DGMJGMQw6rSFH4XKZxMiiBRoxKhb5TQhB7DSDIGZW1mNY0UfhcpnEyKoEaNSoS+UUInx1Jso6tK05OfKfwuYtl4rCyMqFGJ0FFXaEKPYSSJdSn8LmIoEZWJETUqETrqKjLyaXokiRfF/rtIrT8+D4kalQh9o6RwGY00xX4yKYJEjTPK7jsNeaPEMPJpU78t2olEDUnNLmJocuSTyuINdBuTiZA0ed9pmYtamtC2flu0EyNqSJqs77QNo9FxPzclEcSCETUkTbaIoQ2j0VE/3ytnekELcVK/okAaSNSQNFnf6ajR6NLKajKJa9TPbabck1Db9pNAvEjUkDTZIoZxo+1UEteon3vlxKnM5288ObXhigJpoEaNM4p2X2T1QA+re6/kSerKWT/3Hfc/kbsQp237SSBejKgxseHR6Ch1Ja4ySxEhpaAUNidCO5CoMZWbd87rob3X6fv7b9J8w4mrzFJESCmobftJIF6UPirWpRavppeDl12KyCsFxbCqEt1Aoq5QG/qMi2g6cTWxtWWb9pNAvHJLH2Z2vpl9w8yOmdljZvaBOgJrA7oC6kUpAm0VMqJ+XtJ17v6smfUk/YeZfdndv15xbMnrWldA01cQTY/ogarkJmp3d0nP9v/Z6//xKoNqi7bdZSJP6N1aBpqo33dpzgDtEdT1YWabzeyopGckfdXdH854zm4zWzSzxeXl5ZLDTFPXLsWLXEFUsaov7zVZSYhUBSVqdz/t7ldIukjSVWb2+oznHHD3BXdfmJubKznMNNV1y6JY9pso0ldcRf0+7zWZM0CqCnV9uPuKmT0o6QZJj1YSUctU3RXQdF14oyLteVXU7/Nes2tzBmiPkK6POTOb7X88I+nNkh6vOC4EimmUWOQKoopVfbNbe2Nfk5WESFXIiPpCSZ80s81aT+x3uvu91YaFULGNEkOvIMpeHHPwyJKe/cVa5mMnTq7p4JGlxhfkAJMK6fr4tqSdNcSCCaTaWVJ2K90d9z+hUy9kNyP95MQp3Xr3I9p3y+Xad8vllXV9ZHWUDGKjywTTsPXuu3ItLCz44uJi6a+Lcw3XqKX1UWIVk5Yxu3jvfbk9o/OzM3po73WVfP+s30Nvk0kmnTr9YmRd/N0gjJkddveFrMdYQp64aUembekrHnVlsVGV5aCsuYKsEX7d276iHUjUEZg2WU7aWRJTx8i0QvbGrrIcVOQkUPSE0ZaTKSZHom7YNMly2jdw0ZWEMdt4ZbG0sirT2ctnq540DBnRb3xuqDadTDE59qNu0MEjS3rfnccmaq8rY5VdbB0j0xrsjf2D/Tfpw++4ovKFRhtlrULtbTL1Np99S4WiJ4yY2i/RHEbUDRkk2tMjJnOXVlZ18MjSyORSxmg4r2Mk5UvuOrcfHRyn1VOntdlMp901X1LXR9tOppgMibohWYl22LhL3DLewOP6ipu+5E7lJDF8nE67nzmGg3iniTvV9kuUi9JHQ0IS6rhL3DJW2Y1bSdjkJXfI5kox7G0iVV+a6NrGXsjGiLohoZNPoxJ6WavsRpUImrzkzkt+ez5/7Ezr29LKqvZ8/pik4iPXMhaoVH2c2GMbEom6MSHtZNLoEXLVb+AmL7nHJb/b73nsnP7kUy+4br/nsUI/e1ZpZ8/nj521QCWk3FPHceJ2XyBRN2Q40b5ypqfnTq6ds4pt3Ai5zDfw8Ojy2tfN6a7DS7XuizGIYdQKw3FXISurp855nXEnsLIWqLB/COpAom7QcKJtagIta3R51+El/c5vzOuBx5driSdrCfZGM73NuvZ1c/r01/+70OuMGhWXtUCF0gTqQKKOSFOXuKNqwg88vlzZ3hjS2SemTf22tiyDVrdxE3QX9Lc4DW1bLHOBCqUJVI2uj5YL6ZBoYuJwuLNjVJI2SQ/tvU4375wfG89tb7lMkkYm3+HPV7VABagCI+oIlVUCCS0DNDFxGNJHPhzDqDhnZ3pnfp5NJmXtdrrp7Pw7smQhSbff89iZmvf5PcYyaB7/CyNT5g1YQ3t8m+jVDRmtD8cwKs7b33rZmX+P2JI68/ODJeff33/TmVG7JD2/9sKZ5wz2suYGuGgSiToyZS6gCC1p1HUT3o1GjdY3m42MoY442VsDMaL0UbFBGWNpZfWcfSAmXWgSWhopUtKoe0JsVFtbXuLNi3N2pndWq97Gz2cZPpajatzsrYEmRTOijmlZcFk2ljGkFyfMxpUz8paGFymNxLz8uKrR8e1vvWz9ziob9DbZWeWRgaxjaec8ax17a6BJUdyKK9XbSeWNbHftPzS2BSzr1lDjjoUkve/OY5kdEqNuM5XK5kZlCv2ZR/1+svayjv3/ItIX/a24UtzAPqSjIu9yOevxcd0I47ZFHfW9utjjG/ozjzpmrvUTX5dObohbFIk6xT13Q04ueYsqxu3jMZwYdu0/1NhtplIzbQ2/ypvgApPIrVGb2avN7AEzO25mj5nZe8oOoowtO+sWcnLJqhEPFK0VjztpxVJ3jkFbavjARiGTiWuS3ufuvybpakl/YmaXlhlErG+YcROcISeXjRNm0nrrmTTZxNm4djbqpy8q0l7XRFsiMInc0oe7Py3p6f7HPzez45LmJX2nrCBi3NgmrwYdumtaWTXiSdvZuqZoGa2LNXykp1CN2sx2SNop6eGMx3ZL2i1J27dvLxxIbG+YvBp03SeXGE9mMeLWVWij4PY8M3uZpH+X9EF3v3vcc4u258Xo4r33Ze6LbJK+v/+musNBoFRbPYFx7XlBC17MrCfpLkmfyUvSbZHiBCeoO6OdcksfZmaSPi7puLt/qPqQ6jWqlavqO3dMshBl2sUrXVn8ElsZDZhWSI16l6R3S3rEzI72P/d+d/9SZVHVJGTRShWJLXT70Wm/psyvb1oTJ5munNgQvyiWkDdl1BLi4QUPZb9hQ7/vtF9T5tc3qYm6M7Vu1G3qGnVbhe5UV9b+0EW+bxlfU+bXN6mJrUfZ7hQxiWIJeVNCWrnK2IdkeEQ+u7Wnn5w4dyvOcROV07adpdy21sRJJuUTG9qn0yPqkBWR075hs0bkz/5irfC9+aZdvRnr6s8QTXTg0PWDmHQ6UYe0ck37hs0akZ96wfXS87YUaiGbtO1ssAz+vZ87qpds2aQLtvaSa1tr4iRT5fds497rqFanJxNDTDup1OTCmby9rVPqaGhL1weTlBhl3GQiiTrANG/YJrstRn3v2Zmenl97gWTRgJS7b5rW9nbJ6G8cEItR/xGmWUBR9cKZcUbV0bPuKRj7jRragknKyaS+DmBa0deo66rnVdGGJzW7pLnoxBfJonpMUk6m6+2SUY+oi55Fp7k0qvJ2YE0tac4azY9Dsqhek1dYKev6lUjUibpI8pz20qiN/xE2LoMfd0swaXyyaHttsE5sVzuZlNcBlCHqRF0keU46Ih4koVFTqq+c6WnX/kNJvamyEuu4ZD0/5ufqem2wCmwaVVzXr0SirlEXqedNMiLeWJfO0ttkeu7kWul16yqNqrVf+7q5zL7gj7zjCj2097qRiaPrtUHEoevb10Y9oi5yFp3k0igrCQ3Mz87oxMm1c5Z6x94dMSqxPvD4svbdcnnhS+42loSQpi5fiUSdqIvU8ya5NBqVbEzSQ3uv08V77yv0dTEYl1gn+Y/e9dogEIOoE7UUfhadZJImLwmlmKTKjrnrtUEgBtEn6iKKjhjzklCKSarsmOlSAJrXqkRdVF4SaipJTdMOV0XMXa4NAjFgr4/ItHXTHnqxgfGS3euji2/uKldINoVebGA60fZRV7X3Ruza2A5HLzYwnWgTdVff3G3ctKeNJx+gTtEm6q6+uVO+ZdYoTZ18uJMK2iI3UZvZJ8zsGTN7tI6ABto4sgzRxqWyTZx8ulo6QzOqHhSETCb+s6S/lfSpUr9zjhR7mMvStna4Jtoc2zgpizjVMVmem6jd/WtmtqOU71YACy3ape6TT1dLZ6hfHYOC0trzzGy3pN2StH379lJes20jS9QnxeX/SFMdg4LSJhPd/YC7L7j7wtzcXFkvC0ykjZOyiFMd82nRdn0A02jjpCziVMegIOqVicA0KJ2hDnXMp+UmajP7rKRrJG0zsycl3ebuHy8tAgBIXNWDgpCuj3dV9t0BALmoUQNA5EjUABA5JhOBKXRxK17Uj0QNTIh9tlEXSh/AhLq6FS/qR6IGJsR+IqgLiRqYUFe34kX9SNTAhNhPBHVhMhGYEFvxoi4kamAK7CeCOlD6AIDIkagBIHIkagCIHIkaACJHogaAyNH1AQRg8yU0iUQN5GDzJTSN0geQg82X0DQSNZCDzZfQNBI1kIPNl9A0EjWQg82X0DQmE4EcbL6EpgUlajO7QdJHJW2W9DF3319pVEBk2HwJTcotfZjZZkl/J+m3JF0q6V1mdmnVgQEA1oXUqK+S9F13/567n5T0r5LeVm1YAICBkEQ9L+lHG/79ZP9zZzGz3Wa2aGaLy8vLZcUHAJ0Xkqgt43N+zifcD7j7grsvzM3NTR8ZAEBSWKJ+UtKrN/z7IklPVRMOAGCYuZ8zOD77CWZbJP2npDdJWpL0TUm/6+6PjfmaZUk/nCCebZJ+PMHXxSLl+FOOXUo7/pRjl9KOP6bYf9XdM8sRue157r5mZn8q6X6tt+d9YlyS7n/NRLUPM1t094VJvjYGKcefcuxS2vGnHLuUdvypxB7UR+3uX5L0pYpjAQBkYAk5AEQutkR9oOkAppRy/CnHLqUdf8qxS2nHn0TsuZOJAIBmxTaiBgAMIVEDQOQaSdRmdoOZPWFm3zWzvRmPm5n9Tf/xb5vZlU3EmSUg9mvM7KdmdrT/56+aiDOLmX3CzJ4xs0dHPB7tcZeC4o/52L/azB4ws+Nm9piZvSfjOVEe/8DYYz7255vZN8zsWD/+D2Q8J8pjf4a71/pH673Y/yXpNZLOk3RM0qVDz7lR0pe1vnz9akkP1x3nFLFfI+nepmMdEf8bJV0p6dERj0d53AvEH/Oxv1DSlf2PX671RWSp/L8PiT3mY2+SXtb/uCfpYUlXp3DsB3+aGFGH7Mb3Nkmf8nVflzRrZhfWHWiGpHcSdPevSfq/MU+J9bhLCoo/Wu7+tLt/q//xzyUd17mbm0V5/ANjj1b/eD7b/2ev/2e4iyLKYz/QRKIO2Y0vaMe+BoTG9Yb+ZdaXzeyyekIrRazHvYjoj72Z7ZC0U+sju42iP/5jYpciPvZmttnMjkp6RtJX3T2pY9/ErbhCduML2rGvASFxfUvra/afNbMbJR2U9NqqAytJrMc9VPTH3sxeJukuSX/u7j8bfjjjS6I5/jmxR33s3f20pCvMbFbSF83s9e6+ca4j6mPfxIg6ZDe+WHfsy43L3X82uMzy9aX3PTPbVl+IU4n1uAeJ/dibWU/rie4z7n53xlOiPf55scd+7AfcfUXSg5JuGHoo2mMvNZOovynptWZ2sZmdJ+mdku4Zes49kn6/PxN7taSfuvvTdQeaITd2M/sVM7P+x1dp/Rj/b+2RTibW4x4k5mPfj+vjko67+4dGPC3K4x8Se+THfq4/kpaZzUh6s6THh54W5bEfqL304SN24zOzP+o//g9a3wDqRknflXRC0h/WHWeWwNjfLumPzWxN0qqkd3p/WrlpZvZZrc/ObzOzJyXdpvWJlaiP+0BA/NEee0m7JL1b0iP9WqkkvV/Sdin64x8Se8zH/kJJn7T1+79uknSnu9+bQs4ZYAk5AESOlYkAEDkSNQBEjkQNAJEjUQNA5EjUABA5EjUARI5EDQCR+385kQdgo6n4EgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(df_data_heterosc['x'], df_data_heterosc['y_2'])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "5d793d8e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.9776629134087167e-07"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = sm.OLS(df_data_heterosc['y_2'], df_data_heterosc[['constant','x']])\n",
    "results = model.fit()\n",
    "het_white(results.resid, df_data_heterosc[['constant','x']])[1] # heteroscedasticity!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "078683ec",
   "metadata": {},
   "source": [
    "If heteroscedasticity, the estimator is not necessarily the best (Gauss-Markov conditions are not satistifed). Indeed, the <font color='red'>estimator does not have the lowest variance</font> anymore. In other words, the estimator becomes quite unstable.\n",
    "\n",
    "Below is an illustration of the estimator unstability when changing the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "cd218a74",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "run 0: coefficient=1.133651791040297\n",
      "run 1: coefficient=1.0049504400112035\n",
      "run 2: coefficient=0.8995029371979437\n"
     ]
    }
   ],
   "source": [
    "for i in range(3):\n",
    "    np.random.seed(i)\n",
    "    df_data_heterosc = pd.DataFrame()\n",
    "    df_data_heterosc['constant'] = np.ones(data_rotated.shape[0])\n",
    "    df_data_heterosc['x'] = data_rotated[:,0]\n",
    "    df_data_heterosc['y'] = data_rotated[:,1]\n",
    "    df_data_heterosc = df_data_heterosc.sort_values(by='x').reset_index()\n",
    "    df_data_heterosc['y_2'] = df_data_heterosc\\\n",
    "                        .apply(lambda x: x['y']+x.name**1.1/100*np.random.normal(0,1), axis=1)\n",
    "    model = sm.OLS(df_data_heterosc['y_2'], df_data_heterosc[['constant','x']])\n",
    "    results = model.fit()\n",
    "    het_white(results.resid, df_data_heterosc[['constant','x']])[1] # heteroscedasticity!\n",
    "    print('run {}: coefficient={}'.format(i,results.params[1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "faf3b0cb",
   "metadata": {},
   "source": [
    "Identify heteroscedasticity:\n",
    "- Plot independent variables against dependent variables and see whether there is a trend\n",
    "- Run heteroscedasticity test"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9a25bbf",
   "metadata": {},
   "source": [
    "##### Assumption 2: no correlation between the error and the exploratory variables (exogeneity)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f6b6424",
   "metadata": {},
   "source": [
    "$$\\mathbb{E}[\\epsilon | X]=0$$\n",
    "\n",
    "This means that the errors do not depend on X. This assumption is needed to make sure there is no important variable missing that may also explain the relationship. \n",
    "\n",
    "Put it differently, Y has no effect on X (?)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e421b9e",
   "metadata": {},
   "source": [
    "Endogeneity: \n",
    "- important variable is omitted\n",
    "- omitted variable is correlated with one or several explanatory variables\n",
    "- omitted variable is correlated with the dependent variable (= contributes to explain the result)\n",
    "\n",
    "=> <font color='red'>estimator is biased</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f07bf4e",
   "metadata": {},
   "source": [
    "We consider the following true (linear) model that gives the price of a car:\n",
    "\n",
    "$$price = 20000-2000*Age-0.01*Kms$$\n",
    "\n",
    "Other interesting example: https://statisticsbyjim.com/regression/confounding-variables-bias/ (bone density explained by activity and weights)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3c93502",
   "metadata": {},
   "source": [
    "<img src='omitted-variable.png' width='300px'></img>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48f9ff8f",
   "metadata": {},
   "source": [
    "The omitted variable is the Age."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "f0e2194d",
   "metadata": {},
   "outputs": [],
   "source": [
    "data_population = pd.DataFrame(data={\n",
    "    'price':[8000,10000,10500,12500,13000,15000,15500,17500],\n",
    "    'age':[5,4,4,3,3,2,2,1],\n",
    "    'kms':[200000,200000,150000,150000,100000,100000,50000,50000],\n",
    "    'rdm':[0,0,0,0,0,0,0,0]\n",
    "})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "3551e95b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEICAYAAABfz4NwAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAXdElEQVR4nO3df5Bdd3nf8fcnkmIWHFvEEsRaGeQGo4nNL8HGNXVIHZxWTkuxypBUHRIb6omKx5O20IpYpQEyaYsdkTjjpECc4LFNgn+UCOFCjKA4rSFj7FkjiGwTBRFjWyuDBUbGLhtXEk//uN91r1Z3pd3Vau+u9v2auaNzn3POvc892r2fe873nLupKiRJ+pF+NyBJmhsMBEkSYCBIkhoDQZIEGAiSpMZAkCQBBoL0rCQXJNl9DOt/OMlvzGRP0mwyEKRpSPLWJF/srlXV26vqt/rVE0CS30qyI8mBJO87yrJJcnWS77bbbyfJLLWqOchA0AkpyeLJ1E5Au4B3AZ+exLIbgHXAK4FXAG8A/vVx60xznoGgOSnJGUm2JNnbPr3+QZIfSfKfkjyc5PEkNyU5tS2/KkkluSzJI8Cd7VP8Xya5JskTwPuSnJTkA0keSfLtdphnYIIerkzyjSRPJXkwyT9v9Z8CPgy8NsnTSfa1+g1J/nPX+r+aZFeSJ5LcnmRF17xK8vYkX0/yvST/bSY+nVfVjVV1B/DUJBa/FPidqtpdVSPA7wBvPdYeNH8ZCJpzkiwCPgU8DKwCBoFb6LxZvRX4OeDvAScDfzBu9X8I/BSwtt3/+8DfAi8A/gtwNfBS4FXAS9pjv2eCVr4BvA44FfhN4E+SnF5VXwPeDtxdVSdX1dIer+H1wPuBXwJOb6/llnGLvQH4aTqf0H9prOckP5Nk3xFuPzNBv1N1DvDVrvtfbTUtUAaC5qJzgRXAxqr6P1X1d1X1ReAtwO9W1d9W1dPAJmD9uENB72vrjLb7e6rq96vqAPB3wK8C76iqJ6rqKeC/Aut7NVFV/72q9lTVD6vqVuDrrbfJeAtwfVV9uaqeab2+NsmqrmWuqqp9VfUI8Bd0Qoqq+mJVLT3C7Yvjn2yaTgae7Lr/JHCy4wgLl4GguegM4OH2Jt5tBZ1P2mMeBhYDL+yqPTpune77y4HnAveNfdoGPtPqh0lySZKvdC37MmDZJF/DIb22APsunT2SMd/qmv4BnTfoSUvyQDtk9XSS101l3eZp4JSu+6cAT5ffeLlgLYRBNs0/jwIvSrJ4XCjsAV7cdf9FwAHg28DKVhv/ZtZ9/zvAKHBOO2Y+oSQvBv4IuJDOoaGDSb4CjH16Ptqb5iG9JnkecBpwxOdty74OuOMIi/xCVX2hqo718M4DdA5X3dvuv7LVtEC5h6C56F7gMeCqJM9L8pwk5wM3A+9IcmaSk+kc7rm1x55ET1X1Qzpv8tckeQFAksEka3ss/jw6b/p723Jvo7OHMObbwMokPzrB030MeFuSVyU5qfV6T1V9cxJ9fqGNTUx0+8JE6yZZkuQ5dH63F7dtt2iCxW8C3tm2wQrg3wM3HK0/nbgMBM05VXUQ+Gd0Bn0fAXYD/wK4HvgocBfwEJ0xgV+b4sP/Op1TM7+U5PvA/wRW9+jhQTpn3dxN583/5cBfdi1yJ51P099K8p0e638e+A3gz+iE208ywVjFDPsjOntB/xJ4d5v+FejseSR5umvZPwT+B7ADuJ/Oqap/OAs9ao6KhwslSeAegiSpMRAkSYCBIElqDARJEjCPr0NYtmxZrVq1qt9tSNK8ct99932nqnpejDlvA2HVqlUMDw/3uw1JmleSPDzRPA8ZSZIAA0GS1BgIkiTAQJAkNQaCJAmYx2cZTcfW7SNs3raTPftGWbF0gI1rV7NuzeDRV5SkBWDBBMLW7SNs2rKD0f0HARjZN8qmLTsADAVJYgEdMtq8beezYTBmdP9BNm/b2aeOJGluWTCBsGff6JTqkrTQLJhAWLF0YEp1SVpoFkwgbFy7moElh/4lwYEli9i49rA/liVJC9KCGVQeGzj2LCNJ6m3BBAJ0QsEAkKTeFswhI0nSkRkIkiTAQJAkNQaCJAkwECRJjYEgSQIMBElSYyBIkgADQZLUGAiSJMBAkCQ1BoIkCTAQJEmNgSBJAgwESVJjIEiSAANBktQYCJIkwECQJDUGgiQJMBAkSY2BIEkCDARJUnPUQEhyfZLHk9w/rv5rSXYmeSDJb3fVNyXZ1eat7aq/JsmONu/aJGn1k5Lc2ur3JFk1g69PkjRJk9lDuAG4qLuQ5OeAi4FXVNU5wAda/WxgPXBOW+eDSRa11T4EbADOarexx7wM+F5VvQS4Brj6GF6PJGmajhoIVXUX8MS48uXAVVX1TFvm8Va/GLilqp6pqoeAXcC5SU4HTqmqu6uqgJuAdV3r3NimPw5cOLb3IEmaPdMdQ3gp8Lp2iOd/J/npVh8EHu1abnerDbbp8fVD1qmqA8CTwGm9njTJhiTDSYb37t07zdYlSb1MNxAWA88HzgM2Are1T/W9PtnXEeocZd6hxarrqmqoqoaWL18+9a4lSROabiDsBrZUx73AD4FlrX5G13IrgT2tvrJHne51kiwGTuXwQ1SSpONsuoGwFXg9QJKXAj8KfAe4HVjfzhw6k87g8b1V9RjwVJLz2p7EJcAn22PdDlzapt8M3NnGGSRJs2jx0RZIcjNwAbAsyW7gvcD1wPXtVNT/C1za3sQfSHIb8CBwALiiqg62h7qczhlLA8Ad7QbwEeCjSXbR2TNYPzMvTZI0FZmvH8aHhoZqeHi4321I0ryS5L6qGuo1zyuVJUmAgSBJao46hqCFa+v2ETZv28mefaOsWDrAxrWrWbdm8OgrSpqXDAT1tHX7CJu27GB0f+ecgJF9o2zasgPAUJBOUB4yUk+bt+18NgzGjO4/yOZtO/vUkaTjzUBQT3v2jU6pLmn+MxDU04qlA1OqS5r/DAT1tHHtagaWLDqkNrBkERvXru5TR5KONweV1dPYwLFnGUkLh4GgCa1bM2gASAuIh4wkSYCBIElqPGSkCXmlsrSwGAjqySuVpYXHQ0bqySuVpYXHQFBPXqksLTwGgnrySmVp4TEQ1JNXKksLj4PK6skrlaWFx0DQhLxSWVpYPGQkSQIMBElSYyBIkgADQZLUGAiSJMBAkCQ1BoIkCTAQJEmNgSBJAgwESVJjIEiSAANBktQYCJIkYBKBkOT6JI8nub/HvP+QpJIs66ptSrIryc4ka7vqr0myo827Nkla/aQkt7b6PUlWzdBrkyRNwWT2EG4ALhpfTHIG8I+AR7pqZwPrgXPaOh9MMvZXVj4EbADOarexx7wM+F5VvQS4Brh6Oi9EknRsjhoIVXUX8ESPWdcA7wKqq3YxcEtVPVNVDwG7gHOTnA6cUlV3V1UBNwHruta5sU1/HLhwbO9BkjR7pjWGkOSNwEhVfXXcrEHg0a77u1ttsE2Prx+yTlUdAJ4ETpvgeTckGU4yvHfv3um0LkmawJQDIclzgXcD7+k1u0etjlA/0jqHF6uuq6qhqhpavnz5ZNqVJE3SdPYQfhI4E/hqkm8CK4EvJ/kJOp/8z+hadiWwp9VX9qjTvU6SxcCp9D5EJUk6jqYcCFW1o6peUFWrqmoVnTf0V1fVt4DbgfXtzKEz6Qwe31tVjwFPJTmvjQ9cAnyyPeTtwKVt+s3AnW2cQZI0iyZz2unNwN3A6iS7k1w20bJV9QBwG/Ag8Bngiqo62GZfDvwxnYHmbwB3tPpHgNOS7ALeCVw5zdciSToGma8fxoeGhmp4eLjfbUjSvJLkvqoa6jXPK5UlSYCBIElqDARJEmAgSJIaA0GSBBgIkqTGQJAkAQaCJKkxECRJgIEgSWoMBEkSYCBIkprF/W5AOlFs3T7C5m072bNvlBVLB9i4djXr1gwefUVpjjAQpBmwdfsIm7bsYHR/59veR/aNsmnLDgBDQfOGh4ykGbB5285nw2DM6P6DbN62s08dSVNnIEgzYM++0SnVpbnIQJBmwIqlA1OqS3ORgSDNgI1rVzOwZNEhtYEli9i4dnWfOpKmzkFlaQaMDRx7lpHmMwNBmiHr1gwaAJrXPGQkSQIMBElS4yEjaYZ4pbLmOwNBmgFeqawTgYeMpBnglco6ERgI0gzwSmWdCAwEaQZ4pbJOBAaCNAO8UlknAgeVpRnglco6ERgI0gzxSmXNdx4ykiQBBoIkqTEQJEnAJAIhyfVJHk9yf1dtc5K/TvJXST6RZGnXvE1JdiXZmWRtV/01SXa0edcmSauflOTWVr8nyaqZfYmSpMmYzB7CDcBF42qfA15WVa8A/gbYBJDkbGA9cE5b54NJxs7F+xCwATir3cYe8zLge1X1EuAa4OrpvhhJ0vQdNRCq6i7giXG1z1bVgXb3S8DKNn0xcEtVPVNVDwG7gHOTnA6cUlV3V1UBNwHruta5sU1/HLhwbO9BkjR7ZmIM4V8Bd7TpQeDRrnm7W22wTY+vH7JOC5kngdN6PVGSDUmGkwzv3bt3BlqXJI05pkBI8m7gAPCnY6Uei9UR6kda5/Bi1XVVNVRVQ8uXL59qu5KkI5h2ICS5FHgD8JZ2GAg6n/zP6FpsJbCn1Vf2qB+yTpLFwKmMO0QlSTr+phUISS4Cfh14Y1X9oGvW7cD6dubQmXQGj++tqseAp5Kc18YHLgE+2bXOpW36zcCdXQEjSZolR/3qiiQ3AxcAy5LsBt5L56yik4DPtfHfL1XV26vqgSS3AQ/SOZR0RVWNfUn85XTOWBqgM+YwNu7wEeCjSXbR2TNYPzMvTZI0FZmvH8aHhoZqeHi4321I0ryS5L6qGuo1zyuVJUmAgSBJagwESRJgIEiSGgNBkgQYCJKkxkCQJAEGgiSpMRAkSYCBIElqDARJEmAgSJIaA0GSBBgIkqTGQJAkAQaCJKkxECRJgIEgSWoMBEkSYCBIkhoDQZIEGAiSpGZxvxuQtHBt3T7C5m072bNvlBVLB9i4djXr1gz2u60Fy0CQ1Bdbt4+wacsORvcfBGBk3yibtuwAMBT6xENGkvpi87adz4bBmNH9B9m8bWefOpKBIKkv9uwbnVJdx5+BIKkvViwdmFJdx5+BIKkvNq5dzcCSRYfUBpYsYuPa1X3qSA4qS+qLsYFjzzKaOwwESX2zbs2gATCHeMhIkgQYCJKk5qiBkOT6JI8nub+r9uNJPpfk6+3f53fN25RkV5KdSdZ21V+TZEebd22StPpJSW5t9XuSrJrh1yhpjtq6fYTzr7qTM6/8NOdfdSdbt4/0u6UFbTJ7CDcAF42rXQl8vqrOAj7f7pPkbGA9cE5b54NJxk4j+BCwATir3cYe8zLge1X1EuAa4OrpvhhJ88fYlcoj+0Yp/v+VyoZC/xw1EKrqLuCJceWLgRvb9I3Auq76LVX1TFU9BOwCzk1yOnBKVd1dVQXcNG6dscf6OHDh2N6DpBOXVyrPPdMdQ3hhVT0G0P59QasPAo92Lbe71Qbb9Pj6IetU1QHgSeC0Xk+aZEOS4STDe/funWbrkuYCr1See2Z6ULnXJ/s6Qv1I6xxerLquqoaqamj58uXTbFHSXOCVynPPdAPh2+0wEO3fx1t9N3BG13IrgT2tvrJH/ZB1kiwGTuXwQ1SSTjBeqTz3TDcQbgcubdOXAp/sqq9vZw6dSWfw+N52WOmpJOe18YFLxq0z9lhvBu5s4wySTmDr1gzy/je9nMGlAwQYXDrA+9/0ci9U66OjXqmc5GbgAmBZkt3Ae4GrgNuSXAY8AvwiQFU9kOQ24EHgAHBFVY2NGl1O54ylAeCOdgP4CPDRJLvo7Bmsn5FXJmnO80rluSXz9cP40NBQDQ8P97sNSZpXktxXVUO95nmlsiQJMBAkSY2BIEkCDARJUmMgSJIAA0GS1BgIkiTAQJAkNQaCJAkwECRJjYEgSQIMBElSYyBIkgADQZLUGAiSJMBAkCQ1BoIkCTAQJEmNgSBJAgwESVJjIEiSAANBktQYCJIkwECQJDUGgiQJMBAkSY2BIEkCDARJUmMgSJIAA0GS1BgIkiTAQJAkNQaCJAk4xkBI8o4kDyS5P8nNSZ6T5MeTfC7J19u/z+9aflOSXUl2JlnbVX9Nkh1t3rVJcix9SdKJaOv2Ec6/6k7OvPLTnH/VnWzdPjKjjz/tQEgyCPwbYKiqXgYsAtYDVwKfr6qzgM+3+yQ5u80/B7gI+GCSRe3hPgRsAM5qt4um25cknYi2bh9h05YdjOwbpYCRfaNs2rJjRkPhWA8ZLQYGkiwGngvsAS4GbmzzbwTWtemLgVuq6pmqegjYBZyb5HTglKq6u6oKuKlrHUkSsHnbTkb3HzykNrr/IJu37Zyx55h2IFTVCPAB4BHgMeDJqvos8MKqeqwt8xjwgrbKIPBo10PsbrXBNj2+fpgkG5IMJxneu3fvdFuXpHlnz77RKdWn41gOGT2fzqf+M4EVwPOS/PKRVulRqyPUDy9WXVdVQ1U1tHz58qm2LEnz1oqlA1OqT8exHDL6eeChqtpbVfuBLcA/AL7dDgPR/n28Lb8bOKNr/ZV0DjHtbtPj65KkZuPa1QwsWXRIbWDJIjauXT1jz3EsgfAIcF6S57azgi4EvgbcDlzalrkU+GSbvh1Yn+SkJGfSGTy+tx1WeirJee1xLulaR5IErFszyPvf9HIGlw4QYHDpAO9/08tZt6bnEfZpWTzdFavqniQfB74MHAC2A9cBJwO3JbmMTmj8Ylv+gSS3AQ+25a+oqrERksuBG4AB4I52kyR1WbdmcEYDYLx0TuyZf4aGhmp4eLjfbUjSvJLkvqoa6jXPK5UlSYCBIElqDARJEmAgSJKaeTuonGQv8PA0V18GfGcG25kp9jU19jV1c7U3+5qaY+nrxVXV88reeRsIxyLJ8ESj7P1kX1NjX1M3V3uzr6k5Xn15yEiSBBgIkqRmoQbCdf1uYAL2NTX2NXVztTf7mprj0teCHEOQJB1uoe4hSJLGMRAkScAJHAhJrk/yeJL7J5ifJNcm2ZXkr5K8eo70dUGSJ5N8pd3eM0t9nZHkL5J8LckDSf5tj2VmfZtNsq9Z32ZJnpPk3iRfbX39Zo9l+rG9JtNXX37G2nMvSrI9yad6zOvL7+Qk+urX7+Q3k+xoz3nYN3kel+1VVSfkDfhZ4NXA/RPM/yd0vmY7wHnAPXOkrwuAT/Vhe50OvLpN/xjwN8DZ/d5mk+xr1rdZ2wYnt+klwD3AeXNge02mr778jLXnfifwsV7P36/fyUn01a/fyW8Cy44wf8a31wm7h1BVdwFPHGGRi4GbquNLwNKxv/TW5776oqoeq6ovt+mn6Pyxo/FfvD7r22ySfc26tg2ebneXtNv4MzT6sb0m01dfJFkJ/FPgjydYpC+/k5Poa66a8e11wgbCJAwCj3bd380ceKNpXtt2+e9Ics5sP3mSVcAaOp8uu/V1mx2hL+jDNmuHGb5C58/Efq6q5sT2mkRf0J+fsd8D3gX8cIL5/fr5+j2O3Bf0Z3sV8Nkk9yXZ0GP+jG+vhRwI6VGbC5+kvkznu0ZeCfw+sHU2nzzJycCfAf+uqr4/fnaPVWZlmx2lr75ss6o6WFWvovN3wM9N8rJxi/Rle02ir1nfXkneADxeVfcdabEeteO6vSbZV79+J8+vqlcDvwBckeRnx82f8e21kANhN3BG1/2VwJ4+9fKsqvr+2C5/Vf05sCTJstl47iRL6Lzp/mlVbemxSF+22dH66uc2a8+5D/hfwEXjZvX1Z2yivvq0vc4H3pjkm8AtwOuT/Mm4ZfqxvY7aV79+vqpqT/v3ceATwLnjFpnx7bWQA+F24JI2Un8e8GRVPdbvppL8RJK06XPp/B99dxaeN8BHgK9V1e9OsNisb7PJ9NWPbZZkeZKlbXoA+Hngr8ct1o/tddS++rG9qmpTVa2sqlXAeuDOqvrlcYvN+vaaTF99+vl6XpIfG5sG/jEw/szEGd9ei49l5bksyc10zg5YlmQ38F46A2xU1YeBP6czSr8L+AHwtjnS15uBy5McAEaB9dVOKTjOzgd+BdjRjj8D/EfgRV299WObTaavfmyz04Ebkyyi8wZxW1V9Ksnbu/rqx/aaTF/9+hk7zBzYXpPpqx/b64XAJ1oOLQY+VlWfOd7by6+ukCQBC/uQkSSpi4EgSQIMBElSYyBIkgADQZLUGAiSJMBAkCQ1/w/GVkLWrmssTQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(data_population['age'],data_population['price'])\n",
    "plt.title('correlation={}'.format(round(np.corrcoef(x=data_population['age'], y=data_population['price'])[0][1],2)))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "26b551c2",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_population = data_population[['age','kms']]\n",
    "y_population = data_population['price']\n",
    "lr_true = LinearRegression().fit(X_population,y_population)\n",
    "error = y_population-\\\n",
    "    (lr_true.intercept_ +\\\n",
    "     lr_true.coef_[0]*X_population['age'] +\\\n",
    "     lr_true.coef_[1]*X_population['kms']) # \"true\" error"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "713e4dc1",
   "metadata": {},
   "outputs": [],
   "source": [
    "data_sample = data_population[data_population.index.isin([0,1,2,4,6])]\n",
    "X_sample = data_sample[['kms','age']]\n",
    "y_sample = data_sample['price']\n",
    "lr_sample = LinearRegression().fit(np.array(X_sample['kms']).reshape(-1, 1),y_sample)\n",
    "residual = y_sample-\\\n",
    "    (lr_sample.intercept_ +\\\n",
    "     lr_sample.coef_[0]*X_sample['kms'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "09eb10cb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEICAYAAABbOlNNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAZKUlEQVR4nO3dfZQldX3n8fcnjPgAPgww4IDgKItGjCtqByXqriiozHEdYJcNHMXR1XDcDR7haM7igUSSNbuIT9ldjTpG1jESVOTxABFh4i6QDWAP4WFwJMMzw0yGVqNoNGuA7/5R1Xir7TvdTfVM34H365x7btWvfr+q79w7937urarblapCkqRJv7bQBUiSRovBIEnqMBgkSR0GgySpw2CQJHUYDJKkDoNBT0hJXpdkY4/xn0vy+/NZkzQqDAZpBknemeSawbaqem9V/ZeFqgkgybIk307ysyTfS3LYVvr+ZZKfDtx+keSWdtl+U5b9NEkl+cD2+9dolBgM2uElWTSbtsehc4C/BXYHTgW+kWTJdB2r6oiq2nXyBvxf4Nx22b1Tlr0EeAQ4b7v8KzRyDAYtuCT7Jjk/yUSSHyT5dJJfS3JaknuSPJDky0me2fZf1n6ifXeSe4G/aj/V/3WSTyX5IXB6kicn+XiSe5NsaXf/PHVIDackuSPJT5J8N8lRbfuLgM8Bh7SfpH/Utn8pyUcGxv9OktuT/DDJxUn2HlhWSd6bZEOSf0jymSTp+Zi9AHg58OGq+nlVnQfcAvzbWYxdBrwW+PMhXd4BXFVVd/epUTsug0ELKslOwCXAPcAyYB/gq8A729uhwPOBXYFPTxn+r4EXAW9q518J3AnsCfwx8FHgBcBBwL9o1/0HQ0q5g+bN8pnAHwJfSbK0qtYD7wX+pv1E/axp/g2vB/4b8O+Bpe2/5atTur0F+E3gpW2/N7VjX5PkR1u5vWZIvS8G7qyqnwy03dS2z+QdwNVVdddWlq+exXr0OLXDBkOSs9pPkuvmaX3fbF+Il0xpPzvJbUnWtdt80nxsT486GNgb+L2q+seq+qequgZ4G/DJqrqzqn4KfAg4dsouotPbMT9v5zdV1f+sqoeAfwJ+Bzi5qn7YvoH+V+DY6YqoqnOralNVPVJVXwM2tLXNxtuAs6rqhqr6f22th7SfzCedUVU/qqp7gW/ThBVVdU1VPWsrt2umbqy1K/DjKW0/Bp4+i3rfAXxpugVJXgvsBXxjFuvR49QOGww0/7HfPI/r+xhw/DTtZwO/TrPf9anAe+Zxm4J9gXvaN/NBe9N88p50D7CI5k1r0n1TxgzOLwGeBqyd/PQNfLNt/xVJ3pHkxoG+vwHsMct/Q6fWNsh+QPMNZdLfD0z/jOaNfdaS3DpwYPi1wE+BZ0zp9gzgJ786urOe1wDPZvgb/0rgvPbfoCeoHTYYquoq4IeDbUn2bz/5r01ydZJfn8P61jDNi6qqLqsWcD3wnL61q+M+YL9pDhZvAp47ML8f8BCwZaBt6p8GHpz/PvBz4MUDn76f2R5c7UjyXOALwInA7u3uonXA5HGAmf4EcafWJLvQHBC+f4ZxJHntNGcE/XRKCFBVLx44QHw1cCvw/CSD3xBe2rZvzUrg/One+NvjL8fgbqQnvB02GIZYBbyvql4BfBD40/lacbsL6XiaT52aP9cDm4EzkuyS5ClJXk1zxs3JSZ6XZFea3UBfm+abxbSq6hGaN/tPJdkTIMk+Sd40TfddaN78J9p+76L5xjBpC/CcJDsP2dxfAO9KclCSJ7e1Xjebg7dVdfXgGUHT3K4eMu7vgBuBD7eP2VHAv2QrZxINvPF/aUiXo4Af0ezq0hPY4yYY2jeP3wLOTXIj8HmaA4EkObo9RjD1dvkcNvGnNGdqTPtC1WNTVQ8D/4bm4PC9wEbgt4GzaM6auQq4i+aYwfvmuPr/DNwOXJvkQeBK4IXT1PBd4BPA39CEwEuAvx7o8lc0n8T/Psn3pxm/Bvh9mjflzcD+DDmWMc+OBcaAfwDOAP5dVU2G22uTTP1WcCTNcYhhb/wrgS+XF2l5wsuO/H+gPbh3SVX9RpJnALdV1dIe63sd8MGqesuU9g8DLwOObj+JStLj1uPmG0NVPQjcleQYgDRe2ne9Sd5Dc2rhcYaCpCeCHfYbQ5JzgNfRnDmyBfgwzVf+z9LsQnoS8NWq+qNZru9qmrOPdqU5o+TdVXV5kodozjiZPDB9/mzXKUk7oh02GCRJ28bjZleSJGl+7JB/aGyPPfaoZcuWLXQZkrRDWbt27feratofeQ7aIYNh2bJljI+PL3QZkrRDSXLPzL3clSRJmsJgkCR1GAySpA6DQZLUYTBIkjrm5aykJG8G/juwE/BnVXXGlOVply+n+Vv076yqG2Yzdr6cduEtnHPdfTxcxU4Jx71yXz5y5Eu2xaY0Bz4vo8fnRL2/MbSXZvwMcARwIHBckgOndDsCOKC9nUDzZytmO7a30y68ha9cey8Pt7/yfriKr1x7L6ddeMt8b0pz4PMyenxOBPOzK+lg4Pb2Eoy/oLnW7YopfVbQ/jnfqroWeFaSpbMc29s510290NfW27V9+LyMHp8TwfwEwz50L6m4ke4lDbfWZzZjAUhyQpLxJOMTExNzKvDhIX8Pali7tg+fl9HjcyKYn2DING1T/xcN6zObsU1j1aqqGquqsSVLZvxFd8dOmW4zw9u1ffi8jB6fE8H8BMNGmgu6T3oOzTVwZ9NnNmN7O+6V+86pXduHz8vo8TkRzE8wfAc4oL027840lxu8eEqfi4F3tBfPeRXw46raPMuxvX3kyJfw9lft9+innp0S3v6q/TzTYoH5vIwenxPBPF2PIcly4E9oTjk9q6r+OMl7Aarqc+3pqp8G3kxzuuq7qmp82NiZtjc2Nlb+ET1Jmpska6tqbMZ+O+KFegwGSZq72QaDv3yWJHUYDJKkDoNBktRhMEiSOgwGSVKHwSBJ6jAYJEkdBoMkqcNgkCR1GAySpA6DQZLUYTBIkjoMBklSh8EgSeowGCRJHQaDJKnDYJAkdRgMkqSOXsGQZLckVyTZ0N4vnqbPvkm+nWR9kluTvH9g2elJ7k9yY3tb3qceSVJ/fb8xnAKsqaoDgDXt/FQPAR+oqhcBrwJ+N8mBA8s/VVUHtbfLetYjSeqpbzCsAFa306uBI6d2qKrNVXVDO/0TYD2wT8/tSpK2kb7BsFdVbYYmAIA9t9Y5yTLgZcB1A80nJrk5yVnT7YoaGHtCkvEk4xMTEz3LliQNM2MwJLkyybppbivmsqEkuwLnASdV1YNt82eB/YGDgM3AJ4aNr6pVVTVWVWNLliyZy6YlSXOwaKYOVXXYsGVJtiRZWlWbkywFHhjS70k0oXB2VZ0/sO4tA32+AFwyl+IlSfOv766ki4GV7fRK4KKpHZIE+CKwvqo+OWXZ0oHZo4B1PeuRJPXUNxjOAA5PsgE4vJ0nyd5JJs8wejVwPPD6aU5LPTPJLUluBg4FTu5ZjySppxl3JW1NVf0AeMM07ZuA5e30NUCGjD++z/YlSfPPXz5LkjoMBklSh8EgSeowGCRJHQaDJKnDYJAkdRgMkqQOg0GS1GEwSJI6DAZJUofBIEnqMBgkSR0GgySpw2CQJHUYDJKkDoNBktRhMEiSOgwGSVJHr2BIsluSK5JsaO8XD+l3d3tt5xuTjM91vCRp++n7jeEUYE1VHQCsaeeHObSqDqqqscc4XpK0HfQNhhXA6nZ6NXDkdh4vSZpnfYNhr6raDNDe7zmkXwHfSrI2yQmPYbwkaTtZNFOHJFcCz55m0alz2M6rq2pTkj2BK5J8r6qumsN42kA5AWC//faby1BJ0hzMGAxVddiwZUm2JFlaVZuTLAUeGLKOTe39A0kuAA4GrgJmNb4duwpYBTA2NlYz1S1Jemz67kq6GFjZTq8ELpraIckuSZ4+OQ28EVg32/GSpO2rbzCcARyeZANweDtPkr2TXNb22Qu4JslNwPXApVX1za2NlyQtnBl3JW1NVf0AeMM07ZuA5e30ncBL5zJekrRw/OWzJKnDYJAkdRgMkqQOg0GS1GEwSJI6DAZJUofBIEnqMBgkSR0GgySpw2CQJHUYDJKkDoNBktRhMEiSOgwGSVKHwSBJ6jAYJEkdBoMkqcNgkCR1GAySpI5ewZBktyRXJNnQ3i+eps8Lk9w4cHswyUntstOT3D+wbHmfeiRJ/fX9xnAKsKaqDgDWtPMdVXVbVR1UVQcBrwB+Blww0OVTk8ur6rKe9UiSeuobDCuA1e30auDIGfq/Abijqu7puV1J0jbSNxj2qqrNAO39njP0PxY4Z0rbiUluTnLWdLuiJiU5Icl4kvGJiYl+VUuShpoxGJJcmWTdNLcVc9lQkp2BtwLnDjR/FtgfOAjYDHxi2PiqWlVVY1U1tmTJkrlsWpI0B4tm6lBVhw1blmRLkqVVtTnJUuCBrazqCOCGqtoysO5Hp5N8AbhkdmVLkraVvruSLgZWttMrgYu20vc4puxGasNk0lHAup71SJJ66hsMZwCHJ9kAHN7Ok2TvJI+eYZTkae3y86eMPzPJLUluBg4FTu5ZjySppxl3JW1NVf2A5kyjqe2bgOUD8z8Ddp+m3/F9ti9Jmn/+8lmS1GEwSJI6DAZJUofBIEnqMBgkSR0GgySpw2CQJHUYDJKkDoNBktRhMEiSOgwGSVKHwSBJ6jAYJEkdBoMkqcNgkCR1GAySpA6DQZLUYTBIkjp6BUOSY5LcmuSRJGNb6ffmJLcluT3JKQPtuyW5IsmG9n5xn3okSf31/cawDjgauGpYhyQ7AZ8BjgAOBI5LcmC7+BRgTVUdAKxp5yVJC6hXMFTV+qq6bYZuBwO3V9WdVfUL4KvAinbZCmB1O70aOLJPPZKk/rbHMYZ9gPsG5je2bQB7VdVmgPZ+z2ErSXJCkvEk4xMTE9usWEl6ols0U4ckVwLPnmbRqVV10Sy2kWnaahbjugOqVgGrAMbGxuY8XpI0OzMGQ1Ud1nMbG4F9B+afA2xqp7ckWVpVm5MsBR7ouS1JUk/bY1fSd4ADkjwvyc7AscDF7bKLgZXt9EpgNt9AJEnbUN/TVY9KshE4BLg0yeVt+95JLgOoqoeAE4HLgfXA16vq1nYVZwCHJ9kAHN7OS5IWUKp2vN31Y2NjNT4+vtBlSNIOJcnaqhr6m7NJ/vJZktRhMEiSOgwGSVKHwSBJ6jAYJEkdBoMkqcNgkCR1GAySpA6DQZLUYTBIkjoMBklSh8EgSeowGCRJHQaDJKnDYJAkdRgMkqQOg0GS1GEwSJI6+l7z+ZgktyZ5JMm0l4tLsm+SbydZ3/Z9/8Cy05Pcn+TG9ra8Tz2SpP4W9Ry/Djga+PxW+jwEfKCqbkjydGBtkiuq6rvt8k9V1cd71iFJmie9gqGq1gMk2VqfzcDmdvonSdYD+wDfHTpIkrRgtusxhiTLgJcB1w00n5jk5iRnJVm8lbEnJBlPMj4xMbGtS5WkJ6wZgyHJlUnWTXNbMZcNJdkVOA84qaoebJs/C+wPHETzreITw8ZX1aqqGquqsSVLlsxl05KkOZhxV1JVHdZ3I0meRBMKZ1fV+QPr3jLQ5wvAJX23JUnqZ5vvSkpzAOKLwPqq+uSUZUsHZo+iOZgtSVpAfU9XPSrJRuAQ4NIkl7fteye5rO32auB44PXTnJZ6ZpJbktwMHAqc3KceSVJ/qaqFrmHOxsbGanx8fKHLkKQdSpK1VTXtb84G+ctnSVKHwSBJ6jAYJEkdBoMkqcNgkCR1GAySpA6DQZLUYTBIkjoMBklSh8EgSeowGCRJHQaDJKnDYJAkdRgMkqQOg0GS1GEwSJI6DAZJUofBIEnq6HvN52OS3JrkkSRDLxeX5O722s43JhkfaN8tyRVJNrT3i/vUI0nqr+83hnXA0cBVs+h7aFUdNOV6o6cAa6rqAGBNOy9JWkC9gqGq1lfVbT1WsQJY3U6vBo7sU48kqb/tdYyhgG8lWZvkhIH2vapqM0B7v+ewFSQ5Icl4kvGJiYltXK4kPXEtmqlDkiuBZ0+z6NSqumiW23l1VW1KsidwRZLvVdVsdj89qqpWAasAxsbGai5jJUmzN2MwVNVhfTdSVZva+weSXAAcTHNcYkuSpVW1OclS4IG+25Ik9bPNdyUl2SXJ0yengTfSHLQGuBhY2U6vBGb7DUSStI30PV31qCQbgUOAS5Nc3rbvneSytttewDVJbgKuBy6tqm+2y84ADk+yATi8nZckLaBU7Xi768fGxmp8fHzmjpKkRyVZO+UnA9Pyl8+SpA6DQZLUYTBIkjoMBklSh8EgSeowGCRJHQaDJKnDYJAkdRgMkqQOg0GS1GEwSJI6DAZJUofBIEnqMBgkSR0GgySpw2CQJHUYDJKkDoNBktTR95rPxyS5NckjSaa9XFySFya5ceD2YJKT2mWnJ7l/YNnyPvVIkvpb1HP8OuBo4PPDOlTVbcBBAEl2Au4HLhjo8qmq+njPOiRJ86RXMFTVeoAksx3yBuCOqrqnz3YlSdvO9j7GcCxwzpS2E5PcnOSsJIuHDUxyQpLxJOMTExPbtkpJegKbMRiSXJlk3TS3FXPZUJKdgbcC5w40fxbYn2ZX02bgE8PGV9WqqhqrqrElS5bMZdOSpDmYcVdSVR02T9s6ArihqrYMrPvR6SRfAC6Zp21Jkh6j7bkr6Tim7EZKsnRg9iiag9mSpAXU93TVo5JsBA4BLk1yedu+d5LLBvo9DTgcOH/KKs5MckuSm4FDgZP71CNJ6q/vWUkX0D31dLJ9E7B8YP5nwO7T9Du+z/Yl6YnitAtv4Zzr7uPhKnZKOO6V+/KRI1+yTbbV93cMkqRt7LQLb+Er19776PzDVY/Ob4tw8E9iSNKIO+e6++bU3pfBIEkj7uGqObX3ZTBI0ojbachflxjW3pfBIEkj7rhX7jun9r48+CxJI27yAPP2OisptY32UW1LY2NjNT4+vtBlSNIOJcnaqpr2EgmD3JUkSeowGCRJHQaDJKnDYJAkdRgMkqSOHfKspCQTwKhcHnQP4PsLXcQMrLG/Ua8PRr/GUa8PHv81PreqZrzS2Q4ZDKMkyfhsTv9aSNbY36jXB6Nf46jXB9Y4yV1JkqQOg0GS1GEw9LdqoQuYBWvsb9Trg9GvcdTrA2sEPMYgSZrCbwySpA6DQZLUVVXemt1pzwK+AXwPWA8cAuwGXAFsaO8XD/T/EHA7cBvwpoH2VwC3tMv+B7/cXfdk4Gtt+3XAssdQ48nArcA64BzgKQtdI3AW8ACwbqBtu9QErGy3sQFYOYf6PtY+zzcDFwDPWqj6htU4sOyDQAF7jNJj2La/r63hVuDMUXsMgYOAa4EbgXHg4AV8DPcFvk3z3nIr8P5Re6106p3rm9Pj9QasBt7TTu9MExRnAqe0bacAH22nDwRuap+I5wF3ADu1y66nCZUAfwkc0bb/J+Bz7fSxwNfmWN8+wF3AU9v5rwPvXOgagX8FvJzuC3Kb19S+oO5s7xe304tnWd8bgUXt9EcXsr5hNbbt+wKX0/yYc48RewwPBa4EntzO7zlqjyHwrYFtLAf+9wI+hkuBl7fTTwf+rq1jZF4rnXq3xZvsjnYDnkHzppsp7bcBSwee2Nva6Q8BHxrod3n7RC0FvjfQfhzw+cE+7fQiml8uZg417gPc1z65i4BLaN7gFrxGYBndF+Q2r2mwT7vs88Bxs6lvyrKjgLMXsr5hNdJ8g30pcDe/DIaReAxpPpgcNk2/kXkM2/X+9sD2/mKhaxzodxFwOCP2Wpm8eYyh8XxgAvhfSf42yZ8l2QXYq6o2A7T3e7b9J9+kJ21s2/Zpp6e2d8ZU1UPAj4HdZ1tgVd0PfBy4F9gM/LiqvjVKNQ7YHjUNW9dc/QeaT10jVV+StwL3V9VNUxaNSo0vAF6b5Lok/yfJb45YfQAnAR9Lch/Na+dDo1BjkmXAy2h294zka8VgaCyi+Rr62ap6GfCPNF/rhpnuCty1lfatjZmVJIuBFTRfK/cGdkny9lGqcRbms6betSY5FXgIOHuU6kvyNOBU4A+mWzwKNdK8ZhYDrwJ+D/h6koxQfQD/ETi5qvalOT73xR7bm5cak+wKnAecVFUPDuu3kDWCwTBpI7Cxqq5r579BExRbkiwFaO8fGOg/eBXu5wCb2vbnTNPeGZNkEfBM4IdzqPEw4K6qmqiqfwbOB35rxGqctD1qGrauWUmyEngL8LZqv1+PUH3703wAuCnJ3e3YG5I8e4Rq3AicX43rgUdo/rjbqNQHzQHX89vpc4GDp25ve9aY5Ek0oXB2VU3WNZqvlZn2hT1RbsDVwAvb6dNpzlz5GN0DQ2e20y+me2DoTn55YOg7NJ+iJg8MLW/bf5fugaGvz7G+V9KczfC0dt2rac4KWfAa+dV9u9u8JppjLXfRfGpd3E7vNsv63gx8F1gypd+C1DddjVOW3c0vjzGMymP4XuCP2ukX0OyqyCg9hjRnAL2unX4DsHahHsN2fV8G/mRK+0i9Vh6t67G8iT4ebzSnto3TnMJ4YfsA7g6soTnFa83gg0nzVf8OmoNHRwy0j9GcTnoH8Gl+eSrZU2g+tdxOc1bB8x9DjX9Ic5rlOuDP2/80C1ojzWmzm4F/pvlk8u7tVRPN8YHb29u75lDf7TRvZDe2t88tVH3Dapyy/G66p6uOwmO4M/CVdns3AK8ftccQeA2wluYN9jrgFQv4GL6GZvfNzfzy/91yRui1MnjzT2JIkjo8xiBJ6jAYJEkdBoMkqcNgkCR1GAySpA6DQZLUYTBIkjr+P1jo4RjfQdO5AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(X_population['kms'],error)\n",
    "plt.title('correlation={}'.format(round(np.corrcoef(x=X_population['kms'], y=error)[0][1],2)))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "498f3308",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEICAYAAABBBrPDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAeS0lEQVR4nO3dcZxVZ33n8c+3gHFMJJAwSWFAQYNZg1ZJbgkabVPpMnTXCs3G3dm1DVVe8jKbl1vtLjboVqquayKuadNu4mabGBJjEopIqC0iG9xGuwk4BJWQiIzGhAEMowOIOsZAfvvHea65c7nzzMwdmDvDfN+v133Nub9znjPPfULu957znLlHEYGZmVlffq3RHTAzs5HNQWFmZlkOCjMzy3JQmJlZloPCzMyyHBRmZpbloDDLkHSlpM4htP+MpD8/lX0yG24OCrNTRNIfS/p6ZS0i3hMRHxvmfvwHSU9J+pmkDZLOy2z7RknbJR2T9G1Jb6pYN1XSRkkHJIWkmcPyAmzEcVDYmCFp/EBqo5mkOcD/Av4IuBD4OXBLH9ueB2wEVgOTgE8Cfy9pctrkeeDLwL85vb22kc5BYaOGpBmS1kvqkvRjSX8j6dck/df0CfqQpLsknZu2n5k+CS+T9DSwNX3q/2dJN0nqBv5C0lmSPiXpaUnPpNNFTX304XpJ30ufwB+X9Aep/mrgM8AbJP1U0pFUv1PSf6to/25JHZK606f1aRXrQtJ7JO2VdFjS/5SkQQ7TO4C/j4iHIuKnwJ8DV0l6aY1t3wg8ExF/FxEnIuJzQBdwFUBEPBMRtwDfGGQf7AzjoLBRQdI44EvAU8BMoAW4D/jj9Pgd4BXAOcDfVDX/beDVQGt6fjnwfeAC4OPAjcCrgNcDF6V9f7iPrnwPeDNwLvAR4HOSpkbEE8B7gIcj4pyImFTjNbwF+ATwb4Gp6bXcV7XZW4HfBF6XtmtNbd8k6UjmUT5lNAf4VnlnEfE94Jfp9Z3UpfSorr2mj9duY5SDwkaLecA0YEVE/CwifhERX6f4BP3piPh++gS9EmirOqX0F6lNT3p+ICL+OiKOA78A3g28PyK6I+IY8N+BtlqdSJ++D0TE8xFxP7A39W0g3gHcERGPRsSzqa9vqDr3f0NEHImIp4GvUoQXEfH1iJiUeZTnRs4Bjlb93qNArSOK/wdMk/TvJU2QtBR4JfCSAb4eGyMcFDZazACeSm/ulaZRfDIvewoYT3F+vmxfVZvK580Ub4w7yp/OKc7LN9fqhKRrJH2zYtvXAFMG+Bp69TUF248pjmDKflix/HOKN/7B+Ckwsao2EThWvWFE/BhYDPwp8AywCPg/QN1XedmZ6YyayLMz2j7gZZLGV4XFAeDlFc9fBhyneOObnmrVX5Fc+fxHQA8wJyL25zog6eXA/wYWUJxiOiHpm7xw+qa/r2Lu1VdJZwPnA9nfm7Z9M7Aps8nvRcTXgN0Up63K7V4BnAV8t1ajiPgnilNd5Yn97wH/o7/+2NjiIwobLbYDB4EbJJ0t6cWSrgDuBd4vaZakcyhOG91f48ijpoh4nuLN/yZJFwBIapHUWmPzsynCoCtt9056n89/Bpgu6UV9/LrPA++U9HpJZ6W+bouIHwygn19Lcx99Pb6WNr0H+H1Jb05B9FFgfTqldhJJc9Npp4nAp4DOiNhcsf7FFEEDcFZ6bmOMg8JGhYg4Afw+xWTz0xSnR/4dcAdwN/AQ8CTFnMN7B7n7PwM6gEck/YTi9MvFNfrwOMWn7YcpQuG1wD9XbLKV4hP9DyX9qEb7BymuQvoCRei9kj7mQuoVEbspJtXvAQ5RzE38x/L6dEXXZyqafIDiqGofxQT7H1TtsofidBbAd9JzG2PkGxeZmVmOjyjMzCzLQWFmZlkOCjMzy3JQmJlZ1hn3dxRTpkyJmTNnNrobZmajyo4dO34UETX/0PSMC4qZM2fS3t7e6G6YmY0qkp7qa51PPZmZWZaDwszMshwUZmaW5aAwM7MsB4WZmWX1GxTp9pNflfSEpN2S/iTVz5O0Jd22cUvFfXaRtDLd7nFP5bdwSrpM0q607ubybR7TrSjvT/VtlTdykbQ0/Y696cYqZnaG27BzP1fcsJVZ1/8DV9ywlQ07+/0m9jHtdI/XQI4ojgP/OSJeDcwHrpN0CXA98GBEzAYeTM9J69oobsm4CLgl3cYS4FZgOTA7PRal+jLgcERcBNxEcWvK8s3fV1HcunIesKoykMzszLNh535Wrt/F/iM9BLD/SA8r1+9yWPRhOMar36CIiIMR8WhaPgY8QXFHrsXAmrTZGmBJWl4M3BcRz0bEkxRf3zxP0lRgYkQ8HMVX1t5V1aa8r3XAgnS00QpsSbeoPAxs4YVwMbMz0OrNe+h57kSvWs9zJ1i9eU+DejSyDcd4DWqOIp0SmgtsAy6MiINQhAnFjeqhCJHKW012ploLvW+xWK73apNuOHOU4s5ffe2rul/LJbVLau/q6hrMSzKzEebAkdq3vOirPtYNx3gNOCjS3cO+ALwvIn6S27RGLTL1etu8UIi4LSJKEVFqbq75F+hmNkpMm9Q0qPpYNxzjNaCgkDSBIiTuiYj1qfxMOp1E+nko1TuBGRXNp1PcK7iTF+5hXFnv1Sbdt/dcoDuzLzM7Q61ovZimCeN61ZomjGNF60k3HTSGZ7wGctWTgNuBJyLi0xWrNgLlq5CWAg9U1NvSlUyzKCatt6fTU8ckzU/7vKaqTXlfVwNb0zzGZmChpMlpEnthqpnZGWrJ3BY+cdVraZnUhICWSU184qrXsmTuSWedjeEZr35vhSrpTcDXgF3A86n8QYp5irXAyyjuYfz2iOhObT4EvIviiqn3RcSmVC8BdwJNwCbgvRER6Ybtd1PMf3QDbRHx/dTmXen3AXw8Ij6b62+pVAp/KaCZ2eBI2hERpZrrzrR7ZjsozMwGLxcU/stsMzPLclCYmVmWg8LMzLIcFGZmluWgMDOzLAeFmZllOSjMzCzLQWFmZlkOCjMzy3JQmJlZloPCzMyyHBRmZpbloDAzsywHhZmZZTkozMwsy0FhZmZZDgozM8sayD2z75B0SNJjFbXXS3pE0jcltUuaV7FupaQOSXsktVbUL5O0K627Od03m3Rv7ftTfZukmRVtlkramx7le2qbmdkwGsgRxZ3AoqraJ4GPRMTrgQ+n50i6BGgD5qQ2t0gal9rcCiwHZqdHeZ/LgMMRcRFwE3Bj2td5wCrgcmAesErS5EG/QjMzG5J+gyIiHgK6q8vAxLR8LnAgLS8G7ouIZyPiSaADmCdpKjAxIh6O4ibddwFLKtqsScvrgAXpaKMV2BIR3RFxGNjCyYFlZman2fg6270P2CzpUxRh88ZUbwEeqdiuM9WeS8vV9XKbfQARcVzSUeD8ynqNNmZmNkzqncy+Fnh/RMwA3g/cnuqqsW1k6vW26UXS8jRX0t7V1ZXtuJmZDU69QbEUWJ+W/45iDgGKT/0zKrabTnFaqjMtV9d7tZE0nuJUVndmXyeJiNsiohQRpebm5jpfkpmZ1VJvUBwAfjstvwXYm5Y3Am3pSqZZFJPW2yPiIHBM0vw0/3AN8EBFm/IVTVcDW9M8xmZgoaTJaRJ7YaqZmdkw6neOQtK9wJXAFEmdFFcivRv4q3QE8AuKq5mIiN2S1gKPA8eB6yLiRNrVtRRXUDUBm9IDitNWd0vqoDiSaEv76pb0MeAbabuPRkT1pLqZmZ1mKj68nzlKpVK0t7c3uhtmZqOKpB0RUaq1zn+ZbWZmWQ4KMzPLclCYmVmWg8LMzLIcFGZmluWgMDOzLAeFmZllOSjMzCzLQWFmZlkOCjMzy3JQmJlZloPCzMyyHBRmZpbloDAzsywHhZmZZTkozMwsy0FhZmZZDgozM8vqNygk3SHpkKTHqurvlbRH0m5Jn6yor5TUkda1VtQvk7QrrbtZklL9LEn3p/o2STMr2iyVtDc9lp6SV2xmZoMykCOKO4FFlQVJvwMsBn4jIuYAn0r1S4A2YE5qc4ukcanZrcByYHZ6lPe5DDgcERcBNwE3pn2dB6wCLgfmAaskTa7rVZqZWd36DYqIeAjoripfC9wQEc+mbQ6l+mLgvoh4NiKeBDqAeZKmAhMj4uGICOAuYElFmzVpeR2wIB1ttAJbIqI7Ig4DW6gKLDMzO/3qnaN4FfDmdKronyT9Zqq3APsqtutMtZa0XF3v1SYijgNHgfMz+zqJpOWS2iW1d3V11fmSzMyslnqDYjwwGZgPrADWpqMA1dg2MnXqbNO7GHFbRJQiotTc3Nxf383MbBDqDYpOYH0UtgPPA1NSfUbFdtOBA6k+vUadyjaSxgPnUpzq6mtfZmY2jOoNig3AWwAkvQp4EfAjYCPQlq5kmkUxab09Ig4CxyTNT0ce1wAPpH1tBMpXNF0NbE3zGJuBhZImp0nshalmZmbDaHx/G0i6F7gSmCKpk+JKpDuAO9Ils78ElqY3992S1gKPA8eB6yLiRNrVtRRXUDUBm9ID4HbgbkkdFEcSbQAR0S3pY8A30nYfjYjqSXUzMzvNVLy/nzlKpVK0t7c3uhtmZqOKpB0RUaq1zn+ZbWZmWQ4KMzPLclCYmVmWg8LMzLL6verJzIZuw879rN68hwNHepg2qYkVrRezZG7NLxowPF4jjYPC7DTbsHM/K9fvoue54krx/Ud6WLl+F4Df/GrweI08PvVkdpqt3rznV296ZT3PnWD15j0N6tHI5vEaeRwUZqfZgSM9g6qPdR6vkcdBYXaaTZvUNKj6WOfxGnkcFGan2YrWi2maMK5XrWnCOFa0XtygHo1sHq+Rx5PZZqdZeQLWV/EMjMdr5PF3PZmZmb/ryczM6uegMDOzLAeFmZllOSjMzCzLQWFmZln9BoWkOyQdSrc9rV73XySFpCkVtZWSOiTtkdRaUb9M0q607uZ072zS/bXvT/VtkmZWtFkqaW96LMXMzIbdQI4o7gQWVRclzQD+JfB0Re0Sintez0ltbpFU/suZW4HlwOz0KO9zGXA4Ii4CbgJuTPs6j+L+3JcD84BVkiYP7uWZmdlQ9RsUEfEQ0F1j1U3AB4DKP8RYDNwXEc9GxJNABzBP0lRgYkQ8HMUfbtwFLKlosyYtrwMWpKONVmBLRHRHxGFgCzUCy8zMTq+65igkvQ3YHxHfqlrVAuyreN6Zai1pubreq01EHAeOAudn9lWrP8sltUtq7+rqquclmZlZHwYdFJJeAnwI+HCt1TVqkanX26Z3MeK2iChFRKm5ubnWJmZmVqd6jiheCcwCviXpB8B04FFJv07xqX9GxbbTgQOpPr1Gnco2ksYD51Kc6uprX2ZmNowGHRQRsSsiLoiImRExk+IN/dKI+CGwEWhLVzLNopi03h4RB4Fjkuan+YdrgAfSLjcC5Suarga2pnmMzcBCSZPTJPbCVDMzs2HU77fHSroXuBKYIqkTWBURt9faNiJ2S1oLPA4cB66LiPKtqq6luIKqCdiUHgC3A3dL6qA4kmhL++qW9DHgG2m7j0ZErUl1MzM7jfztsWZm5m+PNTOz+jkozMwsy0FhZmZZDgozM8tyUJiZWZaDwszMshwUZmaW5aAwM7MsB4WZmWU5KMzMLMtBYWZmWQ4KMzPLclCYmVmWg8LMzLIcFGZmluWgMDOzLAeFmZllOSjMzCyr36CQdIekQ5Ieq6itlvQdSd+W9EVJkyrWrZTUIWmPpNaK+mWSdqV1N0tSqp8l6f5U3yZpZkWbpZL2psfSU/Wizcxs4AZyRHEnsKiqtgV4TUT8BvBdYCWApEuANmBOanOLpHGpza3AcmB2epT3uQw4HBEXATcBN6Z9nQesAi4H5gGrJE0e/Es0M7Oh6DcoIuIhoLuq9pWIOJ6ePgJMT8uLgfsi4tmIeBLoAOZJmgpMjIiHIyKAu4AlFW3WpOV1wIJ0tNEKbImI7og4TBFO1YFlZman2amYo3gXsCkttwD7KtZ1plpLWq6u92qTwucocH5mXyeRtFxSu6T2rq6uIb0YMzPrbUhBIelDwHHgnnKpxmaRqdfbpncx4raIKEVEqbm5Od9pMzMblLqDIk0uvxV4RzqdBMWn/hkVm00HDqT69Br1Xm0kjQfOpTjV1de+zMxsGNUVFJIWAX8GvC0ifl6xaiPQlq5kmkUxab09Ig4CxyTNT/MP1wAPVLQpX9F0NbA1Bc9mYKGkyWkSe2GqmZnZMBrf3waS7gWuBKZI6qS4EmklcBawJV3l+khEvCcidktaCzxOcUrquog4kXZ1LcUVVE0UcxrleY3bgbsldVAcSbQBRES3pI8B30jbfTQiek2qm5nZ6acXzhqdGUqlUrS3tze6G2Zmo4qkHRFRqrXOf5ltZmZZDgozM8tyUJiZWZaDwszMshwUZmaW5aAwM7MsB4WZmWU5KMzMLMtBYWZmWQ4KMzPLclCYmVmWg8LMzLIcFGZmluWgMDOzLAeFmZllOSjMzCzLQWFmZln9BoWkOyQdkvRYRe08SVsk7U0/J1esWympQ9IeSa0V9csk7Urrbk73zibdX/v+VN8maWZFm6Xpd+yVVL6vto0AG3bu54obtjLr+n/gihu2smHn/kZ3aUTzeNloNpAjijuBRVW164EHI2I28GB6jqRLKO55PSe1uUXSuNTmVmA5MDs9yvtcBhyOiIuAm4Ab077Oo7g/9+XAPGBVZSBZ42zYuZ+V63ex/0gPAew/0sPK9bv85tcHj5eNdv0GRUQ8BHRXlRcDa9LyGmBJRf2+iHg2Ip4EOoB5kqYCEyPi4Shu0n1XVZvyvtYBC9LRRiuwJSK6I+IwsIWTA8saYPXmPfQ8d6JXree5E6zevKdBPRrZPF422tU7R3FhRBwESD8vSPUWYF/Fdp2p1pKWq+u92kTEceAocH5mXyeRtFxSu6T2rq6uOl+SDdSBIz2Dqo91Hi8b7U71ZLZq1CJTr7dN72LEbRFRiohSc3PzgDpq9Zs2qWlQ9bHO42WjXb1B8Uw6nUT6eSjVO4EZFdtNBw6k+vQa9V5tJI0HzqU41dXXvqzBVrReTNOEcb1qTRPGsaL14gb1aGTzeNloV29QbATKVyEtBR6oqLelK5lmUUxab0+np45Jmp/mH66palPe19XA1jSPsRlYKGlymsRemGrWYEvmtvCJq15Ly6QmBLRMauITV72WJXNrnhkc8zxeNtqpeE/ObCDdC1wJTAGeobgSaQOwFngZ8DTw9ojoTtt/CHgXcBx4X0RsSvUSxRVUTcAm4L0REZJeDNwNzKU4kmiLiO+nNu8CPpi68vGI+Gx/L6hUKkV7e/vAXr2ZmQEgaUdElGqu6y8oRhsHhZnZ4OWCwn+ZbWZmWQ4KMzPLclCYmVmWg8LMzLIcFGZmluWgMDOzLAeFmZllOSjMzCzLQWFmZlkOCjMzy3JQmJlZloPCzMyyHBRmZpbloDAzsywHhZmZZTkozMwsy0FhZmZZQwoKSe+XtFvSY5LulfRiSedJ2iJpb/o5uWL7lZI6JO2R1FpRv0zSrrTu5nRfbdK9t+9P9W2SZg6lv2ZmNnh1B4WkFuA/AaWIeA0wDmgDrgcejIjZwIPpOZIuSevnAIuAWySNS7u7FVgOzE6PRam+DDgcERcBNwE31ttfMzOrz1BPPY0HmiSNB14CHAAWA2vS+jXAkrS8GLgvIp6NiCeBDmCepKnAxIh4OIobeN9V1aa8r3XAgvLRhpmZDY+6gyIi9gOfAp4GDgJHI+IrwIURcTBtcxC4IDVpAfZV7KIz1VrScnW9V5uIOA4cBc6v7ouk5ZLaJbV3dXXV+5LMzKyGoZx6mkzxiX8WMA04W9If5prUqEWmnmvTuxBxW0SUIqLU3Nyc77iZmQ3KUE49/S7wZER0RcRzwHrgjcAz6XQS6eehtH0nMKOi/XSKU1Wdabm63qtNOr11LtA9hD6bmdkgDSUongbmS3pJmjdYADwBbASWpm2WAg+k5Y1AW7qSaRbFpPX2dHrqmKT5aT/XVLUp7+tqYGuaxzAzs2Eyvt6GEbFN0jrgUeA4sBO4DTgHWCtpGUWYvD1tv1vSWuDxtP11EXEi7e5a4E6gCdiUHgC3A3dL6qA4kmirt79mZlYfnWkf0EulUrS3tze6G2Zmo4qkHRFRqrXOf5ltZmZZDgozM8tyUJiZWZaDwszMshwUZmaW5aAwM7MsB4WZmWU5KMzMLMtBYWZmWQ4KMzPLclCYmVmWg8LMzLIcFGZmluWgMDOzLAeFmZllOSjMzCzLQWFmZlkOCjMzyxpSUEiaJGmdpO9IekLSGySdJ2mLpL3p5+SK7VdK6pC0R1JrRf0ySbvSupslKdXPknR/qm+TNHMo/TUzs8Eb6hHFXwFfjoh/AbwOeAK4HngwImYDD6bnSLoEaAPmAIuAWySNS/u5FVgOzE6PRam+DDgcERcBNwE3DrG/ZmY2SHUHhaSJwG8BtwNExC8j4giwGFiTNlsDLEnLi4H7IuLZiHgS6ADmSZoKTIyIhyMigLuq2pT3tQ5YUD7aMDOz4TGUI4pXAF3AZyXtlPS3ks4GLoyIgwDp5wVp+xZgX0X7zlRrScvV9V5tIuI4cBQ4v7ojkpZLapfU3tXVNYSXZGZm1YYSFOOBS4FbI2Iu8DPSaaY+1DoSiEw916Z3IeK2iChFRKm5uTnfazMzG5ShBEUn0BkR29LzdRTB8Uw6nUT6eahi+xkV7acDB1J9eo16rzaSxgPnAt1D6LOZmQ1S3UERET8E9km6OJUWAI8DG4GlqbYUeCAtbwTa0pVMsygmrben01PHJM1P8w/XVLUp7+tqYGuaxzAzs2Eyfojt3wvcI+lFwPeBd1KEz1pJy4CngbcDRMRuSWspwuQ4cF1EnEj7uRa4E2gCNqUHFBPld0vqoDiSaBtif83MbJB0pn1AL5VK0d7e3uhumJmNKpJ2RESp1jr/ZbaZmWU5KMzMLMtBYWZmWQ4KMzPLGupVT2eMDTv3s3rzHg4c6WHapCZWtF7Mkrkt/TccozxeZmOHg4LiTW/l+l30PFdcrbv/SA8r1+8C8JtfDR4vs7HFp56A1Zv3/OpNr6znuROs3rynQT0a2TxeZmOLgwI4cKRnUPWxzuNlNrY4KIBpk5oGVR/rPF5mY4uDAljRejFNE8b1qjVNGMeK1ov7aDG2ebzMxhZPZvPCBKyv4hkYj5fZ2OLvejIzM3/Xk5mZ1c9BYWZmWQ4KMzPLclCYmVmWg8LMzLLOuKueJHUBTw1hF1OAH52i7pxK7tfguF+D434NzpnYr5dHRHOtFWdcUAyVpPa+LhFrJPdrcNyvwXG/Bmes9cunnszMLMtBYWZmWQ6Kk93W6A70wf0aHPdrcNyvwRlT/fIchZmZZfmIwszMshwUZmaWNSaDQtIdkg5JeqyP9ZJ0s6QOSd+WdOkI6deVko5K+mZ6fHiY+jVD0lclPSFpt6Q/qbHNsI/ZAPs17GMm6cWStkv6VurXR2ps04jxGki/GvVvbJyknZK+VGNdQ/5/HEC/GjJW6Xf/QNKu9HtP+rrsUz5mETHmHsBvAZcCj/Wx/l8BmwAB84FtI6RfVwJfasB4TQUuTcsvBb4LXNLoMRtgv4Z9zNIYnJOWJwDbgPkjYLwG0q9G/Rv7U+DztX53o/5/HEC/GjJW6Xf/AJiSWX9Kx2xMHlFExENAd2aTxcBdUXgEmCRp6gjoV0NExMGIeDQtHwOeAKrvUjTsYzbAfg27NAY/TU8npEf1VSONGK+B9GvYSZoO/Gvgb/vYpCH/Pw6gXyPZKR2zMRkUA9AC7Kt43skIeANK3pBOHWySNGe4f7mkmcBcik+jlRo6Zpl+QQPGLJ2y+CZwCNgSESNivAbQLxj+8fpL4APA832sb9S/rb8k3y9o3P+PAXxF0g5Jy2usP6Vj5qCoTTVqDf/kBTxK8X0srwP+GtgwnL9c0jnAF4D3RcRPqlfXaDIsY9ZPvxoyZhFxIiJeD0wH5kl6TdUmDRmvAfRrWMdL0luBQxGxI7dZjdppHasB9quR/z9eERGXAr8HXCfpt6rWn9Ixc1DU1gnMqHg+HTjQoL78SkT8pHzqICL+EZggacpw/G5JEyjejO+JiPU1NmnImPXXr0aOWfqdR4D/CyyqWtXQf2N99asB43UF8DZJPwDuA94i6XNV2zRirPrtVyP/bUXEgfTzEPBFYF7VJqd0zBwUtW0ErklXDswHjkbEwUZ3StKvS1Jankfx3+/Hw/B7BdwOPBERn+5js2Efs4H0qxFjJqlZ0qS03AT8LvCdqs0aMV799mu4xysiVkbE9IiYCbQBWyPiD6s2G/axGki/Gvj/49mSXlpeBhYC1VdKntIxG193b0cxSfdSXLEwRVInsIpiYo+I+AzwjxRXDXQAPwfeOUL6dTVwraTjQA/QFukSh9PsCuCPgF3p/DbAB4GXVfStEWM2kH41YsymAmskjaN481gbEV+S9J6KfjVivAbSr0b9G+tlBIzVQPrVqLG6EPhiyqjxwOcj4sunc8z8FR5mZpblU09mZpbloDAzsywHhZmZZTkozMwsy0FhZmZZDgozM8tyUJiZWdb/B4v/cJ5c3+UtAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(data_population['age'],data_population['kms'])\n",
    "plt.title('correlation={}'.format(round(np.corrcoef(x=data_population['age'], y=data_population['kms'])[0][1],2)))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "3a490f01",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEICAYAAABfz4NwAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAXdElEQVR4nO3df5Bdd3nf8fcnkmIWHFvEEsRaGeQGo4nNL8HGNXVIHZxWTkuxypBUHRIb6omKx5O20IpYpQEyaYsdkTjjpECc4LFNgn+UCOFCjKA4rSFj7FkjiGwTBRFjWyuDBUbGLhtXEk//uN91r1Z3pd3Vau+u9v2auaNzn3POvc892r2fe873nLupKiRJ+pF+NyBJmhsMBEkSYCBIkhoDQZIEGAiSpMZAkCQBBoL0rCQXJNl9DOt/OMlvzGRP0mwyEKRpSPLWJF/srlXV26vqt/rVE0CS30qyI8mBJO87yrJJcnWS77bbbyfJLLWqOchA0AkpyeLJ1E5Au4B3AZ+exLIbgHXAK4FXAG8A/vVx60xznoGgOSnJGUm2JNnbPr3+QZIfSfKfkjyc5PEkNyU5tS2/KkkluSzJI8Cd7VP8Xya5JskTwPuSnJTkA0keSfLtdphnYIIerkzyjSRPJXkwyT9v9Z8CPgy8NsnTSfa1+g1J/nPX+r+aZFeSJ5LcnmRF17xK8vYkX0/yvST/bSY+nVfVjVV1B/DUJBa/FPidqtpdVSPA7wBvPdYeNH8ZCJpzkiwCPgU8DKwCBoFb6LxZvRX4OeDvAScDfzBu9X8I/BSwtt3/+8DfAi8A/gtwNfBS4FXAS9pjv2eCVr4BvA44FfhN4E+SnF5VXwPeDtxdVSdX1dIer+H1wPuBXwJOb6/llnGLvQH4aTqf0H9prOckP5Nk3xFuPzNBv1N1DvDVrvtfbTUtUAaC5qJzgRXAxqr6P1X1d1X1ReAtwO9W1d9W1dPAJmD9uENB72vrjLb7e6rq96vqAPB3wK8C76iqJ6rqKeC/Aut7NVFV/72q9lTVD6vqVuDrrbfJeAtwfVV9uaqeab2+NsmqrmWuqqp9VfUI8Bd0Qoqq+mJVLT3C7Yvjn2yaTgae7Lr/JHCy4wgLl4GguegM4OH2Jt5tBZ1P2mMeBhYDL+yqPTpune77y4HnAveNfdoGPtPqh0lySZKvdC37MmDZJF/DIb22APsunT2SMd/qmv4BnTfoSUvyQDtk9XSS101l3eZp4JSu+6cAT5ffeLlgLYRBNs0/jwIvSrJ4XCjsAV7cdf9FwAHg28DKVhv/ZtZ9/zvAKHBOO2Y+oSQvBv4IuJDOoaGDSb4CjH16Ptqb5iG9JnkecBpwxOdty74OuOMIi/xCVX2hqo718M4DdA5X3dvuv7LVtEC5h6C56F7gMeCqJM9L8pwk5wM3A+9IcmaSk+kc7rm1x55ET1X1Qzpv8tckeQFAksEka3ss/jw6b/p723Jvo7OHMObbwMokPzrB030MeFuSVyU5qfV6T1V9cxJ9fqGNTUx0+8JE6yZZkuQ5dH63F7dtt2iCxW8C3tm2wQrg3wM3HK0/nbgMBM05VXUQ+Gd0Bn0fAXYD/wK4HvgocBfwEJ0xgV+b4sP/Op1TM7+U5PvA/wRW9+jhQTpn3dxN583/5cBfdi1yJ51P099K8p0e638e+A3gz+iE208ywVjFDPsjOntB/xJ4d5v+FejseSR5umvZPwT+B7ADuJ/Oqap/OAs9ao6KhwslSeAegiSpMRAkSYCBIElqDARJEjCPr0NYtmxZrVq1qt9tSNK8ct99932nqnpejDlvA2HVqlUMDw/3uw1JmleSPDzRPA8ZSZIAA0GS1BgIkiTAQJAkNQaCJAmYx2cZTcfW7SNs3raTPftGWbF0gI1rV7NuzeDRV5SkBWDBBMLW7SNs2rKD0f0HARjZN8qmLTsADAVJYgEdMtq8beezYTBmdP9BNm/b2aeOJGluWTCBsGff6JTqkrTQLJhAWLF0YEp1SVpoFkwgbFy7moElh/4lwYEli9i49rA/liVJC9KCGVQeGzj2LCNJ6m3BBAJ0QsEAkKTeFswhI0nSkRkIkiTAQJAkNQaCJAkwECRJjYEgSQIMBElSYyBIkgADQZLUGAiSJMBAkCQ1BoIkCTAQJEmNgSBJAgwESVJjIEiSAANBktQYCJIkwECQJDUGgiQJMBAkSY2BIEkCDARJUnPUQEhyfZLHk9w/rv5rSXYmeSDJb3fVNyXZ1eat7aq/JsmONu/aJGn1k5Lc2ur3JFk1g69PkjRJk9lDuAG4qLuQ5OeAi4FXVNU5wAda/WxgPXBOW+eDSRa11T4EbADOarexx7wM+F5VvQS4Brj6GF6PJGmajhoIVXUX8MS48uXAVVX1TFvm8Va/GLilqp6pqoeAXcC5SU4HTqmqu6uqgJuAdV3r3NimPw5cOLb3IEmaPdMdQ3gp8Lp2iOd/J/npVh8EHu1abnerDbbp8fVD1qmqA8CTwGm9njTJhiTDSYb37t07zdYlSb1MNxAWA88HzgM2Are1T/W9PtnXEeocZd6hxarrqmqoqoaWL18+9a4lSROabiDsBrZUx73AD4FlrX5G13IrgT2tvrJHne51kiwGTuXwQ1SSpONsuoGwFXg9QJKXAj8KfAe4HVjfzhw6k87g8b1V9RjwVJLz2p7EJcAn22PdDlzapt8M3NnGGSRJs2jx0RZIcjNwAbAsyW7gvcD1wPXtVNT/C1za3sQfSHIb8CBwALiiqg62h7qczhlLA8Ad7QbwEeCjSXbR2TNYPzMvTZI0FZmvH8aHhoZqeHi4321I0ryS5L6qGuo1zyuVJUmAgSBJao46hqCFa+v2ETZv28mefaOsWDrAxrWrWbdm8OgrSpqXDAT1tHX7CJu27GB0f+ecgJF9o2zasgPAUJBOUB4yUk+bt+18NgzGjO4/yOZtO/vUkaTjzUBQT3v2jU6pLmn+MxDU04qlA1OqS5r/DAT1tHHtagaWLDqkNrBkERvXru5TR5KONweV1dPYwLFnGUkLh4GgCa1bM2gASAuIh4wkSYCBIElqPGSkCXmlsrSwGAjqySuVpYXHQ0bqySuVpYXHQFBPXqksLTwGgnrySmVp4TEQ1JNXKksLj4PK6skrlaWFx0DQhLxSWVpYPGQkSQIMBElSYyBIkgADQZLUGAiSJMBAkCQ1BoIkCTAQJEmNgSBJAgwESVJjIEiSAANBktQYCJIkYBKBkOT6JI8nub/HvP+QpJIs66ptSrIryc4ka7vqr0myo827Nkla/aQkt7b6PUlWzdBrkyRNwWT2EG4ALhpfTHIG8I+AR7pqZwPrgXPaOh9MMvZXVj4EbADOarexx7wM+F5VvQS4Brh6Oi9EknRsjhoIVXUX8ESPWdcA7wKqq3YxcEtVPVNVDwG7gHOTnA6cUlV3V1UBNwHruta5sU1/HLhwbO9BkjR7pjWGkOSNwEhVfXXcrEHg0a77u1ttsE2Prx+yTlUdAJ4ETpvgeTckGU4yvHfv3um0LkmawJQDIclzgXcD7+k1u0etjlA/0jqHF6uuq6qhqhpavnz5ZNqVJE3SdPYQfhI4E/hqkm8CK4EvJ/kJOp/8z+hadiWwp9VX9qjTvU6SxcCp9D5EJUk6jqYcCFW1o6peUFWrqmoVnTf0V1fVt4DbgfXtzKEz6Qwe31tVjwFPJTmvjQ9cAnyyPeTtwKVt+s3AnW2cQZI0iyZz2unNwN3A6iS7k1w20bJV9QBwG/Ag8Bngiqo62GZfDvwxnYHmbwB3tPpHgNOS7ALeCVw5zdciSToGma8fxoeGhmp4eLjfbUjSvJLkvqoa6jXPK5UlSYCBIElqDARJEmAgSJIaA0GSBBgIkqTGQJAkAQaCJKkxECRJgIEgSWoMBEkSYCBIkprF/W5AOlFs3T7C5m072bNvlBVLB9i4djXr1gwefUVpjjAQpBmwdfsIm7bsYHR/59veR/aNsmnLDgBDQfOGh4ykGbB5285nw2DM6P6DbN62s08dSVNnIEgzYM++0SnVpbnIQJBmwIqlA1OqS3ORgSDNgI1rVzOwZNEhtYEli9i4dnWfOpKmzkFlaQaMDRx7lpHmMwNBmiHr1gwaAJrXPGQkSQIMBElS4yEjaYZ4pbLmOwNBmgFeqawTgYeMpBnglco6ERgI0gzwSmWdCAwEaQZ4pbJOBAaCNAO8UlknAgeVpRnglco6ERgI0gzxSmXNdx4ykiQBBoIkqTEQJEnAJAIhyfVJHk9yf1dtc5K/TvJXST6RZGnXvE1JdiXZmWRtV/01SXa0edcmSauflOTWVr8nyaqZfYmSpMmYzB7CDcBF42qfA15WVa8A/gbYBJDkbGA9cE5b54NJxs7F+xCwATir3cYe8zLge1X1EuAa4OrpvhhJ0vQdNRCq6i7giXG1z1bVgXb3S8DKNn0xcEtVPVNVDwG7gHOTnA6cUlV3V1UBNwHruta5sU1/HLhwbO9BkjR7ZmIM4V8Bd7TpQeDRrnm7W22wTY+vH7JOC5kngdN6PVGSDUmGkwzv3bt3BlqXJI05pkBI8m7gAPCnY6Uei9UR6kda5/Bi1XVVNVRVQ8uXL59qu5KkI5h2ICS5FHgD8JZ2GAg6n/zP6FpsJbCn1Vf2qB+yTpLFwKmMO0QlSTr+phUISS4Cfh14Y1X9oGvW7cD6dubQmXQGj++tqseAp5Kc18YHLgE+2bXOpW36zcCdXQEjSZolR/3qiiQ3AxcAy5LsBt5L56yik4DPtfHfL1XV26vqgSS3AQ/SOZR0RVWNfUn85XTOWBqgM+YwNu7wEeCjSXbR2TNYPzMvTZI0FZmvH8aHhoZqeHi4321I0ryS5L6qGuo1zyuVJUmAgSBJagwESRJgIEiSGgNBkgQYCJKkxkCQJAEGgiSpMRAkSYCBIElqDARJEmAgSJIaA0GSBBgIkqTGQJAkAQaCJKkxECRJgIEgSWoMBEkSYCBIkhoDQZIEGAiSpGZxvxuQtHBt3T7C5m072bNvlBVLB9i4djXr1gz2u60Fy0CQ1Bdbt4+wacsORvcfBGBk3yibtuwAMBT6xENGkvpi87adz4bBmNH9B9m8bWefOpKBIKkv9uwbnVJdx5+BIKkvViwdmFJdx5+BIKkvNq5dzcCSRYfUBpYsYuPa1X3qSA4qS+qLsYFjzzKaOwwESX2zbs2gATCHeMhIkgQYCJKk5qiBkOT6JI8nub+r9uNJPpfk6+3f53fN25RkV5KdSdZ21V+TZEebd22StPpJSW5t9XuSrJrh1yhpjtq6fYTzr7qTM6/8NOdfdSdbt4/0u6UFbTJ7CDcAF42rXQl8vqrOAj7f7pPkbGA9cE5b54NJxk4j+BCwATir3cYe8zLge1X1EuAa4OrpvhhJ88fYlcoj+0Yp/v+VyoZC/xw1EKrqLuCJceWLgRvb9I3Auq76LVX1TFU9BOwCzk1yOnBKVd1dVQXcNG6dscf6OHDh2N6DpBOXVyrPPdMdQ3hhVT0G0P59QasPAo92Lbe71Qbb9Pj6IetU1QHgSeC0Xk+aZEOS4STDe/funWbrkuYCr1See2Z6ULnXJ/s6Qv1I6xxerLquqoaqamj58uXTbFHSXOCVynPPdAPh2+0wEO3fx1t9N3BG13IrgT2tvrJH/ZB1kiwGTuXwQ1SSTjBeqTz3TDcQbgcubdOXAp/sqq9vZw6dSWfw+N52WOmpJOe18YFLxq0z9lhvBu5s4wySTmDr1gzy/je9nMGlAwQYXDrA+9/0ci9U66OjXqmc5GbgAmBZkt3Ae4GrgNuSXAY8AvwiQFU9kOQ24EHgAHBFVY2NGl1O54ylAeCOdgP4CPDRJLvo7Bmsn5FXJmnO80rluSXz9cP40NBQDQ8P97sNSZpXktxXVUO95nmlsiQJMBAkSY2BIEkCDARJUmMgSJIAA0GS1BgIkiTAQJAkNQaCJAkwECRJjYEgSQIMBElSYyBIkgADQZLUGAiSJMBAkCQ1BoIkCTAQJEmNgSBJAgwESVJjIEiSAANBktQYCJIkwECQJDUGgiQJMBAkSY2BIEkCDARJUmMgSJIAA0GS1BgIkiTAQJAkNQaCJAk4xkBI8o4kDyS5P8nNSZ6T5MeTfC7J19u/z+9aflOSXUl2JlnbVX9Nkh1t3rVJcix9SdKJaOv2Ec6/6k7OvPLTnH/VnWzdPjKjjz/tQEgyCPwbYKiqXgYsAtYDVwKfr6qzgM+3+yQ5u80/B7gI+GCSRe3hPgRsAM5qt4um25cknYi2bh9h05YdjOwbpYCRfaNs2rJjRkPhWA8ZLQYGkiwGngvsAS4GbmzzbwTWtemLgVuq6pmqegjYBZyb5HTglKq6u6oKuKlrHUkSsHnbTkb3HzykNrr/IJu37Zyx55h2IFTVCPAB4BHgMeDJqvos8MKqeqwt8xjwgrbKIPBo10PsbrXBNj2+fpgkG5IMJxneu3fvdFuXpHlnz77RKdWn41gOGT2fzqf+M4EVwPOS/PKRVulRqyPUDy9WXVdVQ1U1tHz58qm2LEnz1oqlA1OqT8exHDL6eeChqtpbVfuBLcA/AL7dDgPR/n28Lb8bOKNr/ZV0DjHtbtPj65KkZuPa1QwsWXRIbWDJIjauXT1jz3EsgfAIcF6S57azgi4EvgbcDlzalrkU+GSbvh1Yn+SkJGfSGTy+tx1WeirJee1xLulaR5IErFszyPvf9HIGlw4QYHDpAO9/08tZt6bnEfZpWTzdFavqniQfB74MHAC2A9cBJwO3JbmMTmj8Ylv+gSS3AQ+25a+oqrERksuBG4AB4I52kyR1WbdmcEYDYLx0TuyZf4aGhmp4eLjfbUjSvJLkvqoa6jXPK5UlSYCBIElqDARJEmAgSJKaeTuonGQv8PA0V18GfGcG25kp9jU19jV1c7U3+5qaY+nrxVXV88reeRsIxyLJ8ESj7P1kX1NjX1M3V3uzr6k5Xn15yEiSBBgIkqRmoQbCdf1uYAL2NTX2NXVztTf7mprj0teCHEOQJB1uoe4hSJLGMRAkScAJHAhJrk/yeJL7J5ifJNcm2ZXkr5K8eo70dUGSJ5N8pd3eM0t9nZHkL5J8LckDSf5tj2VmfZtNsq9Z32ZJnpPk3iRfbX39Zo9l+rG9JtNXX37G2nMvSrI9yad6zOvL7+Qk+urX7+Q3k+xoz3nYN3kel+1VVSfkDfhZ4NXA/RPM/yd0vmY7wHnAPXOkrwuAT/Vhe50OvLpN/xjwN8DZ/d5mk+xr1rdZ2wYnt+klwD3AeXNge02mr778jLXnfifwsV7P36/fyUn01a/fyW8Cy44wf8a31wm7h1BVdwFPHGGRi4GbquNLwNKxv/TW5776oqoeq6ovt+mn6Pyxo/FfvD7r22ySfc26tg2ebneXtNv4MzT6sb0m01dfJFkJ/FPgjydYpC+/k5Poa66a8e11wgbCJAwCj3bd380ceKNpXtt2+e9Ics5sP3mSVcAaOp8uu/V1mx2hL+jDNmuHGb5C58/Efq6q5sT2mkRf0J+fsd8D3gX8cIL5/fr5+j2O3Bf0Z3sV8Nkk9yXZ0GP+jG+vhRwI6VGbC5+kvkznu0ZeCfw+sHU2nzzJycCfAf+uqr4/fnaPVWZlmx2lr75ss6o6WFWvovN3wM9N8rJxi/Rle02ir1nfXkneADxeVfcdabEeteO6vSbZV79+J8+vqlcDvwBckeRnx82f8e21kANhN3BG1/2VwJ4+9fKsqvr+2C5/Vf05sCTJstl47iRL6Lzp/mlVbemxSF+22dH66uc2a8+5D/hfwEXjZvX1Z2yivvq0vc4H3pjkm8AtwOuT/Mm4ZfqxvY7aV79+vqpqT/v3ceATwLnjFpnx7bWQA+F24JI2Un8e8GRVPdbvppL8RJK06XPp/B99dxaeN8BHgK9V1e9OsNisb7PJ9NWPbZZkeZKlbXoA+Hngr8ct1o/tddS++rG9qmpTVa2sqlXAeuDOqvrlcYvN+vaaTF99+vl6XpIfG5sG/jEw/szEGd9ei49l5bksyc10zg5YlmQ38F46A2xU1YeBP6czSr8L+AHwtjnS15uBy5McAEaB9dVOKTjOzgd+BdjRjj8D/EfgRV299WObTaavfmyz04Ebkyyi8wZxW1V9Ksnbu/rqx/aaTF/9+hk7zBzYXpPpqx/b64XAJ1oOLQY+VlWfOd7by6+ukCQBC/uQkSSpi4EgSQIMBElSYyBIkgADQZLUGAiSJMBAkCQ1/w/GVkLWrmssTQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(data_population['age'],data_population['price'])\n",
    "plt.title('correlation={}'.format(round(np.corrcoef(x=data_population['age'], y=data_population['price'])[0][1],2)))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d8fd61a3",
   "metadata": {},
   "source": [
    "The omitted variable is correlated to:\n",
    "- the independent variable\n",
    "- the dependent variable (=> it also explains the result)\n",
    "- the error (=> it is \"hidden\" in the error)\n",
    "\n",
    "=> endogeneity."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "fc1f8f48",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAD4CAYAAAAHHSreAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAdWUlEQVR4nO3de3Bc9Znm8e9rSbZlg5EvsSzJBpPYKMF2ZoQdhoSQVSAZkUwmeFJhYpYM3g1bVFiWSZjEGxRqN7NT5YnBSWWLSsKMK7BAJuEyrBFUahjD2HTIMsY24IAwIDDBwbqALyBjEdmWxbt/nF/b3X26dWlL6mPr+VSd0un3XPrtttWPzvmd7jZ3R0REJNOEUjcgIiLJo3AQEZEYhYOIiMQoHEREJEbhICIiMeWlbqBYs2bN8vnz55e6jWPee+89pk6dWuo2Ckp6f5D8HpPeHyS/x6T3B6d+j88888w+d//AoCu6+0k5LV261JPk8ccfL3ULA0p6f+7J7zHp/bknv8ek9+d+6vcIPO1DeI3VaSUREYlROIiISIzCQUREYhQOIiISo3AQEZGYk/ZS1mK0bO9g7YY2Ort7qa2qZFVTPcsb6krdlohI4oybcGjZ3kHz+lZ6+/oB6OjupXl9K4ACQkQkx7g5rbR2Qxu9ff0sa9/BDb/5Jyr6++jt62fthrZStyYikjjjJhw6u3sBOK/jZb7x7/dS3t+fVRcRkePGTTjUVlUOqy4iMp6Nm3BY1VRPZUVZVq2yooxVTfUl6khEJLnGzYB0etC545kHAaitmsz1f75Eg9EiInmMm3CAEBCfXgCPwMZvNULCP3lRRKRUxs1pJRERGTqFg4iIxCgcREQkRuEgIiIxCgcREYlROIiISIzCQUREYhQOIiISo3AQEZEYhYOIiMQoHEREJEbhICIiMQoHERGJUTiIiEiMwkFERGIUDiIiEqNwEBGRGIWDiIjEKBxERCRG4SAiIjEKBxERiVE4iIhIjMJBRERiBg0HM7vDzPaY2Qs59evNrM3MdpjZLRn1ZjPbGZY1ZdSXmllrWHarmVmoTzKz+0J9i5nNH8HHJyIiRRjKkcOdwKWZBTP7NHAZ8FF3XwT8INTPBVYAi8I2PzWzsrDZbcA1wMIwpfd5NfCOuy8AfgTcfAKPR0RERsCg4eDuTwBv55SvBda4++Gwzp5Qvwy4190Pu/vrwE7gfDOrAaa5+2Z3d+BuYHnGNneF+QeAS9JHFSIiUhrlRW53DnCRma0GDgHfdvdtQB3wVMZ67aHWF+Zz64SfuwHc/aiZHQBmAvty79TMriE6+qC6uppUKjXsxue99hofAp544gner6wc9vaF9PT0FNXPWEl6f5D8HpPeHyS/x6T3B+oxrdhwKAemAxcAHwPuN7MPAvn+4vcB6gyyLLvovg5YB7Bs2TJvbGwcXtcA27YB8KlPfQqmTh3+9gWkUimK6meMJL0/SH6PSe8Pkt9j0vsD9ZhW7NVK7cB6j2wF3gdmhfq8jPXmAp2hPjdPncxtzKwcOIP4aSwRERlDxYZDC3AxgJmdA0wkOg30MLAiXIF0NtHA81Z37wIOmtkFYTzhKuChsK+HgZVh/svApjAuISIiJTLoaSUzuwdoBGaZWTvwPeAO4I5weesRYGV4Qd9hZvcDLwJHgevcvT/s6lqiK58qgUfCBHA78HMz20l0xLBiZB6aiIgUa9BwcPcrCiz6aoH1VwOr89SfBhbnqR8CLh+sDxERGTt6h7SIiMQoHEREJGZchUPL9g5+8vhOAC75YYqW7R0l7khEJJnGTTi0bO+geX0rB3r7AOjsPkTz+lYFhIhIHsW+Ce6ks3ZDG719/cdu//Lem+icNov3Ns2GL/4J1NZCXV30s7YWTj8d9CkeIjJOjZtw6OzuBWDDOR/nnH1vMOfgPur3/p7Zr2+Hf38wvsHUqdlhkRse6Wny5DF+JCIio2/chENtVSUd3b38fnot3/6zG47V66oqefK/nQ9dXdDRAZ2d0ZQ5/9RT0e3Dh+M7njEDamv56OTJsGRJ/jCprobycfNUi8gpYNy8Yq1qqqd5fWvWqaXKijJWNdXDaafBwoXRVIg7vPNO/vDo7KT85Zfh0UfhzTehvz97W7MoIAY7Epk1S6eyRCQRxk04LG+IPgR27YY2Ort7qa2qZFVT/bH6oMyio4QZM2Bx7L18PJv+IKz+fti7NxYex26/8UZ0JLJ3b/w+KioGP41VV6fxEBEZdeMmHCAKiCGHQbHKymDOnGhaurTweocPR0cZBY5E2LEDHnsM3n03vu3UqYUDJH27pgZG8CPJRWR8GVfhkCiTJsFZZ0XTQHp64uMhmWEy0HjI9OnHwqJ+woQobHLDZM4cjYeISIxeFZKumPGQPEciM373uygcCo2HDHYkMnMmTBg3b4sRGfcUDqeCQcZDADanUjRedNHA4yG7d8OWLYXHQ2pqCo+DpOenTRv2eEjL9g7WbmhjxbyD3LRm0/DGgkRkVCgcxpOxHg8ZaGA9jIek37ne29cP86Cju5fm9a0ACgiRElI4SNxIjYds2TLweEhtLbWHJ/N3k6fz5ukzmVr7WWAOvX39rN3QpnAQKSGFgxRvBMZDJm55kU++uYuanv1sriuDD/1H4Pg72kWkNBQOMroGGQ+5bs0mOt75A7tu+XMmvP/+sXptlS7DFSklXX4iJbWqqZ7KirKs2rF3rotIyejIQUpqeUNddOopfLFs3XDfuS4io0LhICWXDoLZ0ybz5I0Xl7gbEQGdVhIRkTwUDiIiEqNwEBGRGIWDiIjEKBxERCRG4SAiIjEKBxERiVE4iIhIjMJBRERiFA4iIhKjcBARkRiFg4iIxCgcREQkZtBwMLM7zGyPmb2QZ9m3zczNbFZGrdnMdppZm5k1ZdSXmllrWHarWfQt9GY2yczuC/UtZjZ/hB6biIgUaShHDncCl+YWzWwe8FngjYzaucAKYFHY5qdmlv4ml9uAa4CFYUrv82rgHXdfAPwIuLmYByIiIiNn0HBw9yeAt/Ms+hHw3wHPqF0G3Ovuh939dWAncL6Z1QDT3H2zuztwN7A8Y5u7wvwDwCXpowoRESmNor7sx8y+CHS4+3M5r+N1wFMZt9tDrS/M59bT2+wGcPejZnYAmAnsy3O/1xAdfVBdXU0qlSqm/VHR09OTqH5yJbo/dxqBI0eOJLdHEv4cBknvMen9gXpMG3Y4mNkU4CbgT/MtzlPzAeoDbRMvuq8D1gEsW7bMGxsbB2t3zKRSKZLUT65E9+fRP/fEiROT2yMJfw6DpPeY9P5APaYVc7XSh4CzgefMbBcwF3jWzOYQHRHMy1h3LtAZ6nPz1MncxszKgTPIfxpLRETGyLDDwd1b3X22u8939/lEL+7nufubwMPAinAF0tlEA89b3b0LOGhmF4TxhKuAh8IuHwZWhvkvA5vCuISIiJTIUC5lvQfYDNSbWbuZXV1oXXffAdwPvAj8K3Cdu/eHxdcCPyMapH4NeCTUbwdmmtlO4G+AG4t8LCIiMkIGHXNw9ysGWT4/5/ZqYHWe9Z4GFuepHwIuH6wPEREZO3qHtIiIxCgcREQkRuEgIiIxCgcREYlROIiISIzCQUREYhQOIiISo3AQEZEYhYOIiMQoHEREJEbhICIiMQoHERGJUThIybVs7wBgz7uHuHDNpmO3RaR0FA5SUi3bO2he33rsdkd3L83rWxUQIiVW1HdIi4yUtRva6O2LvvJj0eOP8bPnXmPPaTPZt7kaVnwSamuPT7NmwQT9PSMyFhQOUlKd3b1gxtqL/orL332R2q59/HHXK8x67gBsvCt75YoKqKk5HhZ1ddnhkb49bRpYvq8mF5GhUjhISdVWVdLR3ctPPvEVJi85yg9bo/+SZ51Wzq+v+gh0dEBn5/Epffull2DjRjhwIL7TKVMGDo/0VFk5xo9W5OShcJCSWtVUT/P61mOnlgAqK8q44c8Ww5l1cOaZA++gpwe6uuLhkZ62bIl+HjoU33b69IHDo64OqqujIxaRcUbhICW1vKEOiMYe4CB1VZWsaqo/Vh/UaafBwoXRVIg7dHfnD4/MI5GuLujvz97WDGbPhtpaFk+eDEuW5A8TjYfIKUbhICW3vKGO5Q11pFIprr+yceTvwCw6Spg+HRYtKrxefz/s3ZsdHhkBMumVV6ClBfbsiW+r8RA5xSgcRNLKymDOnGg677zY4mdSKRobG+HIEXjzzcKnsjQeIqcAhYPIcE2cGI2FDDYe8t570amqQqeytm6N5jUeIgmkcBAZLVOnwoIF0VRI5njIQEcig4yHpMPjrCNH4NVXT3g8pGV7B2s3tNHZ3UvtcMeB5JSgcBAppaGOh7z/fvZ4SG6AtLfD1q2cvWcP3Hln9ra54yGFjkTCeEj6XevpK8jS71oHFBDjiMJB5GQwYUJ0Gqm6GhoaCq7268ce4z98+MOFT2W9/DJs2hQdreQK4yFn9U9hzeTpvHXaDLbNW8RjCy+gt6+ftRvaFA7jiMJB5BTiFRUwb140DSQ9HpLnSOTIky/wR12vUHNwLyuef5SPfvM+ILybXcYNhYPIeDTAeMjfrNlER3cv3910O1f+9pFj9doqXUE1nuhdOyKSZVVTPZUVZVm1yooyVjXVl6gjKQUdOYhIlvS4wh9+Hb08DPtd63JKUDiISMzyhjo4/0x4townb7y41O1ICei0koiIxCgcREQkRuEgIiIxg4aDmd1hZnvM7IWM2loze9nMnjezB82sKmNZs5ntNLM2M2vKqC81s9aw7Faz6KMpzWySmd0X6lvMbP7IPkQRERmuoRw53AlcmlN7DFjs7h8FXgGaAczsXGAFsChs81MzS18TdxtwDbAwTOl9Xg284+4LgB8BNxf7YEREZGQMGg7u/gTwdk7tUXc/Gm4+BcwN85cB97r7YXd/HdgJnG9mNcA0d9/s7g7cDSzP2Cb9ZcEPAJekjypERKQ0RuJS1q8B94X5OqKwSGsPtb4wn1tPb7MbwN2PmtkBYCawL/eOzOwaoqMPqqurSaVSI9D+yOjp6UlUP7mS3h8kv8ek9wcj2+OHdu+mtr+f34zgYx5vz+FoGYseTygczOwm4Cjwi3Qpz2o+QH2gbeJF93XAOoBly5Z5Y2PjcNodVan0F8EkVNL7g+T3mPT+YIR7/NWvoKxsRB/zuHsOR8lY9Fj01UpmthL4AnBlOFUE0RFB5id+zQU6Q31unnrWNmZWDpxBzmksEREZW0WFg5ldCnwH+KK7/yFj0cPAinAF0tlEA89b3b0LOGhmF4TxhKuAhzK2WRnmvwxsyggbEREpgUFPK5nZPUAjMMvM2oHvEV2dNAl4LIwdP+XuX3f3HWZ2P/Ai0emm69w9/fVV1xJd+VQJPBImgNuBn5vZTqIjhhUj89BERKRYg4aDu1+Rp3z7AOuvBlbnqT8NLM5TPwRcPlgfIiIydvQOaRERiVE4iIhIjMJBRERiFA4iIhKjcBARkRiFg4iIxCgcREQkRuEgIiIxCgcREYlROIiISIzCQUREYhQOIiISo3AQEZEYhYOIiMQoHEREJEbhICIiMQoHERGJUTiIiEiMwkFERGIUDiIiEqNwEBGRGIWDiIjEKBxEJKZlewe/3PoG7x3p58I1m2jZ3lHqlmSMKRxEJEvL9g6a17fSc+goAB3dvTSvb1VAjDPlpW5ARJJl7YY2evv6AajsO8zd9/0P9pw2g3f/32z48oVQW3t8mjMHKipK3LGMBoWDiGTp7O4F4FcfuYgzD7zJnIP7WbB/N7N3vA2pe7JXNoPZs4+HRV1ddnikb8+aBRN0ouJkonAQkSy1VZV0dPfyfM05fP0vbjpWnzdtIr/52hLo7MyeOjqO/9y2Dfbsie+0vBxqamg4/XT48IcLh8kZZ0SBIyWncBCRLKua6mle33rs1BJAZUUZ3/rcudFppDlz4LzzCu/gyBF48814eHR20r9jB7z0EmzcCAcOxLedMiX/kUfm7ZqaaD0ZVQoHEcmyvKEOiMYeOrt7qa2qZFVT/bH6oCZOhDPPjKYcz6dSNDY2Rjfeew+6urLCIytMtm2DlhY4dCh+H1VV8fDIDRKNh5wQhYOIxCxvqBt6GBRr6lRYsCCaCnGH7u784ZGeXnopCpn+/uxtNR5yQhQOIpJcZjB9ejQtWlR4vfffh71784fHEMdD0mGxwB02bx734yEKBxE5+U2YANXV0dTQUHi9I0fgrbcKn8p6+WWq33gDHnwwvu04Gw8ZNBzM7A7gC8Aed18cajOA+4D5wC7gL939nbCsGbga6Af+2t03hPpS4E6gEvgX4Bvu7mY2CbgbWArsB77i7rtG7BGKiKRNnAjz5kVTAU+mUjR+7GPRqapCp7K2bYOHHoLe3vgO8o2H5N4ucjykZXsHaze0sWLeQW5as2l4Y0HDNJQjhzuBHxO9gKfdCGx09zVmdmO4/R0zOxdYASwCaoF/M7Nz3L0fuA24BniKKBwuBR4hCpJ33H2Bma0Abga+MhIPTkSkKEMdDzlwIH94ZByJ0NUFR49mb5s7HlIoTDLGQ9LvXO/t64d5x9+5DoxKQAwaDu7+hJnNzylfBjSG+buAFPCdUL/X3Q8Dr5vZTuB8M9sFTHP3zQBmdjewnCgcLgP+NuzrAeDHZmbu7sU+KBGRUWcWHSVUVcG55xZeL3M8ZKAjkYHGQ2prqeqZyHcmnUHntA8w6YOXAFX09vWzdkNbacKhgGp37wJw9y4zmx3qdURHBmntodYX5nPr6W12h30dNbMDwExgX+6dmtk1REcfVFdXk0qlimx/5PX09CSqn1xJ7w+S32PS+4Pk95j0/mCUe5w6FRYujKYcdvQoE99+m4n79jFp3z4m7d8fze/fz8T9+1nSvYuPv7OfSX/4A9s+MoVvXfjZsOXBUel3pAek8w3l+wD1gbaJF93XAesAli1b5seul06AVOb12wmU9P4g+T0mvT9Ifo9J7w+S2+OFazZxpL2TbT/5K3oO9fPD1ujlu66qkuuvbBzx+yv24t63zKwGIPxMHw+1A5kjPXOBzlCfm6eetY2ZlQNnAG8X2ZeIyClpVVM9kyuyX7IrK8pY1VQ/KvdXbDg8DKwM8yuBhzLqK8xskpmdDSwEtoZTUAfN7AIzM+CqnG3S+/oysEnjDSIi2ZY31PHdz3/k2O26qkq+/6UlpbtayczuIRp8nmVm7cD3gDXA/WZ2NfAGcDmAu+8ws/uBF4GjwHXhSiWAazl+KesjYQK4Hfh5GLx+m+hqJxERyfG5JTVAFAxP3njxqN7XUK5WuqLAoksKrL8aWJ2n/jSwOE/9ECFcREQkGfSBIiIiEqNwEBGRGIWDiIjEKBxERCRG4SAiIjEKBxERiVE4iIhIjMJBRERiFA4iIhKjcBARkRiFg4iIxCgcREQkRuEgIiIxCgcREYlROIiISIzCQUREYhQOIiISo3AQEZEYhYOIiMQoHEREJEbhICIiMQoHERGJUTiIiEiMwkFERGIUDiIiEqNwEBGRGIWDiIjEKBxERCRG4SAiIjEKBxERiVE4iIhIjMJBRERiFA4iIhJzQuFgZjeY2Q4ze8HM7jGzyWY2w8weM7NXw8/pGes3m9lOM2szs6aM+lIzaw3LbjUzO5G+RERORY+0dgHQ0d3LhWs20bK9Y9Tuq+hwMLM64K+BZe6+GCgDVgA3AhvdfSGwMdzGzM4NyxcBlwI/NbOysLvbgGuAhWG6tNi+RERORS3bO/j7f3np2O2O7l6a17eOWkCc6GmlcqDSzMqBKUAncBlwV1h+F7A8zF8G3Ovuh939dWAncL6Z1QDT3H2zuztwd8Y2IiICrN3QxqG+97NqvX39rN3QNir3V17shu7eYWY/AN4AeoFH3f1RM6t2966wTpeZzQ6b1AFPZeyiPdT6wnxuPcbMriE6wqC6uppUKlVs+yOup6cnUf3kSnp/kPwek94fJL/HpPcHye1xxbyDTJphvHr+J5gwt5pv1R8NSw6OSr9Fh0MYS7gMOBvoBv7ZzL460CZ5aj5APV50XwesA1i2bJk3NjYOo+PRlUqlSFI/uZLeHyS/x6T3B8nvMen9QXJ7vGnNJjq6y/n7T3+Xb9Uf5Yet0ct3XVUl11/ZOOL3dyKnlT4DvO7ue929D1gPfAJ4K5wqIvzcE9ZvB+ZlbD+X6DRUe5jPrYuISLCqqZ7KirKsWmVFGaua6kfl/k4kHN4ALjCzKeHqokuAl4CHgZVhnZXAQ2H+YWCFmU0ys7OJBp63hlNQB83sgrCfqzK2ERERYHlDHd//0hLqqiqB6Ijh+19awvKGvGfhT9iJjDlsMbMHgGeBo8B2olM+pwH3m9nVRAFyeVh/h5ndD7wY1r/O3fvD7q4F7gQqgUfCJCIiGZY31LG8oY5UKjUqp5IyFR0OAO7+PeB7OeXDREcR+dZfDazOU38aWHwivYiIyMjRO6RFRCRG4SAiIjEKBxERiVE4iIhIjEWfWHHyMbO9wO9L3UeGWcC+UjcxgKT3B8nvMen9QfJ7THp/cOr3eJa7f2CwlU7acEgaM3va3ZeVuo9Ckt4fJL/HpPcHye8x6f2BekzTaSUREYlROIiISIzCYeSsK3UDg0h6f5D8HpPeHyS/x6T3B+oR0JiDiIjkoSMHERGJUTiIiEicu2sKE1AFPAC8TPTx4x8HZgCPAa+Gn9Mz1m8m+rrTNqApo74UaA3LbuX46btJwH2hvgWYX0SPNwA7gBeAe4DJpe4RuIPoezteyKiNSU9EHwv/aphWDqO/teHf+XngQaCqVP0V6jFj2beJvgBrVpKew1C/PvSwA7glac8h8MdE30D5W+Bp4PwSPofzgMeJXlt2AN9I2u9KVr/DfXE6lSei77z+L2F+IlFY3ALcGGo3AjeH+XOB58I/xtnAa0BZWLaVKFiM6OPHPxfq/xX4hzC/ArhvmP3VAa8DleH2/cB/KnWPwKeA88j+pRz1nsIv1e/Cz+lhfvoQ+/tToDzM31zK/gr1GOrzgA1Eb/iclbDn8NPAvwGTwu3ZSXsOgUcz7uPzQKqEz2ENcF6YPx14JfSRmN+VrH5H40X2ZJyAaUQvvJZTbwNqMv5x28J8M9Ccsd6G8I9VA7ycUb8C+MfMdcJ8OdE7HG0YPdYBu8M/cDnwK6IXuZL3CMwn+5dy1HvKXCcs+0fgiqH0l7PsL4BflLK/Qj0SHcn+EbCL4+GQiOeQ6I+Tz+RZLzHPYdjvVzLu75el7jFjvYeAz5Kw35X0pDGH4z4I7AX+j5ltN7OfmdlUoNqjb6sj/Jwd1k+/UKe1h1pdmM+tZ23j7keBA8DMoTbo7h3AD4i+RKkLOODujyapxwxj0VOhfQ3X1zj+BVOJ6c/Mvgh0uPtzOYuS0uM5wEVmtsXMfm1mH0tYfwDfBNaa2W6i353mJPRoZvOBBqJTP4n8XVE4HFdOdEh6m7s3AO8RHeIVYnlqPkB9oG2GxMymA5cRHWLWAlPN7KtJ6nEIRrKnE+7VzG4i+mbCXySpPzObAtwE/M98i5PQI9HvzHTgAmAV0TdAWoL6g+hbJm9w93lE43W3n8D9jUiPZnYa8H+Bb7r7u4XWK2WPoHDI1A60u/uWcPsBorB4y8xqAMLPPRnrz8vYfi7QGepz89SztjGzcuAM4O1h9PgZ4HV33+vufcB64BMJ6zFtLHoqtK8hMbOVwBeAKz0cayeovw8R/RHwnJntCts+a2ZzEtRjO7DeI1uB94k+EC4p/UE0CLs+zP8zcH7u/Y1lj2ZWQRQMv3D3dF/J/F0Z7LzYeJqA3wD1Yf5via5oWUv2YNEtYX4R2YNFv+P4YNE2or+m0oNFnw/168geLLp/mP39CdFVDlPCvu8iulqk5D0SP9c76j0Rjb28TvTX6/QwP2OI/V1K9H3mH8hZryT95esxZ9kujo85JOU5/Drwd2H+HKLTFpak55DoyqDGMH8J8EypnsOwv7uB/51TT9TvyrG+inkRPVUnosvenia6vLElPIkzgY1El39tzHxCiQ77XyMaUPpcRn0Z0aWmrwE/5vhlZpOJ/nrZSXS1wQeL6PF/EV2C+QLw8/Afp6Q9El1S2wX0Ef2FcvVY9UQ0XrAzTP95GP3tJHox+22Y/qFU/RXqMWf5LrIvZU3CczgR+Kdwf88CFyftOQQ+CTxD9CK7BVhawufwk0Sncp7n+P+7z5Og35XMSR+fISIiMRpzEBGRGIWDiIjEKBxERCRG4SAiIjEKBxERiVE4iIhIjMJBRERi/j+sMUvszgWRbgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "y_pred_true = lr_true.intercept_ + lr_true.coef_[0]*X_population['age'] + lr_true.coef_[1]*X_population['kms']\n",
    "plt.scatter(X_population['kms'], y_population)\n",
    "plt.plot(X_population['kms'], y_pred_true, 'r')\n",
    "plt.grid()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "090bd09e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAD4CAYAAAAHHSreAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAmYklEQVR4nO3de3gV5bn38e9tEjGKgoBSTJCzsRxUIB5RN6AWtFYjyBYrFd0qRSniASyRuu3LBsGiVSkVRaGAukHEGE9VUAG1loNgqhGVAoKawBa1BoVGRHzeP56JrGQFcyDJTLJ+n+taVyb3zKx1Z0HWnZnnZM45REREYh0QdgIiIhI9Kg4iIhJHxUFEROKoOIiISBwVBxERiZMcdgLV1aJFC9e2bduw0/jBzp07OeSQQ8JOY5+inh9EP8eo5wfRzzHq+UHDz3HNmjWfO+eOqPBA51y9fPTs2dNFydKlS8NO4UdFPT/nop9j1PNzLvo5Rj0/5xp+jsBqV4nPWN1WEhGROCoOIiISR8VBRETiqDiIiEgcFQcREYlTb7uyVkduXiFTFq1jS1ExRzVNZUy/DLK6p4WdlohI5CRMccjNKyQ7J5/i3XsAKCwqJjsnH0AFQkSkjIS5rTRl0bofCkOJ4t17mLJoXUgZiYhEV8IUhy1FxVWKi4gksoQpDkc1Ta1SXEQkkSVMcRjTL4PUlKRSsdSUJMb0ywgpIxGR6EqYBumSRmf1VhIRqVjCFAfwBULFQESkYglzW0lERCpPxUFEROKoOIiISBwVBxERiaPiICIicVQcREQkjoqDiIjEUXEQEZE4Kg4iIhJHxUFEROKoOIiISBwVBxERiaPiICIicVQcREQkjoqDiIjEUXEQEZE4Kg4iIhJHxUFEROKoOIiISBwVBxERiaPiICIicVQcREQkjoqDiIjEqbA4mNksM9tmZu+WiY80s3VmttbM/hATzzazDcG+fjHxnmaWH+ybamYWxBuZ2eNBfKWZta3Bn09ERKqhMlcOs4H+sQEz6wNcCBznnOsC3BXEOwODgS7BOfebWVJw2nRgGNApeJQ851XAl865jsA9wJ378fOIiEgNqLA4OOdeA/5VJnwtMNk5tys4ZlsQvxCY75zb5ZzbBGwATjKzVsBhzrnlzjkHzAWyYs6ZE2wvBM4quaoQEZFwJFfzvGOAM8xsIvANMNo59yaQBqyIOa4giO0OtsvGCb5+AuCc+87MtgPNgc/LvqiZDcNffdCyZUuWLVtWzfRr3o4dOyKVT1lRzw+in2PU84Po5xj1/EA5lqhucUgGDgdOAU4EFphZe6C8v/jdj8SpYF/poHMzgBkAmZmZrnfv3lXLuhYtW7aMKOVTVtTzg+jnGPX8IPo5Rj0/UI4lqttbqQDIcd4q4HugRRBvHXNcOrAliKeXEyf2HDNLBpoQfxtLRETqUHWLQy7QF8DMjgEOxN8GegYYHPRAaodveF7lnNsKfG1mpwTtCZcDTwfP9QwwNNi+GFgStEuIiEhIKrytZGbzgN5ACzMrAG4HZgGzgu6t3wJDgw/0tWa2AHgP+A4Y4ZzbEzzVtfieT6nAC8EDYCbwiJltwF8xDK6ZH01ERKqrwuLgnLt0H7uG7OP4icDEcuKrga7lxL8BBlWUh4iI1B2NkBYRkTgqDiIiEqe6XVnrpdy8QqYsWseWomKOaprKmH4ZZHVPq/hEEZEEkzDFITevkOycfIp3+/bxwqJisnPyAVQgRETKSJjbSlMWrfuhMJQo3r2HKYvWhZSRiEh0JUxx2FJUXKW4iEgiS5jicFTT1CrFRUQSWcIUhzH9MkhNSSoVS01JYky/jJAyEhGJroRpkC5pdL7vuXfY9uVOmh7ZTL2VRET2IWGuHMAXiKXNPmTtX67hjS8XkdXi+7BTEhGJpIQqDgCcfDKcfTbcdRe0aweDB8PKlWFnJSISKYlXHDIz4YknYONGuOEGeOEFOOUUOO00WLAAvvsu7AxFREKXeMWhRNu2/uqhoADuuw8+/RQuuQQ6dPDxoqKwM0wYuXmF9Jq8hPzC7fSavITcvMKwUxJJeIlbHEoceihcfz3885+Qm+tvNY0ZA+npMHIkbNgQdoYNWsnI9cJgvEnJyHUVCJFwqTiUSEqCCy+EZctgzRoYOBAefBCOOQYuuACWLgWtQVTjNHJdJJpUHMrTowfMmQMffQTjxsHy5dC37974rl1hZ9hgaOS6SDSpOPyYVq3gf/4HPv4YHnoIvv0WrrgC2rSB8eNh27awM6z3NHJdJJpUHCojNRWuvhrefRcWLfJXELffDkcfvTcu1aKR6yLRpOJQFWbws5/BX/8K773nryL+93+hWzeOGz3ax7/XwLqqyOqexqQB3UgLrhTSmqYyaUA3jVwXCZmKQ3X99KfwwAPwyScwcSKHbN4MP/85dO4M06fDzp1hZ1hvZHVP442xfemW1oQ3xvZVYRCJABWH/dW8Odx6KyvmzYNHH4XGjeG666B1a8jOhkJ1yRSR+kfFoYa4lBS47DJ48014/XXo0wf+8Ac/2O6Xv/RxEZF6QsWhppnB6afDk0/6AXQjR8Jzz8FJJ0GvXrBwoaboEJHIU3GoTe3awR//6KfouOce2LoVBg2Cjh19fPv2sDMUESmXikNdOOwwP8nf+vWQk+O7wN58s5+iY9QoPwmgiEiEqDjUpaQkuOgieO01WL0asrLg/vuhUycff/VVTdEhIpGg4hCWnj3hkUf8FB3Z2b5g9O69N/7tt2FnKCIJTMUhbEcdBRMn+vESDzwAxcVw+eV+io4JE+Dzz8POUEQSkIpDVBx8MPz617B2rV+A6Ljj4Lbb/HiJYcP8iGwRkTqi4hA1BxwA/fv7OZzefRd+9St/m6lLF+jXD158Ue0SIlLrVByirEsXmDHDzwo7YQLk58O55/r4gw/Cv/8ddoYi0kCpONQHRxzh15XYvBnmzoWDDoLhw32X2HHjYMuWsDMUkQZGxaE+OfBAf5tpzRrf7fWMM2DSJN94PWSIj4uI1IAKi4OZzTKzbWYWt2iBmY02M2dmLWJi2Wa2wczWmVm/mHhPM8sP9k01Mwvijczs8SC+0sza1tDP1nCZwZlnwlNP+YF1110HTz8NmZk+npMDe/ZU/DwiIvtQmSuH2UD/skEzaw2cA3wcE+sMDAa6BOfcb2YlK7lMB4YBnYJHyXNeBXzpnOsI3APcWZ0fJGF16AD33een6Lj7bt8+MXCgH1h3773w1VdhZygi9VCFxcE59xrwr3J23QPcAsR2nbkQmO+c2+Wc2wRsAE4ys1bAYc655c45B8wFsmLOmRNsLwTOKrmqkCpo0gRuuslP9vfEE378xI03+ik6brqJg7ZuDTtDEalHkqtzkpldABQ6594u8zmeBqyI+b4giO0OtsvGS875BMA5952ZbQeaA3Gjv8xsGP7qg5YtW7Js2bLqpF8rduzYEZ18WrSACRM49P33SV+4kCOmTuVk5/hs+nQKBg1ie9eu/tZUxETqPSxH1POD6OcY9fxAOf7AOVfhA2gLvBtsHwysBJoE328GWgTbfwaGxJw3ExgInAi8HBM/A3g22F4LpMfs2wg0ryinnj17uihZunRp2Cns2yefuM2//KVzhx/uHDiXmenco486t2tX2JmVEun30EU/P+ein2PU83Ou4ecIrHaV+NyvTm+lDkA74G0z2wykA2+Z2U/wVwStY45NB7YE8fRy4sSeY2bJQBPKv40l1ZWezqZrrvFTdNx/v2+HGDLETyk+aRJ88UXYGYpIxFS5ODjn8p1zRzrn2jrn2uI/3Hs45/4PeAYYHPRAaodveF7lnNsKfG1mpwTtCZcDTwdP+QwwNNi+GFgSVDepaYccAtdeC++/D88/79e7vvVWP0XH8OHwwQdhZygiEVGZrqzzgOVAhpkVmNlV+zrWObcWWAC8B7wIjHDOlfSpvBZ4GN9IvRF4IYjPBJqb2QbgJmBsNX8WqawDDoDzzoOXXoJ33vHLmM6eDT/9qY8vXqwpOkQSXIUN0s65SyvY37bM9xOBieUctxroWk78G2BQRXlILenWDR5+GO64w88Ke//9fg6nLl38AkWXXQapqWFnKSJ1TCOkxTvySPjv//brS8yeDcnJcM01foqO227zS5yKSMJQcZDSGjWCoUMhLw+WLIHTTvPrTbRpszcuIg2eioOUzwz69PHTcqxb59eaePJJ6NHDr1j39NOaokOkAVNxkIp16gR/+pOfomPKFNi0ya9/nZEBU6fC11+HnaGI1DAVB6m8pk1h9GjYuBEef9xPJT5qlO8KO3q0n1JcRBoEFQepuuRk+M//hOXL/aN/fz/JX4cOMGgQ/P3v6gorUs+pOMj+OeUUmD8fPvzQXz28/DL06uXj8+bB7t1hZygi1aDiIDXj6KPhzjv9FB3TpsGXX/rBde3b+/i/NCOKSH2i4iA1q3FjGDHCT8Xx7LNwzDEwdqxvl7juOt/zSUQiT8VBascBB8D558Mrr8A//uHbKGbOhGOP9fGXX1a7hEiEqThI7Tv+ePjLX/wqdbffDqtWwTnn+PisWfDNN2FnKCJlqDhI3WnZEn7/e18kZs3ysauu8u0Vt99OitolRCJDxUHq3kEHwZVXwttvw8sv838Zx8H48ZxyySU8n9mfJY+/FHaGIglPxUHCY0Zus2Pp03s0fa55kLW9z6HPO8voO/hnfHZiL9+g/f33YWcpkpBUHCRUUxato3j3HjY1S+PVK37NqdfOZlLvK/h+/Xq44AI/Rce0abBjR9ipiiQUFQcJ1Zai4lLfb089lAdPvpjTr3nID6Jr1gxGjvRdYW+5xbdXiEitU3GQUB3VtPyFhI5sfigMHgwrVvjpOM45B+6+2w+qu+QSHxeRWqPiIKEa0y+D1JSkUrHUlCTG9Mvw35jBqafCggV+io4bboAXX/Sxkvh339V94iINnIqDhCqrexqTBnQjLbiCSGuayqQB3cjqnhZ/cJs2cNddfurwqVPhs8/8VUT79n4q8aKiuk1epAFTcZDQZXVP442xfemW1oQ3xvYtvzDEOvRQ3w6xbh3k5vrZYG+5BdLT4Te/gfXr6yRvkYZMxUHqr6QkuPBCWLoU3noLBg6EGTN8D6cLLvBxTdEhUi0qDtIwdO8Oc+bARx/B737n15no29fHZ8+GXbvCzlCkXlFxkIalVSsYP953eX3oId9YfeWVvr1i/HjYti3sDOuF3LxCek1eQruxz9Nr8hJy8wrDTknqmIqDNEypqXD11ZCfD4sXQ48eftK/o4/28znl54edYWTl5hWSnZNPYVExDigsKiY7J18FIsGoOEjDZubHSPz1r/Dee/4qYt48OO44OPtseP55TdFRRsmo9VjFu/cwZZHW4kgkKg6SOH76U5g+3a9Wd8cd8P77fm2Jzp19fOfOsDOMhLKj1iuKS8Ok4iCJp3lzyM6GzZvhscd819jrrvNTdIwd68dRJLB9jVrfV1waJhUHSVwpKX6d61Wr4PXXfe+mKVOgbVu49FIfT0AVjlqXhKDiIGIGp58OCxfChg1w/fW+LeLkk6FXLx9PoCk6YketGxWMWpcGKznsBEQipV07+OMf/Yp1f/kL3HcfDBoEbdqQft55ftxEkyZhZ1nrsrqnqRgkOF05iJTnsMNg1Cg/FUdODrRpQ8fp0/0UHaNGwcaNYWcoUqtUHER+TFISXHQRvPoqqx94ALKy4P77oVMnv/3qq5qiQxokFQeRStqRkQGPPOKn6MjO9o3YvXtDz54wdy58+23YKYrUmAqLg5nNMrNtZvZuTGyKmX1gZu+Y2VNm1jRmX7aZbTCzdWbWLybe08zyg31TzcyCeCMzezyIrzSztjX7I4rUsKOOgokT/XiJBx+Eb76BoUP9FB0TJsDnn4edoch+q8yVw2ygf5nYS0BX59xxwD+BbAAz6wwMBroE59xvZiV94qYDw4BOwaPkOa8CvnTOdQTuAe6s7g8jUqcOPhiGDYO1a/0CRMcfD7fd5sdLXHONj4vUUxUWB+fca8C/ysQWO+dK+vatANKD7QuB+c65Xc65TcAG4CQzawUc5pxb7pxzwFwgK+acOcH2QuCskqsKkXrBDPr18wVi7Vr41a/g0Ueha9e9cU3RIfWMuUo0pgW3ep5zznUtZ9+zwOPOuUfNbBqwwjn3aLBvJvACsBmY7Jw7O4ifAfzWOXd+cLuqv3OuINi3ETjZORd3bW5mw/BXH7Rs2bLn/Pnzq/Ej144dO3bQuHHjsNPYp6jnB9HPsSr5pWzfTqtnnyUtN5dGX3zBzqOPpmDgQD792c/4/qCDIpFjGKKeHzT8HPv06bPGOZdZ4YHOuQofQFvg3XLi44Cn2Ftk/gwMidk/ExgInAi8HBM/A3g22F4LpMfs2wg0ryinnj17uihZunRp2Cn8qKjn51z0c6xWfrt2OffII8716OEcONesmXPZ2c4VFNR4fs410PewjjX0HIHVrhKf+9XurWRmQ4HzgcuCFwQoAFrHHJYObAni6eXES51jZslAE8rcxhKptw48EIYMgdWrfbfXM8+EyZP9FB1DhsCaNWFnKFKuahUHM+sP/Ba4wDn375hdzwCDgx5I7fANz6ucc1uBr83slKA94XLg6ZhzhgbbFwNLYoqNSMNg5gvDU0/5KTpGjICnn4bMTDjjDD/Qbs+eip9HpI5UpivrPGA5kGFmBWZ2FTANOBR4ycz+YWYPADjn1gILgPeAF4ERzrmS//HXAg/jG6k34tsiwN96am5mG4CbgLE19cOJRFL79nDvvX7217vv9l1iBw70A+vuvRe++irsDEUq1VvpUudcK+dcinMu3Tk30znX0TnX2jl3QvAYHnP8ROdcB+dchnPuhZj4audc12Dfb0quDpxz3zjnBgXPeZJz7sPa+VFFIqZJE7jpJn8lsXChHz9x441+io4bb4RNm8LOUBKYRkiLhC052V85/O1vfprwX/wCpk2Djh1hwAA/Elt3WqWOqTiIRMmJJ/oFiDZtgltugWXLfFtFSVxTdEgdUXEQiaL0dJg0ybdHTJ8OO3b43k3t2vklTr/4IuwMpYFTcRCJskMOgeHD4b33/AJEXbrAuHF+io7hw/062CK1QMVBpD444AA47zxYvBjy8/3yprNnQ+fOcO65Pq52CalBKg4i9U3XrvDww/DxxzB+POTl+Tmcunal1XPPQXFx2BlKA6DiIFJfHXmknwX2o4/8VURKChl33w1HH+3jW7eGnaHUYyoOIvVdo0Z+PYm8PP5xzz1w2ml+vYk2beDyy/2VhUgVqTiINBRmFJ1wgp+WY906+PWv/bQcPXr4FetyczVFh1SaioNIQ9SpE/zpT36KjilT/LiJiy6CjAyYOhW+/jrsDCXiVBxEGrKmTWH0aNi4ERYs8O0Uo0b5cRQ33wybN4edoUSUioNIIkhOhkGD4O9/hxUrfPfX++6DDh32xtUVVmKoOIgkmpNPhvnz/a2m0aPh5ZehVy8fnzcPdu8OO0OJABUHkUTVujXceaefomPaNCgq8oPr2rXzCxL9S2tuJTIVB5FE17ixX3zogw/g2Wd9o3V2ti8e113nez5JwlFxEBHvgAPg/PPhlVfg7bfhkktg5kw49lj4+c/97Se1SyQMFQcRiXfccTBrlp+i4/bb4c034Zxz4Pjjffybb8LOUGqZioOI7FvLlvD73/siMWuWj111lZ+i4/bb4dNPQ01Pao+Kg4hU7KCD4Mor/e2mV17xPZvGj/dF4oorfFwaFBUHEak8M+jb1zdcr1sHV18NTzwBJ5ywN/7992FnKTVAxUFEqueYY+DPf/ZdYe+8E9avhwsu8L2dpk3zq9dJvaXiICL7p1kzv971hx/6QXTNmsHIkb4r7C23+PYKqXdUHEQkTm5eIb0mL6Hd2OfpNXkJuXmFFZ+UkgKDB8PKlX46jnPOgbvvhvbtfbfYFStqP3GpMSoOIlJKbl4h2Tn5FBYV44DComKyc/IrVyBKnHqqn+jvww/hxhth0SI49VS6jxgBjz8O331Xa/lLzVBxEJFSpixaR/Hu0us+FO/ew5RF1Rgp3aaNnzL8k09g6lQOLCryVxft2/t4UVGN5Cw1T8VBRErZUlT+GtT7ilfKoYfCyJGsnDvXL0bUoYNvj0hPh9/8xjdmS6SoOIhIKUc1Ta1SvEqSknyPpqVL/fKlF18MM2b4Hk4lcU3REQkqDiJSyph+GaSmJJWKpaYkMaZfRs2+0AknwOzZvjfT734Hy5f7sRLdu/v4rl01+3pSJSoOIlJKVvc0Jg3oRlrTVAxIa5rKpAHdyOqeVjsv+JOf+NHWH38MDz/sG6uvvNK3V4wfD9u21c7ryo9KDjsBEYmerO5ptVcM9iU11c/b9F//5WeAvfdeP3/THXfAZZfBDTdAt251m1MC05WDiESLmR8j8fzz8P77/ipi3jw/U+zZZ/u4puiodSoOIhJdxx4L06f7rrCTJvkFic4/Hzp39vGdO8POsMGqsDiY2Swz22Zm78bEmpnZS2a2Pvh6eMy+bDPbYGbrzKxfTLynmeUH+6aamQXxRmb2eBBfaWZta/hnFJH6rnlzGDvWr3v92GO+a+x11/kpOsaOhYKCsDOsEyUj1/MLt1d+5Ho1VebKYTbQv0xsLPCKc64T8ErwPWbWGRgMdAnOud/MSro9TAeGAZ2CR8lzXgV86ZzrCNwD3FndH0ZEGriUFL/O9apV8Le/+d5NU6ZA27Zw6aU+3kDFjlyHao5cr4IKi4Nz7jWg7ErjFwJzgu05QFZMfL5zbpdzbhOwATjJzFoBhznnljvnHDC3zDklz7UQOKvkqkJEpFxm0KsXLFwIGzbAqFHw17/6dSZK4g1sio4aHbleCeYqMeAkuNXznHOua/B9kXOuacz+L51zh5vZNGCFc+7RID4TeAHYDEx2zp0dxM8AfuucOz+4XdXfOVcQ7NsInOyc+7ycPIbhrz5o2bJlz/nz51f7B69pO3bsoHHjxmGnsU9Rzw+in2PU84Po51ib+SXt3MlPXniB9JwcUrdu5ZuWLSkYMICt553Hniq8ZlTfw/zC7T9st0yFT2MGrHdLa1Lp5+nTp88a51xmRcfVdFfW8v7idz8S/7Fz4oPOzQBmAGRmZrrevXtXI8XasWzZMqKUT1lRzw+in2PU84Po51jr+f3853DfffDssxx0zz10nD6djo884rvHXn+9n7Yj7ByradzkJT/cUrq523fcne8/vtOapjLyst41/nrV7a30aXCriOBrySiVAqB1zHHpwJYgnl5OvNQ5ZpYMNCH+NpaISOUkJUFWFrz6KqxZAxdd5Hs2deq0N14Pp+ios5HrgeoWh2eAocH2UODpmPjgoAdSO3zD8yrn3FbgazM7JWhPuLzMOSXPdTGwxFXmXpeISEV69IC5c2HzZrj1Vt+I3bs39Ozp499+G3aGlRY7ch1qf+R6ZbqyzgOWAxlmVmBmVwGTgXPMbD1wTvA9zrm1wALgPeBFYIRzrqQF5VrgYXwj9UZ8WwTATKC5mW0AbiLo+SQiUmOOOgomTPBTdMyY4edtGjrUT9ExYQJ8HtfEGUlZ3dN4Y2xfuqU14Y2xfWt1FHuFbQ7OuUv3seusfRw/EZhYTnw10LWc+DfAoIryEBHZbwcfDNdcA1dfDYsXwz33wG23wcSJMGSIn6JDAI2QFpFEZAb9+sGLL8LatXD55fDoo9C1K8eNGePjCX53W8VBRBJb587w4IN+io4JEzhk0yY491zo0sXH//3vsDMMhYqDiAhAixYwbhwr5s2DRx7xs8QOH+6n6Lj1ViisvakqokjFQUQkhktJ8e0Pq1fDa6/Bf/wHTJ7sp+gYMsR3j00AKg4iIuUxgzPOgJwcP0XHiBF+/evMTDjzTB/fs6fi56mnVBxERCrSvr1ffKigAP74R98+MXCgH1h3773w1VdhZ1jjVBxERCqrSRO48UZYvx6efBLS0vz36en+66ZNYWdYY1QcRESqKjkZBgyA11+HN9+EX/wCpk2Djh39FcXrr9f7rrAqDiIi+yMz0y9AtHkz/Pa3sGyZb5M48UQfr0dTdMRScRARqQlpaXDHHb49Yvp02LHD925q187Hv/gi7AyrRMVBRKQmHXywHx/x3nt+AaIuXWDcOD9eYvhweP/9sDOsFBUHEZHacMABfqT14sWQnw+XXQazZ/sR2SXxCLdLqDiIiNS2rl3hoYf8Lafx4yEvz8/t1K0bPPwwFBdX/Bx1TMVBRKSuHHGEnwX2o49gzhxISfGzxB59tI9v3Rp2hj9QcRARqWuNGvmZYN96C5YuhV69/LThbdr4eF5e2BmqOIiIhMbMr0yXmwv//KdvsM7J8SvY9e7tp+sIaYoOFQcRkSjo2BGmTvVTdNx1lx9tnZUFGRk+/vXXdZqOioOISJQ0bQo33wwbN8KCBdCyJYwa5afouPlmP9iuDqg4iIhEUXIyDBoEb7wBK1fCeefBffdBhw60euaZWn95FQcRkag76SSYN8/fahozhu3HH1/rL6niICJSX7RuDZMn8+82bWr9pVQcREQkjoqDiIjEUXEQEZE4Kg4iIhJHxUFEROKoOIiISBwVBxERiaPiICIicVQcREQkjoqDiIjEUXEQEZE4Kg4iIhJnv4qDmd1oZmvN7F0zm2dmB5lZMzN7yczWB18Pjzk+28w2mNk6M+sXE+9pZvnBvqlmZvuTl4hIQ5SbV0ivyUvIL9xOr8lLyM0rrLXXqnZxMLM04Hog0znXFUgCBgNjgVecc52AV4LvMbPOwf4uQH/gfjNLCp5uOjAM6BQ8+lc3LxGRhig3r5DsnHwKi4oBKCwqJjsnv9YKxP7eVkoGUs0sGTgY2AJcCMwJ9s8BsoLtC4H5zrldzrlNwAbgJDNrBRzmnFvunHPA3JhzREQEmLJoHcW7S68nXbx7D1MWrauV1zP/eVzNk81GAROBYmCxc+4yMytyzjWNOeZL59zhZjYNWOGcezSIzwReADYDk51zZwfxM4DfOufOL+f1huGvMGjZsmXP+fPnVzv3mrZjxw4aN24cdhr7FPX8IPo5Rj0/iH6OUc8PoptjfuH2H7ZbpsKnxXv3dUtrUunn6dOnzxrnXGZFxyVXLb29graEC4F2QBHwhJkN+bFTyom5H4nHB52bAcwAyMzMdL17965CxrVr2bJlRCmfsqKeH0Q/x6jnB9HPMer5QXRzHDd5yQ+3lG7u9h135/uP77SmqYy8rHeNv97+3FY6G9jknPvMObcbyAFOAz4NbhURfN0WHF8AtI45Px1/G6og2C4bFxGRwJh+GaSmJJWKpaYkMaZfRq283v4Uh4+BU8zs4KB30VnA+8AzwNDgmKHA08H2M8BgM2tkZu3wDc+rnHNbga/N7JTgeS6POUdERICs7mlMGtCNtKapgL9imDSgG1nd02rl9ap9W8k5t9LMFgJvAd8BefhbPo2BBWZ2Fb6ADAqOX2tmC4D3guNHOOdKWleuBWYDqfh2iBeqm5eISEOV1T2NrO5pLFu2rFZuJcWqdnEAcM7dDtxeJrwLfxVR3vET8Q3YZeOrga77k4uIiNQcjZAWEZE4Kg4iIhJHxUFEROKoOIiISJz9GiEdJjP7DPgo7DxitAA+DzuJHxH1/CD6OUY9P4h+jlHPDxp+jm2cc0dUdFC9LQ5RY2arKzMkPSxRzw+in2PU84Po5xj1/EA5ltBtJRERiaPiICIicVQcas6MsBOoQNTzg+jnGPX8IPo5Rj0/UI6A2hxERKQcunIQEZE4Kg4iIhLPOadH8ACaAguBD/DTj58KNANeAtYHXw+POT4bv9zpOqBfTLwnkB/sm8re23eNgMeD+EqgbTVyvBFYC7wLzAMOCjtHYBZ+3Y53Y2J1khN+Wvj1wWNoFfKbEvw7vwM8BTQNK7995RizbzR+AawWUXoPg/jIIIe1wB+i9h4CJwArgH8Aq4GTQnwPWwNL8Z8ta4FRUftdKZVvVT+cGvIDv+b11cH2gfhi8QdgbBAbC9wZbHcG3g7+MdoBG4GkYN8qfGEx/PTj5wbx64AHgu3BwONVzC8N2ASkBt8vAK4IO0fgTKAHpX8paz2n4Jfqw+Dr4cH24ZXM72dAcrB9Z5j57SvHIN4aWIQf8NkiYu9hH+BloFHw/ZFRew+BxTGvcR6wLMT3sBXQI9g+FPhnkEdkfldK5VsbH7L18QEchv/gtTLxdUCrmH/cdcF2NpAdc9yi4B+rFfBBTPxS4MHYY4LtZPwIR6tCjmnAJ8E/cDLwHP5DLvQcgbaU/qWs9Zxijwn2PQhcWpn8yuy7CHgszPz2lSP+SvZ4/FrrLcLMsZx/4wXA2eUcF5n3MHjeS2Je73/DzjHmuKeBc4jY70rJQ20Oe7UHPgP+YmZ5ZvawmR0CtHR+tTqCr0cGx5d8UJcoCGJpwXbZeKlznHPfAduB5pVN0DlXCNyFX0RpK7DdObc4SjnGqIuc9vVcVfVf7F1gKjL5mdkFQKFz7u0yu6KS4zHAGWa20sxeNbMTI5YfwA3AFDP7BP+7kx2FHM2sLdAdf+snkr8rKg57JeMvSac757oDO/GXePti5cTcj8R/7JxKMbPDgQvxl5hHAYeY2ZAo5VgJNZnTfudqZuPwKxM+FqX8zOxgYBzw3+XtjkKO+N+Zw4FTgDH4FSAtQvmBX2XyRudca3x73cz9eL0aydHMGgNPAjc4577a13Fh5ggqDrEKgALn3Mrg+4X4YvGpmbUCCL5uizm+dcz56cCWIJ5eTrzUOWaWDDQB/lWFHM8GNjnnPnPO7QZygNMilmOJushpX89VKWY2FDgfuMwF19oRyq8D/o+At81sc3DuW2b2kwjlWADkOG8V8D1+Qrio5Ae+ETYn2H4COKns69VljmaWgi8MjznnSvKK5u9KRffFEukBvA5kBNu/x/domULpxqI/BNtdKN1Y9CF7G4vexP81VdJYdF4QH0HpxqIFVczvZHwvh4OD556D7y0Seo7E3+ut9ZzwbS+b8H+9Hh5sN6tkfv3x65kfUea4UPIrL8cy+zazt80hKu/hcGB8sH0M/raFRek9xPcM6h1snwWsCes9DJ5vLnBvmXikfld+yKs6H6IN9YHv9rYa370xN3gTmwOv4Lt/vRL7huIv+zfiG5TOjYln4ruabgSmsbeb2UH4v1424HsbtK9Gjv8P3wXzXeCR4D9OqDniu9RuBXbj/0K5qq5ywrcXbAgeV1Yhvw34D7N/BI8HwspvXzmW2b+Z0l1Zo/AeHgg8GrzeW0DfqL2HwOnAGvyH7EqgZ4jv4en4WznvsPf/3XlE6Hcl9qHpM0REJI7aHEREJI6Kg4iIxFFxEBGROCoOIiISR8VBRETiqDiIiEgcFQcREYnz/wFqnyGJ9Z0/FAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "y_pred = lr_sample.intercept_ + lr_sample.coef_[0]*X_sample['kms']\n",
    "plt.scatter(X_population['kms'], y_population)\n",
    "plt.plot(X_sample['kms'], y_pred, 'r')\n",
    "plt.grid()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6461c2c7",
   "metadata": {},
   "source": [
    "The estimator is **biased**. Intuitively, if the true model is built on Age and Kms but we only consider Kms, obviously the estimation is biased since it uses **part** of the rules.\n",
    "\n",
    "Note: a biased estimator can have the effect of either underestimate the relationship **or** overestimate the relationship.\n",
    "\n",
    "To detect omitted variables and if all variables are available, try out several combinations of independent variables. If the coefficients change, there is an omitted variable:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "ea5c0c8c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-0.04294117647058824\n",
      "-0.010000000000000002\n",
      "-0.010000000000000002\n"
     ]
    }
   ],
   "source": [
    "print(LinearRegression().fit(np.array(data_sample['kms']).reshape(-1,1), y_sample).coef_[0])\n",
    "print(LinearRegression().fit(data_sample[['kms','age']], y_sample).coef_[0])\n",
    "print(LinearRegression().fit(data_sample[['kms','age','rdm']], y_sample).coef_[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6cba2295",
   "metadata": {},
   "source": [
    "We can see that when removing/adding Age, the coefficients strongly change => Age should be included in the model.\n",
    "\n",
    "However when adding the RDM it doesn't change => Rdm shouldn't be included in the model."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef1aeed4",
   "metadata": {},
   "source": [
    "##### Assumption 3: the errors are uncorrelated"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac976246",
   "metadata": {},
   "source": [
    "When errors are correlated, the variance is not minimal anymore.\n",
    "\n",
    "$$\\begin{align*}\n",
    "  Cov(\\widehat{\\theta}) &= \\mathbb{V}[(X^TX)^{-1}X^TY] \\\\\n",
    "            &= \\mathbb{V}[(X^TX)^{-1}X^T(X \\theta^* + \\epsilon)] \\\\\n",
    "\t   &= 0 + ((X^TX)^{-1}X^T)^T\\mathbb{V}[\\epsilon] (X^TX)^{-1}X^T \\\\\n",
    "\t   &> (X^TX)^{-1}\\sigma^2 ~~~~\\text{ since $X^TX$ is symmetric}\n",
    "\\end{align*}$$\n",
    "\n",
    "Reminder: $\\mathbb{V}[\\epsilon] = \\mathbb{E}[\\epsilon^T \\epsilon]$ when $\\mathbb{E}[\\epsilon]=0$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2519e788",
   "metadata": {},
   "source": [
    "### Multivariate example "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "87c53954",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('HPRICE3.raw', delim_whitespace=True, header=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "d847c1ae",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(321, 19)"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "56271c66",
   "metadata": {},
   "outputs": [],
   "source": [
    "price = df[7]/100\n",
    "age = df[1]\n",
    "nbh = df[3]\n",
    "inst = df[5]\n",
    "rooms = df[8]\n",
    "area = df[9]\n",
    "land = df[10]\n",
    "baths = df[11]\n",
    "dist = df[12]\n",
    "y81 = df[15]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "4a617f2f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-211.14276727814251\n",
      "[-2.28878493e+00 -1.98891522e+01 -5.40056344e-03  4.13147513e+01\n",
      "  2.17429764e-01  1.21874610e-03  1.30025473e+02  5.05836600e-03\n",
      "  3.58654274e+02]\n",
      "[-2.11142767e+02 -2.28878493e+00 -1.98891522e+01 -5.40056344e-03\n",
      "  4.13147513e+01  2.17429764e-01  1.21874610e-03  1.30025473e+02\n",
      "  5.05836600e-03  3.58654274e+02]\n"
     ]
    }
   ],
   "source": [
    "X_home = np.column_stack((age,nbh,inst,rooms,area,land,baths,dist,y81))\n",
    "y_home = price\n",
    "# Scikit learn\n",
    "reg_home = LinearRegression().fit(X_home, y_home)\n",
    "print(reg_home.intercept_)\n",
    "print(reg_home.coef_)\n",
    "\n",
    "# Manual\n",
    "const = np.ones(np.shape(price))\n",
    "X_home_const = np.column_stack((const,age,nbh,inst,rooms,area,land,baths,dist,y81))\n",
    "X_gram = np.linalg.inv(np.dot(X_home_const.T, X_home_const))\n",
    "theta = np.dot(X_gram, np.dot(X_home_const.T, y_home)) \n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "226f9ee0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                            OLS Regression Results                            \n",
      "==============================================================================\n",
      "Dep. Variable:                      7   R-squared:                       0.715\n",
      "Model:                            OLS   Adj. R-squared:                  0.707\n",
      "Method:                 Least Squares   F-statistic:                     86.73\n",
      "Date:                Thu, 01 Sep 2022   Prob (F-statistic):           2.14e-79\n",
      "Time:                        22:44:39   Log-Likelihood:                -2201.6\n",
      "No. Observations:                 321   AIC:                             4423.\n",
      "Df Residuals:                     311   BIC:                             4461.\n",
      "Df Model:                           9                                         \n",
      "Covariance Type:            nonrobust                                         \n",
      "==============================================================================\n",
      "                 coef    std err          t      P>|t|      [0.025      0.975]\n",
      "------------------------------------------------------------------------------\n",
      "const       -211.1428    105.234     -2.006      0.046    -418.204      -4.081\n",
      "x1            -2.2888      0.491     -4.659      0.000      -3.255      -1.322\n",
      "x2           -19.8892      6.470     -3.074      0.002     -32.620      -7.158\n",
      "x3            -0.0054      0.004     -1.331      0.184      -0.013       0.003\n",
      "x4            41.3148     19.323      2.138      0.033       3.294      79.335\n",
      "x5             0.2174      0.028      7.863      0.000       0.163       0.272\n",
      "x6             0.0012      0.000      3.239      0.001       0.000       0.002\n",
      "x7           130.0255     28.530      4.557      0.000      73.889     186.162\n",
      "x8             0.0051      0.004      1.280      0.201      -0.003       0.013\n",
      "x9           358.6543     27.666     12.964      0.000     304.218     413.091\n",
      "==============================================================================\n",
      "Omnibus:                       87.991   Durbin-Watson:                   1.596\n",
      "Prob(Omnibus):                  0.000   Jarque-Bera (JB):              581.142\n",
      "Skew:                           0.942   Prob(JB):                    6.41e-127\n",
      "Kurtosis:                       9.317   Cond. No.                     4.94e+05\n",
      "==============================================================================\n",
      "\n",
      "Notes:\n",
      "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
      "[2] The condition number is large, 4.94e+05. This might indicate that there are\n",
      "strong multicollinearity or other numerical problems.\n",
      "440.1909469667165\n"
     ]
    }
   ],
   "source": [
    "# Statsmodel\n",
    "model = sm.OLS(y_home, X_home_const)\n",
    "results = model.fit()\n",
    "print(results.summary())\n",
    "\n",
    "s=y81==0 # prix moyen des maisons en 78\n",
    "p0=np.mean(price[s])\n",
    "s=y81==1 # prix moyen des maisons en 81\n",
    "p1=np.mean(price[s])\n",
    "print(p1-p0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba59d9d3",
   "metadata": {},
   "source": [
    "*Residuals*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "6fe69d83",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      166.095540\n",
       "1     -255.639839\n",
       "2       77.930895\n",
       "3      304.295116\n",
       "4       35.210462\n",
       "          ...    \n",
       "316   -210.619723\n",
       "317      9.310803\n",
       "318     -2.880461\n",
       "319     53.206644\n",
       "320    639.064852\n",
       "Name: 7, Length: 321, dtype: float64"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_home-(reg_home.intercept_+np.dot(X_home, reg_home.coef_))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "6011a800",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                            OLS Regression Results                            \n",
      "==============================================================================\n",
      "Dep. Variable:                      y   R-squared:                       0.281\n",
      "Model:                            OLS   Adj. R-squared:                  0.261\n",
      "Method:                 Least Squares   F-statistic:                     13.53\n",
      "Date:                Thu, 01 Sep 2022   Prob (F-statistic):           2.57e-18\n",
      "Time:                        22:44:39   Log-Likelihood:                -4234.7\n",
      "No. Observations:                 321   AIC:                             8489.\n",
      "Df Residuals:                     311   BIC:                             8527.\n",
      "Df Model:                           9                                         \n",
      "Covariance Type:            nonrobust                                         \n",
      "==============================================================================\n",
      "                 coef    std err          t      P>|t|      [0.025      0.975]\n",
      "------------------------------------------------------------------------------\n",
      "const       2.307e+04   5.93e+04      0.389      0.697   -9.35e+04     1.4e+05\n",
      "x1          -615.5935    276.676     -2.225      0.027   -1159.986     -71.201\n",
      "x2         -3018.7279   3643.672     -0.828      0.408   -1.02e+04    4150.638\n",
      "x3            -3.0761      2.285     -1.346      0.179      -7.571       1.419\n",
      "x4         -7635.9441   1.09e+04     -0.702      0.483    -2.9e+04    1.38e+04\n",
      "x5            89.8939     15.573      5.772      0.000      59.252     120.536\n",
      "x6             1.5736      0.212      7.426      0.000       1.157       1.991\n",
      "x7         -2.721e+04   1.61e+04     -1.694      0.091   -5.88e+04    4401.122\n",
      "x8            -2.0748      2.225     -0.932      0.352      -6.454       2.304\n",
      "x9          7940.4165   1.56e+04      0.510      0.611   -2.27e+04    3.86e+04\n",
      "==============================================================================\n",
      "Omnibus:                      366.671   Durbin-Watson:                   1.916\n",
      "Prob(Omnibus):                  0.000   Jarque-Bera (JB):            22235.419\n",
      "Skew:                           4.972   Prob(JB):                         0.00\n",
      "Kurtosis:                      42.542   Cond. No.                     4.94e+05\n",
      "==============================================================================\n",
      "\n",
      "Notes:\n",
      "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
      "[2] The condition number is large, 4.94e+05. This might indicate that there are\n",
      "strong multicollinearity or other numerical problems.\n"
     ]
    }
   ],
   "source": [
    "# Using residuals as dependent variable may be useful to understand what drive the variance\n",
    "u = results.resid\n",
    "u2 = u**2\n",
    "model = sm.OLS(u2, X_home_const)\n",
    "results = model.fit()\n",
    "print(results.summary())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "90bb9f4d",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "db0d9f1b",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a8b74290",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a3bf24a0",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "38878224",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "a0afa596",
   "metadata": {},
   "outputs": [],
   "source": [
    "import statsmodels as sm_model\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cdc4b744",
   "metadata": {},
   "source": [
    "H0: m = a\n",
    "\n",
    "H1: m != a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "64a5b6de",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlYAAAE/CAYAAACEto0QAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAZOklEQVR4nO3dfbRldX3f8fdHRrEICnQuBJiBi3EWFa0aOxKNqRpRwTUEaLuoY8WOSkJN0KjR6iBGiQ3tGFMfGmvNFJGJCoQYXYwSLZMxLJLUp0FBeRAhMMIIMoPUZ0WBb/84e+Lhcu/ch/O795x7eb/WYu2zf/vh9z2/o9wPv73PPqkqJEmSNLiHDbsASZKkpcJgJUmS1IjBSpIkqRGDlSRJUiMGK0mSpEYMVpIkSY0YrKQlLMkHkvxBo3MdnuSHSfbq1i9P8lstzt2d79NJ1rU63yz6/aMkdyX59jz3M5T3J2lhxedYSYtTku3AwcC9wH3AdcCfAxur6v45nOu3qupvZnHM5cBHqurc2fTVHXs28LiqOnW2x7aUZCXwDeCIqtrZ8Lxns0DvL8k4cAvw8Kq6d777k7RnzlhJi9tvVtV+wBHABuBNwAdbd5JkWetzjogjgO+0DFWLzRL+bKWhMFhJS0BVfa+qNgMvAtYleSJAkvOT/FH3enmSTyX5bpK7k/xdkocl+TBwOPDJ7lLfG5OMJ6kkpyW5FfhsX1v/H+JfTvLFJN9LckmSA7u+npNkR3+NSbYneV6S44E3Ay/q+ru62/5Plxa7ut6S5JtJdib58ySP6bbtrmNdklu7y3hnTTU2SR7THb+rO99buvM/D9gCHNrVcf4Ux5+Q5Kpu3P5vkif1bXtTkm8l+UGSG5IcO8P397Ik/5Dk3d15b07ya137bd17XtfXz5okX0ny/W772X0lXtEtv9v194wZjt8/fbZTjZ2k2TNYSUtIVX0R2AH860k2v77bNkbvEuKbe4fUS4Fb6c1+7VtVf9x3zLOBxwPHTdHlfwReARxK75Lk/5hBjZ8B/ivwF11/T55kt5d1//wG8FhgX+B9E/b5deAo4FjgrUkeP0WXfwo8pjvPs7uaX95d9nwhcHtXx8smHpjkqcB5wH8C/jnwZ8DmJHsnOQp4FfC0btbwOGD7DN8fwK8CX+3OewFwEfA04HHAqcD7kuzb7fujru79gTXA7yQ5udv2rG65f9ff52Y4ftN9tpLmwGAlLT23AwdO0v5z4BB69xP9vKr+rqa/yfLsqvpRVf1kiu0frqprqupHwB8A/373ze0Degnwrqq6uap+CJwJrJ0wW/aHVfWTqroauBp4UIDpankRcGZV/aCqtgP/HXjpDOv4beDPquoLVXVfVW0C7gGeTu++tr2Bo5M8vKq2V9U/zuI93lJVH6qq+4C/AFYCb6+qe6rqMuBn9EIWVXV5VX2tqu6vqq8CF9ILRlOZyfhN99lKmgODlbT0HAbcPUn7O4GbgMu6S0/rZ3Cu22ax/ZvAw4HlM6pyzw7tztd/7mX0Ztp26/8W34/pzcpMtBx4xCTnOmyGdRwBvL67XPfdJN+lF4AOraqbgNcCZwM7k1yU5NAZnhfgzr7XPwGoqolt+wIk+dUkf9tdzvwe8Er2PM4zGb/pPltJc2CwkpaQJE+jFxr+fuK2bsbm9VX1WOA3gd9PcuzuzVOccroZrZV9rw+nNyt2F71LV/v01bUXvUuQMz3v7fRCTf+57+WBYWQm7upqmniub83w+NuAc6pq/75/9qmqCwGq6oKq+vXu/AW8ozuu9detLwA2Ayur6jHAB4Dsoa+ZjJ9fCZfmgcFKWgKSPDrJCfTu0/lIVX1tkn1OSPK4JAG+T+9S1n3d5jvp3YszW6cmOTrJPsDbgY91l7a+ATyyu+n64cBb6F022+1OYDzJVP8OuhB4XZIju/uMdt+zNKvHCXS1XAyck2S/JEcAvw98ZIan+N/AK7sZoyR5VPee9ktyVJLnJtkb+Cm9Gab+8dzT+5ut/YC7q+qnSY4B/kPftl3A/Tzw82syfpJmz2AlLW6fTPIDejMrZwHvAl4+xb6rgL8Bfgh8Dnh/VV3ebftvwFu6y11vmEX/HwbOp3dZ7pHA70HvW4rA7wLn0psd+hG9G+d3+8tu+Z0kX57kvOd1576C3jOafgq8ehZ19Xt11//N9GbyLujOP62q2kbvPqv3Af+P3qXUl3Wb96b3iIu76L3/g+h9IQCmf3+z9bvA27vP+q30wuLuGn8MnAP8Q/f5PZ224ydpFnxAqCRJUiPOWEmSJDVisJIkSWrEYCVJktSIwUqSJKkRg5UkSVIjI/Gr5suXL6/x8fFhl9HWDTf0lkcdNdw6JElSU1deeeVdVTU22baRCFbj4+Ns27Zt2GW09Zzn9JaXXz7MKiRJUmNJvjnVNi8FSpIkNWKwkiRJasRgJUmS1IjBSpIkqRGDlSRJUiMGK0mSpEYMVpIkSY0YrCRJkhoxWEmSJDVisJIkSWrEYCVJktTISPxWoKSHnvH1l857H9s3rJn3PiSpnzNWkiRJjRisJEmSGjFYSZIkNWKwkiRJasRgJUmS1IjBSpIkqZFpg1WS85LsTHLNJNvekKSSLO9rOzPJTUluSHJc64IlSZJG1UxmrM4Hjp/YmGQl8Hzg1r62o4G1wBO6Y96fZK8mlUqSJI24aYNVVV0B3D3JpncDbwSqr+0k4KKquqeqbgFuAo5pUagkSdKom9M9VklOBL5VVVdP2HQYcFvf+o6uTZIkacmb9U/aJNkHOAt4wWSbJ2mrSdpIcjpwOsDhhx8+2zIkSZJGzlxmrH4ZOBK4Osl2YAXw5SS/RG+GamXfviuA2yc7SVVtrKrVVbV6bGxsDmVIkiSNllkHq6r6WlUdVFXjVTVOL0w9taq+DWwG1ibZO8mRwCrgi00rliRJGlEzedzChcDngKOS7Ehy2lT7VtW1wMXAdcBngDOq6r5WxUqSJI2yae+xqqoXT7N9fML6OcA5g5UlSZK0+PjkdUmSpEYMVpIkSY3M+nELkpa+8fWXDrsESVqUnLGSJElqxGAlSZLUiMFKkiSpEYOVJElSIwYrSZKkRgxWkiRJjRisJEmSGjFYSZIkNWKwkiRJasRgJUmS1IjBSpIkqRGDlSRJUiMGK0mSpEYMVpIkSY0YrCRJkhoxWEmSJDVisJIkSWrEYCVJktSIwUqSJKkRg5UkSVIjBitJkqRGDFaSJEmNGKwkSZIamTZYJTkvyc4k1/S1vTPJ15N8Ncknkuzft+3MJDcluSHJcfNUtyRJ0siZyYzV+cDxE9q2AE+sqicB3wDOBEhyNLAWeEJ3zPuT7NWsWkmSpBE2bbCqqiuAuye0XVZV93arnwdWdK9PAi6qqnuq6hbgJuCYhvVKkiSNrBb3WL0C+HT3+jDgtr5tO7o2SZKkJW+gYJXkLOBe4KO7mybZraY49vQk25Js27Vr1yBlSJIkjYQ5B6sk64ATgJdU1e7wtANY2bfbCuD2yY6vqo1VtbqqVo+Njc21DEmSpJExp2CV5HjgTcCJVfXjvk2bgbVJ9k5yJLAK+OLgZUqSJI2+ZdPtkORC4DnA8iQ7gLfR+xbg3sCWJACfr6pXVtW1SS4GrqN3ifCMqrpvvoqXJEkaJdMGq6p68STNH9zD/ucA5wxSlCRJ0mI0bbCSpMVqfP2lC9LP9g1rFqQfSaPPn7SRJElqxGAlSZLUiMFKkiSpEYOVJElSIwYrSZKkRgxWkiRJjRisJEmSGjFYSZIkNWKwkiRJasRgJUmS1IjBSpIkqRGDlSRJUiMGK0mSpEYMVpIkSY0YrCRJkhoxWEmSJDVisJIkSWrEYCVJktSIwUqSJKkRg5UkSVIjBitJkqRGlg27AEkzN77+0mGXIEnaA2esJEmSGjFYSZIkNeKlQEka0EJcot2+Yc289yFpcM5YSZIkNTJtsEpyXpKdSa7pazswyZYkN3bLA/q2nZnkpiQ3JDluvgqXJEkaNTOZsTofOH5C23pga1WtArZ26yQ5GlgLPKE75v1J9mpWrSRJ0gibNlhV1RXA3ROaTwI2da83ASf3tV9UVfdU1S3ATcAxbUqVJEkabXO9x+rgqroDoFse1LUfBtzWt9+Ork2SJGnJa33zeiZpq0l3TE5Psi3Jtl27djUuQ5IkaeHNNVjdmeQQgG65s2vfAazs228FcPtkJ6iqjVW1uqpWj42NzbEMSZKk0THXYLUZWNe9Xgdc0te+NsneSY4EVgFfHKxESZKkxWHaB4QmuRB4DrA8yQ7gbcAG4OIkpwG3AqcAVNW1SS4GrgPuBc6oqvvmqXZJkqSRMm2wqqoXT7Hp2Cn2Pwc4Z5CiJEmSFiOfvC5JktSIwUqSJKkRg5UkSVIjBitJkqRGDFaSJEmNGKwkSZIaMVhJkiQ1YrCSJElqxGAlSZLUiMFKkiSpEYOVJElSIwYrSZKkRqb9EWZJMzO+/tJhlyBJGjJnrCRJkhoxWEmSJDVisJIkSWrEYCVJktSIwUqSJKkRg5UkSVIjBitJkqRGDFaSJEmNGKwkSZIaMVhJkiQ1YrCSJElqxGAlSZLUiMFKkiSpkYGCVZLXJbk2yTVJLkzyyCQHJtmS5MZueUCrYiVJkkbZnINVksOA3wNWV9UTgb2AtcB6YGtVrQK2duuSJElL3qCXApcB/yzJMmAf4HbgJGBTt30TcPKAfUiSJC0Kcw5WVfUt4E+AW4E7gO9V1WXAwVV1R7fPHcBBLQqVJEkadYNcCjyA3uzUkcChwKOSnDqL409Psi3Jtl27ds21DEmSpJExyKXA5wG3VNWuqvo58HHg14A7kxwC0C13TnZwVW2sqtVVtXpsbGyAMiRJkkbDIMHqVuDpSfZJEuBY4HpgM7Cu22cdcMlgJUqSJC0Oy+Z6YFV9IcnHgC8D9wJfATYC+wIXJzmNXvg6pUWhkiRJo27OwQqgqt4GvG1C8z30Zq8kSZIeUnzyuiRJUiMGK0mSpEYMVpIkSY0YrCRJkhoxWEmSJDVisJIkSWrEYCVJktSIwUqSJKkRg5UkSVIjBitJkqRGDFaSJEmNDPRbgZKkhTG+/tJ572P7hjXz3oe01DljJUmS1IjBSpIkqRGDlSRJUiMGK0mSpEYMVpIkSY0YrCRJkhoxWEmSJDVisJIkSWrEYCVJktSIwUqSJKkRg5UkSVIjBitJkqRGDFaSJEmNGKwkSZIaMVhJkiQ1MlCwSrJ/ko8l+XqS65M8I8mBSbYkubFbHtCqWEmSpFE26IzVe4HPVNW/AJ4MXA+sB7ZW1Spga7cuSZK05M05WCV5NPAs4IMAVfWzqvoucBKwqdttE3DyYCVKkiQtDoPMWD0W2AV8KMlXkpyb5FHAwVV1B0C3PGiyg5OcnmRbkm27du0aoAxJkqTRMEiwWgY8FfhfVfUrwI+YxWW/qtpYVauravXY2NgAZUiSJI2GQYLVDmBHVX2hW/8YvaB1Z5JDALrlzsFKlCRJWhzmHKyq6tvAbUmO6pqOBa4DNgPrurZ1wCUDVShJkrRILBvw+FcDH03yCOBm4OX0wtrFSU4DbgVOGbAPSZKkRWGgYFVVVwGrJ9l07CDnlSRJWox88rokSVIjBitJkqRGDFaSJEmNDHrzurQojK+/dNglSJIeApyxkiRJasRgJUmS1IjBSpIkqRGDlSRJUiMGK0mSpEYMVpIkSY0YrCRJkhoxWEmSJDVisJIkSWrEYCVJktSIwUqSJKkRg5UkSVIj/gizJAlYuB8r375hzYL0Iw2DM1aSJEmNGKwkSZIaMVhJkiQ1YrCSJElqxGAlSZLUiMFKkiSpEYOVJElSIwYrSZKkRgxWkiRJjQwcrJLsleQrST7VrR+YZEuSG7vlAYOXKUmSNPpazFi9Bri+b309sLWqVgFbu3VJkqQlb6BglWQFsAY4t6/5JGBT93oTcPIgfUiSJC0Wg85YvQd4I3B/X9vBVXUHQLc8aMA+JEmSFoU5B6skJwA7q+rKOR5/epJtSbbt2rVrrmVIkiSNjEFmrJ4JnJhkO3AR8NwkHwHuTHIIQLfcOdnBVbWxqlZX1eqxsbEBypAkSRoNcw5WVXVmVa2oqnFgLfDZqjoV2Ays63ZbB1wycJWSJEmLwHw8x2oD8PwkNwLP79YlSZKWvGUtTlJVlwOXd6+/Axzb4rySJEmLiU9elyRJasRgJUmS1IjBSpIkqRGDlSRJUiMGK0mSpEYMVpIkSY00edyCNFfj6y8ddgmSJDXjjJUkSVIjBitJkqRGDFaSJEmNGKwkSZIaMVhJkiQ1YrCSJElqxMctaEo+CkGSpNlxxkqSJKkRg5UkSVIjBitJkqRGDFaSJEmNGKwkSZIaMVhJkiQ1YrCSJElqxOdYSZIW1EI8I2/7hjXz3oc0GWesJEmSGjFYSZIkNWKwkiRJasRgJUmS1IjBSpIkqZE5B6skK5P8bZLrk1yb5DVd+4FJtiS5sVse0K5cSZKk0TXIjNW9wOur6vHA04EzkhwNrAe2VtUqYGu3LkmStOTNOVhV1R1V9eXu9Q+A64HDgJOATd1um4CTB6xRkiRpUWhyj1WSceBXgC8AB1fVHdALX8BBUxxzepJtSbbt2rWrRRmSJElDNXCwSrIv8FfAa6vq+zM9rqo2VtXqqlo9NjY2aBmSJElDN1CwSvJweqHqo1X18a75ziSHdNsPAXYOVqIkSdLiMMi3AgN8ELi+qt7Vt2kzsK57vQ64ZO7lSZIkLR6D/AjzM4GXAl9LclXX9mZgA3BxktOAW4FTBqpQkiRpkZhzsKqqvwcyxeZj53peSZKkxconr0uSJDVisJIkSWrEYCVJktSIwUqSJKkRg5UkSVIjgzxuQZKkkTS+/tIF6Wf7hjUL0o8WD2esJEmSGjFYSZIkNWKwkiRJasRgJUmS1Ig3ry9CC3VTpiRJmh1nrCRJkhoxWEmSJDVisJIkSWrEYCVJktSIwUqSJKkRg5UkSVIjBitJkqRGDFaSJEmNGKwkSZIaMVhJkiQ18pD6SZuF+CmY7RvWzHsfkiRpNDljJUmS1IjBSpIkqZGH1KXAhbD7cuNFN38HgLULcPlRkjQc3mKiiZyxkiRJamTeZqySHA+8F9gLOLeqNsxXX5IkafQ9FGb45mXGKslewP8EXggcDbw4ydHz0ZckSdKomK8Zq2OAm6rqZoAkFwEnAdfNU3+SJC1JCzHLo3bm6x6rw4Db+tZ3dG2SJElL1nzNWGWStnrADsnpwOnd6g+T3DBPtUy0HLhrvjt5xu4X7zhhvrtaCAsyZkuQ4zY3jtvcOG6z55jNzUiPW96xIN0cMdWG+QpWO4CVfesrgNv7d6iqjcDGeep/Skm2VdXqhe53MXPM5sZxmxvHbW4ct9lzzObGcduz+boU+CVgVZIjkzwCWAtsnqe+JEmSRsK8zFhV1b1JXgX8H3qPWzivqq6dj74kSZJGxbw9x6qq/hr46/k6/wAW/PLjEuCYzY3jNjeO29w4brPnmM2N47YHqarp95IkSdK0/EkbSZKkRpZ8sEqyV5KvJPlUt/6UJJ9PclWSbUmOGXaNo2aSMXtyks8l+VqSTyZ59LBrHEVJtndjdFWSbV3bgUm2JLmxWx4w7DpHyRRjdkqSa5Pcn8RvHk1iinF7Z5KvJ/lqkk8k2X/IZY6cKcbtv3RjdlWSy5IcOuw6R81k49a37Q1JKsnyYdU3apZ8sAJeA1zft/7HwB9W1VOAt3breqCJY3YusL6q/iXwCeA/D6WqxeE3quopfV9FXg9srapVwNZuXQ80ccyuAf4tcMUQa1oMJo7bFuCJVfUk4BvAmcMrbaRNHLd3VtWTur8Jn6L3d0EPNnHcSLISeD5w6/DKGj1LOlglWQGsoRcMditg94zLY5jwfK2HuinG7Ch+8UduC/DvFrquRewkYFP3ehNw8vBKWRyq6vqqWqgHBi8ZVXVZVd3brX6e3vMDNY2q+n7f6qOY8DBr7dG7gTfimD3Akg5WwHvofej397W9FnhnktuAP8H/qpvoPTx4zK4BTuxen8IDH/6qXyjgsiRXdr8sAHBwVd0B0C0PGlp1o2myMdP0phu3VwCfXuCaFoNJxy3JOd3fhJfgjNVkHjRuSU4EvlVVVw+3tNGzZINVkhOAnVV15YRNvwO8rqpWAq8DPrjgxY2oPYzZK4AzklwJ7Af8bMGLWxyeWVVPBV5Ib7yeNeyCFgHHbG6mHLckZwH3Ah8dVnEjbNJxq6qzur8JHwVeNcwCR9Rk43YWhtBJLdlgBTwTODHJduAi4LlJPgKsAz7e7fOXgDev/8KkY1ZVX6+qF1TVvwIuBP5xmEWOqqq6vVvupHcv2jHAnUkOAeiWO4dX4eiZYsw0janGLck64ATgJeWzdB5kBv97uwBvdXiQScbt2cCRwNXd34sVwJeT/NLQihwhSzZYVdWZVbWiqsbp/aTOZ6vqVHr3VD272+25wI1DKnHkTDVmSQ4CSPIw4C3AB4ZY5khK8qgk++1+DbyA3iXUzfTCPN3ykuFUOHr2MGbag6nGLcnxwJuAE6vqx8OscRTtYdxW9e12IvD1YdQ3qqYYty9V1UFVNd79vdgBPLWqvj3EUkfGvD15fYT9NvDeJMuAnwLe1zG9Fyc5o3v9ceBDwyxmRB0MfCIJ9P5/dUFVfSbJl4CLk5xG75szpwyxxlEz1Zj9G+BPgTHg0iRXVdVxQ6xz1Ew1bjcBewNbum2fr6pXDq/MkTPVuP1VkqPo3Vf6TcAxe6BJx224JY02n7wuSZLUyJK9FChJkrTQDFaSJEmNGKwkSZIaMVhJkiQ1YrCSJElqxGAlSZLUiMFKkiSpEYOVJElSI/8fCNnOXvJQFjMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Generate data\n",
    "rnd = np.random.RandomState(3)\n",
    "n_size = 1000\n",
    "sample = [rnd.randint(100) for i in range(n_size)]\n",
    "real_life_value = np.mean(sample)\n",
    "pop_std = 30\n",
    "a = 51\n",
    "\n",
    "distr = [rnd.normal(a, pop_std/np.sqrt(n_size)) for i in range(n_size)]\n",
    "plt.figure(figsize=(10,5))\n",
    "plt.hist(distr, bins=20)\n",
    "plt.axvline(x=real_life_value, color='r')\n",
    "plt.title('Distribution of estimator')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "35918f06",
   "metadata": {},
   "outputs": [],
   "source": [
    "# sm_model.stats.weightstats.ztest(x1=sample, value=a)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "96f3e5a8",
   "metadata": {},
   "source": [
    "- Case when we don't know the population variance?\n",
    "- Case when we have a very small sample?\n",
    "- How to display power of the test?\n",
    "\n",
    "Example: \"most of French people are impolite (I met 5 of them, 3 were douchbags)\""
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}