6_)Regression_multiple_linear_regression.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Nikhila-KS/Unravel_ML/blob/main/6_)multiple_linear_regression.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CazISR8X_HUG"
      },
      "source": [
        "# Multiple Linear Regression"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pOyqYHTk_Q57"
      },
      "source": [
        "## Importing the libraries"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "import pandas as pd"
      ],
      "metadata": {
        "id": "qHtb8Opjy7pa"
      },
      "execution_count": 58,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vgC61-ah_WIz"
      },
      "source": [
        "## Importing the dataset"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "dataset=pd.read_csv(\"50_Startups.csv\")\n",
        "x= dataset.iloc[:,:-1].values\n",
        "y=dataset.iloc[:,-1].values\n"
      ],
      "metadata": {
        "id": "lWWsFMymzI1D"
      },
      "execution_count": 59,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(x)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iwIJiCVNDTKi",
        "outputId": "c741f915-e6d4-4f5b-f256-2930f0f1e1c9"
      },
      "execution_count": 60,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[[165349.2 136897.8 471784.1 'New York']\n",
            " [162597.7 151377.59 443898.53 'California']\n",
            " [153441.51 101145.55 407934.54 'Florida']\n",
            " [144372.41 118671.85 383199.62 'New York']\n",
            " [142107.34 91391.77 366168.42 'Florida']\n",
            " [131876.9 99814.71 362861.36 'New York']\n",
            " [134615.46 147198.87 127716.82 'California']\n",
            " [130298.13 145530.06 323876.68 'Florida']\n",
            " [120542.52 148718.95 311613.29 'New York']\n",
            " [123334.88 108679.17 304981.62 'California']\n",
            " [101913.08 110594.11 229160.95 'Florida']\n",
            " [100671.96 91790.61 249744.55 'California']\n",
            " [93863.75 127320.38 249839.44 'Florida']\n",
            " [91992.39 135495.07 252664.93 'California']\n",
            " [119943.24 156547.42 256512.92 'Florida']\n",
            " [114523.61 122616.84 261776.23 'New York']\n",
            " [78013.11 121597.55 264346.06 'California']\n",
            " [94657.16 145077.58 282574.31 'New York']\n",
            " [91749.16 114175.79 294919.57 'Florida']\n",
            " [86419.7 153514.11 0.0 'New York']\n",
            " [76253.86 113867.3 298664.47 'California']\n",
            " [78389.47 153773.43 299737.29 'New York']\n",
            " [73994.56 122782.75 303319.26 'Florida']\n",
            " [67532.53 105751.03 304768.73 'Florida']\n",
            " [77044.01 99281.34 140574.81 'New York']\n",
            " [64664.71 139553.16 137962.62 'California']\n",
            " [75328.87 144135.98 134050.07 'Florida']\n",
            " [72107.6 127864.55 353183.81 'New York']\n",
            " [66051.52 182645.56 118148.2 'Florida']\n",
            " [65605.48 153032.06 107138.38 'New York']\n",
            " [61994.48 115641.28 91131.24 'Florida']\n",
            " [61136.38 152701.92 88218.23 'New York']\n",
            " [63408.86 129219.61 46085.25 'California']\n",
            " [55493.95 103057.49 214634.81 'Florida']\n",
            " [46426.07 157693.92 210797.67 'California']\n",
            " [46014.02 85047.44 205517.64 'New York']\n",
            " [28663.76 127056.21 201126.82 'Florida']\n",
            " [44069.95 51283.14 197029.42 'California']\n",
            " [20229.59 65947.93 185265.1 'New York']\n",
            " [38558.51 82982.09 174999.3 'California']\n",
            " [28754.33 118546.05 172795.67 'California']\n",
            " [27892.92 84710.77 164470.71 'Florida']\n",
            " [23640.93 96189.63 148001.11 'California']\n",
            " [15505.73 127382.3 35534.17 'New York']\n",
            " [22177.74 154806.14 28334.72 'California']\n",
            " [1000.23 124153.04 1903.93 'New York']\n",
            " [1315.46 115816.21 297114.46 'Florida']\n",
            " [0.0 135426.92 0.0 'California']\n",
            " [542.05 51743.15 0.0 'New York']\n",
            " [0.0 116983.8 45173.06 'California']]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(y)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YvofpTznDVNy",
        "outputId": "8aef925a-af36-479f-e448-56d351cd90ff"
      },
      "execution_count": 61,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[192261.83 191792.06 191050.39 182901.99 166187.94 156991.12 156122.51\n",
            " 155752.6  152211.77 149759.96 146121.95 144259.4  141585.52 134307.35\n",
            " 132602.65 129917.04 126992.93 125370.37 124266.9  122776.86 118474.03\n",
            " 111313.02 110352.25 108733.99 108552.04 107404.34 105733.54 105008.31\n",
            " 103282.38 101004.64  99937.59  97483.56  97427.84  96778.92  96712.8\n",
            "  96479.51  90708.19  89949.14  81229.06  81005.76  78239.91  77798.83\n",
            "  71498.49  69758.98  65200.33  64926.08  49490.75  42559.73  35673.41\n",
            "  14681.4 ]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VadrvE7s_lS9"
      },
      "source": [
        "## Encoding categorical data"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# two types - different for dependent and independent varianles\n",
        "#check out  data preprocessing tools.ipynb\n",
        "\n",
        "from sklearn.compose import ColumnTransformer\n",
        "from sklearn.preprocessing import OneHotEncoder\n",
        "ct= ColumnTransformer(transformers=[('encoder',OneHotEncoder(),[3])],remainder='passthrough')\n",
        "# In the first argument we actually have to specify three things., First, the kind of transformation which is encoding.\n",
        "# Second, what kind of encoding we want to dowhich is, one hot encoding\n",
        "# And third, the indexes of the columns we want to encode\n",
        "\n",
        "# In the second argument remainder.Here we want to specify in quote the following code name, which is passthrough, and which is a code name\n",
        "# that will say that we indeed want to keep, the columns that won't be applied some transformation, that won't be one hot encoded\n",
        "# Which are of course age and salary.\n",
        "\n",
        "x=np.array(ct.fit_transform(x))\n",
        "\n",
        "#column transformer class actually has a method called fit transform which will do exactly the process of fitting  and transforming at once at the same time.\n",
        "# the fit transfer method actually doesn't return the output as a NumPy array.So here we want to force the output of this fit transfer method to be a NumPy array.\n"
      ],
      "metadata": {
        "id": "R7wUYHvz-xla"
      },
      "execution_count": 62,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(x)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "b6mnASsTE3te",
        "outputId": "f4ba65e0-cf81-4d6e-e064-dbcb87bd5a22"
      },
      "execution_count": 63,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[[0.0 0.0 1.0 165349.2 136897.8 471784.1]\n",
            " [1.0 0.0 0.0 162597.7 151377.59 443898.53]\n",
            " [0.0 1.0 0.0 153441.51 101145.55 407934.54]\n",
            " [0.0 0.0 1.0 144372.41 118671.85 383199.62]\n",
            " [0.0 1.0 0.0 142107.34 91391.77 366168.42]\n",
            " [0.0 0.0 1.0 131876.9 99814.71 362861.36]\n",
            " [1.0 0.0 0.0 134615.46 147198.87 127716.82]\n",
            " [0.0 1.0 0.0 130298.13 145530.06 323876.68]\n",
            " [0.0 0.0 1.0 120542.52 148718.95 311613.29]\n",
            " [1.0 0.0 0.0 123334.88 108679.17 304981.62]\n",
            " [0.0 1.0 0.0 101913.08 110594.11 229160.95]\n",
            " [1.0 0.0 0.0 100671.96 91790.61 249744.55]\n",
            " [0.0 1.0 0.0 93863.75 127320.38 249839.44]\n",
            " [1.0 0.0 0.0 91992.39 135495.07 252664.93]\n",
            " [0.0 1.0 0.0 119943.24 156547.42 256512.92]\n",
            " [0.0 0.0 1.0 114523.61 122616.84 261776.23]\n",
            " [1.0 0.0 0.0 78013.11 121597.55 264346.06]\n",
            " [0.0 0.0 1.0 94657.16 145077.58 282574.31]\n",
            " [0.0 1.0 0.0 91749.16 114175.79 294919.57]\n",
            " [0.0 0.0 1.0 86419.7 153514.11 0.0]\n",
            " [1.0 0.0 0.0 76253.86 113867.3 298664.47]\n",
            " [0.0 0.0 1.0 78389.47 153773.43 299737.29]\n",
            " [0.0 1.0 0.0 73994.56 122782.75 303319.26]\n",
            " [0.0 1.0 0.0 67532.53 105751.03 304768.73]\n",
            " [0.0 0.0 1.0 77044.01 99281.34 140574.81]\n",
            " [1.0 0.0 0.0 64664.71 139553.16 137962.62]\n",
            " [0.0 1.0 0.0 75328.87 144135.98 134050.07]\n",
            " [0.0 0.0 1.0 72107.6 127864.55 353183.81]\n",
            " [0.0 1.0 0.0 66051.52 182645.56 118148.2]\n",
            " [0.0 0.0 1.0 65605.48 153032.06 107138.38]\n",
            " [0.0 1.0 0.0 61994.48 115641.28 91131.24]\n",
            " [0.0 0.0 1.0 61136.38 152701.92 88218.23]\n",
            " [1.0 0.0 0.0 63408.86 129219.61 46085.25]\n",
            " [0.0 1.0 0.0 55493.95 103057.49 214634.81]\n",
            " [1.0 0.0 0.0 46426.07 157693.92 210797.67]\n",
            " [0.0 0.0 1.0 46014.02 85047.44 205517.64]\n",
            " [0.0 1.0 0.0 28663.76 127056.21 201126.82]\n",
            " [1.0 0.0 0.0 44069.95 51283.14 197029.42]\n",
            " [0.0 0.0 1.0 20229.59 65947.93 185265.1]\n",
            " [1.0 0.0 0.0 38558.51 82982.09 174999.3]\n",
            " [1.0 0.0 0.0 28754.33 118546.05 172795.67]\n",
            " [0.0 1.0 0.0 27892.92 84710.77 164470.71]\n",
            " [1.0 0.0 0.0 23640.93 96189.63 148001.11]\n",
            " [0.0 0.0 1.0 15505.73 127382.3 35534.17]\n",
            " [1.0 0.0 0.0 22177.74 154806.14 28334.72]\n",
            " [0.0 0.0 1.0 1000.23 124153.04 1903.93]\n",
            " [0.0 1.0 0.0 1315.46 115816.21 297114.46]\n",
            " [1.0 0.0 0.0 0.0 135426.92 0.0]\n",
            " [0.0 0.0 1.0 542.05 51743.15 0.0]\n",
            " [1.0 0.0 0.0 0.0 116983.8 45173.06]]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**One-hot encoding** is a type of encoding that is used to convert categorical data into a format that can be used by machine learning algorithms. In one-hot encoding, each unique category is represented by a separate binary column. The column will have a value of 1 for the row that belongs to that category and 0 for all other rows.\n",
        "\n",
        "For example, if you have a categorical variable called \"color\" with the values \"red\", \"green\", and \"blue\", then one-hot encoding would create three new columns: \"color_red\", \"color_green\", and \"color_blue\". The \"color_red\" column would have a value of 1 for rows where the \"color\" value is \"red\", 0 for rows where the \"color\" value is \"green\", and 0 for rows where the \"color\" value is \"blue\".\n",
        "\n",
        "One-hot encoding is a common technique that is used in machine learning because it allows machine learning algorithms to work with categorical data. Many machine learning algorithms, such as decision trees and support vector machines, require numeric input data. One-hot encoding converts categorical data into numeric data, which allows these algorithms to be used.\n",
        "\n",
        "One-hot encoding is not without its drawbacks. One drawback is that it can create a large number of new columns, which can make the data set larger and more difficult to work with. Another drawback is that it can introduce sparsity into the data set, which can make some machine learning algorithms less effective."
      ],
      "metadata": {
        "id": "Re0_6IDr-x26"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WemVnqgeA70k"
      },
      "source": [
        "## Splitting the dataset into the Training set and Test set"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "x_train,x_test,y_train,y_test= train_test_split(x,y,test_size=0.2, random_state=0)"
      ],
      "metadata": {
        "id": "cq-FnxVvzlNF"
      },
      "execution_count": 64,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k-McZVsQBINc"
      },
      "source": [
        "## Training the Multiple Linear Regression model on the Training set"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.linear_model import LinearRegression\n",
        "regressor=LinearRegression()\n",
        "regressor.fit(x_train,y_train)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 74
        },
        "id": "7i4YF_znAHj2",
        "outputId": "374798b7-e093-4cb6-c983-18fafe42fe0b"
      },
      "execution_count": 65,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LinearRegression()"
            ],
            "text/html": [
              "<style>#sk-container-id-6 {color: black;background-color: white;}#sk-container-id-6 pre{padding: 0;}#sk-container-id-6 div.sk-toggleable {background-color: white;}#sk-container-id-6 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-6 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-6 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-6 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-6 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-6 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-6 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-6 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-6 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-6 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-6 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-6 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-6 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-6 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-6 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-6 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-6 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-6 div.sk-item {position: relative;z-index: 1;}#sk-container-id-6 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-6 div.sk-item::before, #sk-container-id-6 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-6 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-6 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-6 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-6 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-6 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-6 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-6 div.sk-label-container {text-align: center;}#sk-container-id-6 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-6 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-6\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>LinearRegression()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-6\" type=\"checkbox\" checked><label for=\"sk-estimator-id-6\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LinearRegression</label><div class=\"sk-toggleable__content\"><pre>LinearRegression()</pre></div></div></div></div></div>"
            ]
          },
          "metadata": {},
          "execution_count": 65
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "I would like to answer two important questions.\n",
        "\n",
        "**The first question is, do we have to avoid, you know to do something to avoid the dummy variable trap?**\n",
        "\n",
        "And the answer is no\n",
        "\n",
        "because indeed the Multiple Linear Regression class that we're about to import and which will build the Multiple Linear Regression model itself and actually also train it because remember that a class actually can complete several actions including building it and training it.\n",
        "\n",
        "And while this class will automatically avoid this trap which means that indeed, you know one of the three first columns here because remember Kyril explained\n",
        " that one is redundant, well, we'll automatically be outcast.\n",
        "\n",
        "So you have nothing to worry\n",
        "\n",
        "about regarding the dummy variable trap,you don't have to remove one of the columns here. And that's the beauty of classes.\n",
        "\n",
        "You know, these are advanced implementations that allow you to build a model, a machinery model, in just a few lines of code. And this will take care of the dummy variable trap for you.\n",
        "\n",
        "**Now, second question, do we have to work\n",
        "on the features to select the best ones with, you knowthe techniques that Kyril introduced to you, like, for example, backward elimination.**\n",
        "\n",
        "**Do we have to deploy the backward elimination technique in order to select the features that havethe highest P valuesand that are the most statistically significant?**\n",
        "\n",
        "And the answer is once again, no.\n",
        "\n",
        "Why is that?\n",
        "\n",
        "Well, for the exact same reason as the dummy variable trap.\n",
        "\n",
        "The class that we're about to call\n",
        "\n",
        "to build our Multiple Linear Regression model\n",
        "\n",
        "will automatically identify the best features, you know\n",
        "\n",
        "the features that have the highest P values\n",
        "\n",
        "or that are the most statistically significant\n",
        "\n",
        "to figure out how to predict the dependent variable,\n",
        "\n",
        "you know the profit with the highest accuracy.\n",
        "\n",
        "So once again, you don't have to worry about this."
      ],
      "metadata": {
        "id": "2V5yTXoo-MEj"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<hr>\n",
        "So the good news is actually that the class that we're about to use to build and train this Multiple Linear Regression model is actually the exact same class as for the Simple Linear Regression model, it will just recognize that there are several features and therefore that we are doing multiple linear regression, but the rest will be exactly the same. It will be trained to understand the correlations between all your features - actually, I can open it here - all your features and the profit, which is your dependent variable,\n",
        "\n",
        "and then it will take care of the dummy variable trap and it will also take care of selecting the best features that are the most statistically significant."
      ],
      "metadata": {
        "id": "Q2W38JAy_Nw_"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xNkXL1YQBiBT"
      },
      "source": [
        "## Predicting the Test set results"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "y_pred = regressor.predict(x_test)\n",
        "np.set_printoptions(precision=3)\n",
        "print(np.concatenate((y_pred.reshape(len(y_pred),1),y_test.reshape(len(y_test),1)),1))\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jlohpc9mxw8h",
        "outputId": "d043fe7a-2396-47b7-c2fe-4a2ac2d0f573"
      },
      "execution_count": 66,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[[103015.202 103282.38 ]\n",
            " [132582.278 144259.4  ]\n",
            " [132447.738 146121.95 ]\n",
            " [ 71976.099  77798.83 ]\n",
            " [178537.482 191050.39 ]\n",
            " [116161.242 105008.31 ]\n",
            " [ 67851.692  81229.06 ]\n",
            " [ 98791.734  97483.56 ]\n",
            " [113969.435 110352.25 ]\n",
            " [167921.066 166187.94 ]]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Just keep this for this Backward Elimination implementation, but keep in mind that in general you don't have to remove manually a dummy variable\n",
        "# column because Scikit-Learn takes care of it.And also, please find the whole code implementing this Backward Elimination technique:\n",
        "'''\n",
        "# Multiple Linear Regression\n",
        "\n",
        "# Importing the libraries\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "import pandas as pd\n",
        "\n",
        "# Importing the dataset\n",
        "dataset = pd.read_csv('50_Startups.csv')\n",
        "X = dataset.iloc[:, :-1].values\n",
        "y = dataset.iloc[:, -1].values\n",
        "print(X)\n",
        "\n",
        "# Encoding categorical data\n",
        "from sklearn.compose import ColumnTransformer\n",
        "from sklearn.preprocessing import OneHotEncoder\n",
        "ct = ColumnTransformer(transformers=[('encoder', OneHotEncoder(), [3])], remainder='passthrough')\n",
        "X = np.array(ct.fit_transform(X))\n",
        "print(X)\n",
        "\n",
        "# Avoiding the Dummy Variable Trap\n",
        "X = X[:, 1:]\n",
        "\n",
        "# Splitting the dataset into the Training set and Test set\n",
        "from sklearn.model_selection import train_test_split\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 0)\n",
        "\n",
        "# Training the Multiple Linear Regression model on the Training set\n",
        "from sklearn.linear_model import LinearRegression\n",
        "regressor = LinearRegression()\n",
        "regressor.fit(X_train, y_train)\n",
        "\n",
        "# Predicting the Test set results\n",
        "y_pred = regressor.predict(X_test)\n",
        "np.set_printoptions(precision=2)\n",
        "print(np.concatenate((y_pred.reshape(len(y_pred),1), y_test.reshape(len(y_test),1)),1))\n",
        "\n",
        "# Building the optimal model using Backward Elimination\n",
        "import statsmodels.api as sm\n",
        "X = np.append(arr = np.ones((50, 1)).astype(int), values = X, axis = 1)\n",
        "X_opt = X[:, [0, 1, 2, 3, 4, 5]]\n",
        "X_opt = X_opt.astype(np.float64)\n",
        "regressor_OLS = sm.OLS(endog = y, exog = X_opt).fit()\n",
        "regressor_OLS.summary()X_opt = X[:, [0, 1, 3, 4, 5]]\n",
        "X_opt = X_opt.astype(np.float64)\n",
        "regressor_OLS = sm.OLS(endog = y, exog = X_opt).fit()\n",
        "regressor_OLS.summary()X_opt = X[:, [0, 3, 4, 5]]\n",
        "X_opt = X_opt.astype(np.float64)\n",
        "regressor_OLS = sm.OLS(endog = y, exog = X_opt).fit()\n",
        "regressor_OLS.summary()X_opt = X[:, [0, 3, 5]]\n",
        "X_opt = X_opt.astype(np.float64)\n",
        "regressor_OLS = sm.OLS(endog = y, exog = X_opt).fit()\n",
        "regressor_OLS.summary()X_opt = X[:, [0, 3]]\n",
        "X_opt = X_opt.astype(np.float64)regressor_OLS = sm.OLS(endog = y, exog = X_opt).fit()\n",
        "regressor_OLS.summary()\n",
        "'''"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 162
        },
        "id": "DCpdZ0sf9fDa",
        "outputId": "e77a657a-0971-4d46-8fce-a3321700e307"
      },
      "execution_count": 67,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "\"\\n# Multiple Linear Regression\\n \\n# Importing the libraries\\nimport numpy as np\\nimport matplotlib.pyplot as plt\\nimport pandas as pd\\n \\n# Importing the dataset\\ndataset = pd.read_csv('50_Startups.csv')\\nX = dataset.iloc[:, :-1].values\\ny = dataset.iloc[:, -1].values\\nprint(X)\\n \\n# Encoding categorical data\\nfrom sklearn.compose import ColumnTransformer\\nfrom sklearn.preprocessing import OneHotEncoder\\nct = ColumnTransformer(transformers=[('encoder', OneHotEncoder(), [3])], remainder='passthrough')\\nX = np.array(ct.fit_transform(X))\\nprint(X)\\n \\n# Avoiding the Dummy Variable Trap\\nX = X[:, 1:]\\n \\n# Splitting the dataset into the Training set and Test set\\nfrom sklearn.model_selection import train_test_split\\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 0)\\n \\n# Training the Multiple Linear Regression model on the Training set\\nfrom sklearn.linear_model import LinearRegression\\nregressor = LinearRegression()\\nregressor.fit(X_train, y_train)\\n \\n# Predicting the Test set results\\ny_pred = regressor.predict(X_test)\\nnp.set_printoptions(precision=2)\\nprint(np.concatenate((y_pred.reshape(len(y_pred),1), y_test.reshape(len(y_test),1)),1))\\n \\n# Building the optimal model using Backward Elimination\\nimport statsmodels.api as sm\\nX = np.append(arr = np.ones((50, 1)).astype(int), values = X, axis = 1)\\nX_opt = X[:, [0, 1, 2, 3, 4, 5]]\\nX_opt = X_opt.astype(np.float64)\\nregressor_OLS = sm.OLS(endog = y, exog = X_opt).fit()\\nregressor_OLS.summary()X_opt = X[:, [0, 1, 3, 4, 5]]\\nX_opt = X_opt.astype(np.float64)\\nregressor_OLS = sm.OLS(endog = y, exog = X_opt).fit()\\nregressor_OLS.summary()X_opt = X[:, [0, 3, 4, 5]]\\nX_opt = X_opt.astype(np.float64)\\nregressor_OLS = sm.OLS(endog = y, exog = X_opt).fit()\\nregressor_OLS.summary()X_opt = X[:, [0, 3, 5]]\\nX_opt = X_opt.astype(np.float64)\\nregressor_OLS = sm.OLS(endog = y, exog = X_opt).fit()\\nregressor_OLS.summary()X_opt = X[:, [0, 3]]\\nX_opt = X_opt.astype(np.float64)regressor_OLS = sm.OLS(endog = y, exog = X_opt).fit()\\nregressor_OLS.summary()\\n\""
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 67
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# New Section\n",
        "Question 1: How do I use my multiple linear regression model to make a single prediction, for example, the profit of a startup with R&D Spend = 160000, Administration Spend = 130000, Marketing Spend = 300000 and State = California?\n",
        "\n",
        "Question 2: How do I get the final regression equation y = b0 + b1 x1 + b2 x2 + ... with the final values of the coefficients?"
      ],
      "metadata": {
        "id": "CMcAweDy-vex"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 1.] Making a single prediction (for example the profit of a startup with R&D Spend = 160000, Administration Spend = 130000, Marketing Spend = 300000 and State = 'California')"
      ],
      "metadata": {
        "id": "VPKt3P6K-9xl"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "print(regressor.predict([[1, 0, 0, 160000, 130000, 300000]]))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GwJdXX0A-9UL",
        "outputId": "5b2b47e3-7241-4653-9c21-25a0cabc4e72"
      },
      "execution_count": 68,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[181566.924]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Therefore, our model predicts that the profit of a Californian startup which spent 160000 in R&D, 130000 in Administration and 300000 in Marketing is $ 181566,92.\n",
        "\n",
        "Important note 1: Notice that the values of the features were all input in a double pair of square brackets. That's because the \"predict\" method always expects a 2D array as the format of its inputs. And putting our values into a double pair of square brackets makes the input exactly a 2D array. Simply put:\n",
        "\n",
        "1,0,0,160000,130000,300000→scalars\n",
        "\n",
        "[1,0,0,160000,130000,300000]→1D array\n",
        "\n",
        "[[1,0,0,160000,130000,300000]]→2D array\n",
        "\n",
        "Important note 2: Notice also that the \"California\" state was not input as a string in the last column but as \"1, 0, 0\" in the first three columns. That's because of course the predict method expects the one-hot-encoded values of the state, and as we see in the second row of the matrix of features X, \"California\" was encoded as \"1, 0, 0\". And be careful to include these values in the first three columns, not the last three ones, because the dummy variables are always created in the first columns."
      ],
      "metadata": {
        "id": "hNS-Fsee_E_9"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Getting the final linear regression equation with the values of the coefficients"
      ],
      "metadata": {
        "id": "uAIq_BmT_MV4"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "print(regressor.coef_)\n",
        "print(regressor.intercept_)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0gv_JycP_Zcz",
        "outputId": "67acc9ff-6155-404e-bf3f-21082b3b578f"
      },
      "execution_count": 69,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[ 8.664e+01 -8.726e+02  7.860e+02  7.735e-01  3.288e-02  3.661e-02]\n",
            "42467.52924853278\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Therefore, the equation of our multiple linear regression model is:\n",
        "\n",
        "$$\\textrm{Profit} = 86.6 \\times \\textrm{Dummy State 1} - 873 \\times \\textrm{Dummy State 2} + 786 \\times \\textrm{Dummy State 3} + 0.773 \\times \\textrm{R&D Spend} + 0.0329 \\times \\textrm{Administration} + 0.0366 \\times \\textrm{Marketing Spend} + 42467.53$$\n",
        "\n",
        "**Important Note:** To get these coefficients we called the \"coef_\" and \"intercept_\" attributes from our regressor object. Attributes in Python are different than methods and usually return a simple value or an array of values."
      ],
      "metadata": {
        "id": "QYcyXdf9_i-T"
      }
    }
  ]
}