001_Python_Pandas_Methods/009_Python_Pandas_DataFrame_set_index.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<small><small><i>\n",
    "All the IPython Notebooks in this lecture series by Dr. Milan Parmar are available @ **[GitHub](https://github.com/milaan9/10_Python_Pandas_Module/tree/main/001_Python_Pandas_Methods)**\n",
    "</i></small></small>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Set index in pandas DataFrame\n",
    "\n",
    "In this class, we learn how to set the index of the **[Pandas DataFrame](https://github.com/milaan9/10_Python_Pandas_Module/blob/main/001_Python_Pandas_DataFrame.ipynb)** using existing columns or a list of labels. We cover all the cases of replacing the existing row labels or assign new ones.\n",
    "\n",
    "DataFrame is the tabular structure in the Python pandas library. It represents each row and column by the label. Row label is called an **index**, whereas column label is called column index/header.\n",
    "\n",
    "By default, while creating DataFrame, Python pandas assign a range of numbers (starting at 0) as a row index. Row indexes are used to identify each row. We can set a new row index or replace the existing ones using **`DataFrame.set_index()`** function, which we discuss further in more detail."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true
   },
   "source": [
    "## The `DataFrame.set_index()` function\n",
    "\n",
    "This function is used to re-assign a row label using the existing column of the DataFrame. It can assign one or multiple columns as a row index. Let’s see how to use **`DataFrame.set_index()`** function to set row index or replace existing.\n",
    "\n",
    "**Syntax:**\n",
    "```python\n",
    "DataFrame.set_index(keys, drop=True, append=False, inplace=False, verify_integrity=False)\n",
    "```\n",
    "\n",
    "**Parameters:**\n",
    "\n",
    "1. **`keys`**: It takes a single or list of column labels to set as an index. It also takes a list of new labels as input.\n",
    "2. **`drop`**: It is a flag to specify if columns to be used as the new index should be deleted From DataFrame or not. By default value is **`True`**, i.e., assign the column as an index and **delete** it.\n",
    "3. **`append`**: It is a flag to specify whether to append columns to the existing index. By default, it is **`False`**, i.e., it replaces the current index rather than appending.\n",
    "4. **`inplace`**: It is used to specify whether to return a new DataFrame or update an existing one. It is a boolean flag with default **`False`**.\n",
    "5. **`verify_integrity`**: It is a boolean flag,\n",
    "    * If **`True`**, then it checks the new index for duplicates and throws **`ValueError`**.\n",
    "    * If **`False`**, then it defers the check until necessary.\n",
    "    \n",
    ">**Note:** Setting to **`False`** will improve the performance of this method.\n",
    "\n",
    "**Return:**\n",
    "\n",
    "DataFrame with the changed row labels or None if **`inplace=True`**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set index using a column\n",
    "\n",
    "How to set index in pandas DataFrame?\n",
    "\n",
    "1. **Create pandas DataFrame**\n",
    "    * We can create a DataFrame from a CSV file or **`dict`**.\n",
    "\n",
    "\n",
    "2. **Identify the columns to set as index**\n",
    "    * We can set a specific column or multiple columns as an index in pandas DataFrame. Create a list of column labels to be used to set an index.\n",
    "**`['col_label1', 'col_label2'...]`**\n",
    "\n",
    "\n",
    "3. **Use **`DataFrame.set_index()`** function**\n",
    "    * We need to pass the column or list of column labels as input to the DataFrame.set_index() function to set it as an index of DataFrame. **By default, these new index columns are deleted from the DataFrame**.\n",
    "**`df = df.set_index(['col_label1', 'col_label2'…])`**\n",
    "\n",
    "\n",
    "4. **Set the index in place**\n",
    "    * We can use the parameter **`inplace`** to set the index in the existing DataFrame rather than create a new copy.\n",
    "**`df.set_index(inplace=True)`**\n",
    "\n",
    "**Example:**\n",
    "\n",
    "Let’s see how we can set a specific column as an index in the DataFrame. In the below example, we have default index as a range of numbers replaced with set index using first column **`Name`** of the student DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-17T12:00:33.128743Z",
     "start_time": "2021-06-17T12:00:32.551117Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Before set index: \n",
      "     Name  Age  Marks\n",
      "0    Joe   20  85.10\n",
      "1    Nat   21  77.80\n",
      "2  Harry   19  91.54\n",
      "\n",
      "After set index: \n",
      "        Age  Marks\n",
      "Name             \n",
      "Joe     20  85.10\n",
      "Nat     21  77.80\n",
      "Harry   19  91.54\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "student_dict = {'Name': ['Joe', 'Nat', 'Harry'], 'Age': [20, 21, 19], 'Marks': [85.10, 77.80, 91.54]}\n",
    "\n",
    "# create DataFrame from dict\n",
    "student_df = pd.DataFrame(student_dict)\n",
    "print(\"Before set index: \\n\", student_df)\n",
    "\n",
    "# set index using column\n",
    "student_df = student_df.set_index('Name')\n",
    "print(\"\\nAfter set index: \\n\", student_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set index using a list\n",
    "\n",
    "As we have seen, we can pass column labels of the DataFrame to assign it as an index of the DataFrame. We can also give a list of labels which can be strings or numbers to **`DataFrame.set_index()`** function to set a new index in the DataFrame.\n",
    "\n",
    "First, we create a Python **`Index`** object from a list of labels and then pass it as input to the **`DataFrame.set_index()`** function.\n",
    "\n",
    "**Example:**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-17T12:00:44.674022Z",
     "start_time": "2021-06-17T12:00:44.654495Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Before set index: \n",
      "     Name  Age  Marks\n",
      "0    Joe   20  85.10\n",
      "1    Nat   21  77.80\n",
      "2  Harry   19  91.54\n",
      "\n",
      "After set index: \n",
      "      Name  Age  Marks\n",
      "s1    Joe   20  85.10\n",
      "s2    Nat   21  77.80\n",
      "s3  Harry   19  91.54\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "student_dict = {'Name': ['Joe', 'Nat', 'Harry'], 'Age': [20, 21, 19], 'Marks': [85.10, 77.80, 91.54]}\n",
    "\n",
    "# create DataFrame from dict\n",
    "student_df = pd.DataFrame(student_dict)\n",
    "print(\"Before set index: \\n\", student_df)\n",
    "\n",
    "index = pd.Index(['s1', 's2', 's3'])\n",
    "student_df = student_df.set_index(index)\n",
    "print(\"\\nAfter set index: \\n\", student_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set index using multiple columns\n",
    "\n",
    "Python pandas have DataFrame with multiple columns or rows as an index, and they are also called **multi-index** DataFrame. If we want to set multiple columns as row labels, we can use **`DataFrame.set_index()`** function.\n",
    "\n",
    "**Example:**\n",
    "\n",
    "In the below example, we pass a list of existing column labels **`Name`** and **`Marks`** to set a **multi-level** index in the student DataFrame.\n",
    "\n",
    ">**Note:** It throws **`KeyError`** for unknown column labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-17T12:00:55.218336Z",
     "start_time": "2021-06-17T12:00:55.199781Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Before set index: \n",
      "     Name  Age  Marks\n",
      "0    Joe   20  85.10\n",
      "1    Nat   21  77.80\n",
      "2  Harry   19  91.54\n",
      "\n",
      "After set index: \n",
      "              Age\n",
      "Name  Marks     \n",
      "Joe   85.10   20\n",
      "Nat   77.80   21\n",
      "Harry 91.54   19\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "student_dict = {'Name': ['Joe', 'Nat', 'Harry'], 'Age': [20, 21, 19], 'Marks': [85.10, 77.80, 91.54]}\n",
    "\n",
    "# create DataFrame from dict\n",
    "student_df = pd.DataFrame(student_dict)\n",
    "print(\"Before set index: \\n\", student_df)\n",
    "\n",
    "# set multi-index\n",
    "student_df = student_df.set_index(['Name', 'Marks'])\n",
    "print(\"\\nAfter set index: \\n\", student_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set multi-index using a list and column\n",
    "\n",
    "If there is a case where we want to create a two-level row index of the DataFrame, where one level is the new list of labels and another level is created from the existing column.\n",
    "\n",
    "We can use **`DataFrame.set_index()`** to set the multi-level index of pandas DataFrame using a combination of a new list and the existing column.\n",
    "\n",
    "We need to create a Python **`Index`** object from a list of new labels and pass that **`Index`** object and an existing column label as input to the **`DataFrame.set_index()`** function to create a two-level index.\n",
    "\n",
    "**Example:**\n",
    "\n",
    "Here, we are passing two parameters to the DataFrame.set_index() function. The first parameter is the **Python Index** created using multiple strings of size matches to the length of DataFrame. The second parameter is the existing column label **`Name`** of student DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-17T12:01:03.478492Z",
     "start_time": "2021-06-17T12:01:03.448219Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Before set index: \n",
      "     Name  Age  Marks\n",
      "0    Joe   20  85.10\n",
      "1    Nat   21  77.80\n",
      "2  Harry   19  91.54\n",
      "\n",
      "After set index: \n",
      "           Age  Marks\n",
      "   Name             \n",
      "s1 Joe     20  85.10\n",
      "s2 Nat     21  77.80\n",
      "s3 Harry   19  91.54\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "student_dict = {'Name': ['Joe', 'Nat', 'Harry'], 'Age': [20, 21, 19], 'Marks': [85.10, 77.80, 91.54]}\n",
    "\n",
    "# create DataFrame from dict\n",
    "student_df = pd.DataFrame(student_dict)\n",
    "print(\"Before set index: \\n\", student_df)\n",
    "\n",
    "index = pd.Index(['s1', 's2', 's3'])\n",
    "student_df = student_df.set_index([index, 'Name'])\n",
    "print(\"\\nAfter set index: \\n\", student_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set multi-index using two Python series\n",
    "\n",
    "When we want to replace the existing index with the multiple new series rather than the existing columns, we can create such a multi-index DataFrame by assigning new series using **`DataFrame.set_index()`** function.\n",
    "\n",
    "**Example:**\n",
    "\n",
    "Let’s see how we can pass two Python **`series`** of numbers as a first and second-level index of the DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-17T12:01:08.660074Z",
     "start_time": "2021-06-17T12:01:08.574657Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Before set index: \n",
      "     Name  Age  Marks\n",
      "0    Joe   20  85.10\n",
      "1    Nat   21  77.80\n",
      "2  Harry   19  91.54\n",
      "\n",
      "After set index: \n",
      "       Name  Age  Marks\n",
      "1 1    Joe   20  85.10\n",
      "2 4    Nat   21  77.80\n",
      "3 9  Harry   19  91.54\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "student_dict = {'Name': ['Joe', 'Nat', 'Harry'], 'Age': [20, 21, 19], 'Marks': [85.10, 77.80, 91.54]}\n",
    "\n",
    "# create DataFrame from dict\n",
    "student_df = pd.DataFrame(student_dict)\n",
    "print(\"Before set index: \\n\", student_df)\n",
    "\n",
    "# set multi-index\n",
    "s = pd.Series([1, 2, 3])\n",
    "student_df = student_df.set_index([s, s ** 2])\n",
    "print(\"\\nAfter set index: \\n\", student_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set index using a Python range\n",
    "\n",
    "Suppose we need to set a sequence of numbers as an index of the DataFrame such that it should start at any number. For example, we want to assign a roll number to the student DataFrame beginning from 1.\n",
    "\n",
    "It is not feasible to pass all the numbers as a list to the **`DataFrame.set_index()`** function. In such a case, we can use the **[Python range()](https://github.com/milaan9/04_Python_Functions/blob/main/002_Python_Functions_Built_in/053_Python_range().ipynb)** function.\n",
    "\n",
    "We can create pandas Index using range() function and pass it to the **`DataFrame.set_index()`** function.\n",
    "\n",
    "**Example:**\n",
    "\n",
    "Let’s see how we can use **[Python range()](https://github.com/milaan9/04_Python_Functions/blob/main/002_Python_Functions_Built_in/053_Python_range%28%29.ipynb)** function with **`DataFrame.set_index()`** to assign a sequential index to the DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-17T12:01:15.517414Z",
     "start_time": "2021-06-17T12:01:15.489099Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Before set index: \n",
      "      Name  Age  Marks\n",
      "s1    Joe   20  85.10\n",
      "s2    Nat   21  77.80\n",
      "s3  Harry   19  91.54\n",
      "\n",
      "After set index: \n",
      "     Name  Age  Marks\n",
      "1    Joe   20  85.10\n",
      "2    Nat   21  77.80\n",
      "3  Harry   19  91.54\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "student_dict = {'Name': ['Joe', 'Nat', 'Harry'], 'Age': [20, 21, 19], 'Marks': [85.10, 77.80, 91.54]}\n",
    "\n",
    "# create DataFrame from dict\n",
    "student_df = pd.DataFrame(student_dict, index=['s1', 's2', 's3'])\n",
    "print(\"Before set index: \\n\", student_df)\n",
    "\n",
    "# set index\n",
    "index = pd.Index(range(1, 4, 1))\n",
    "student_df = student_df.set_index(index)\n",
    "print(\"\\nAfter set index: \\n\", student_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set index but keep column\n",
    "\n",
    "By default, **`DataFrame.set_index()`** function takes column name as input which should be used as an index of the DataFrame. After setting the new index, it deletes the column which is used.\n",
    "\n",
    "If we do not want to delete such a column from DataFrame, then we need to use the drop parameter of **`DataFrame.set_index()`**. It is a boolean flag such that,\n",
    "\n",
    "* If **`drop=True`** (default case), it deletes the column and uses it as an index.\n",
    "* If **`drop=False`**, it does not delete the column and uses it as an index.\n",
    "\n",
    "**Example:**\n",
    "\n",
    "In the below example, we use the **`Name`** column as an index of the student DataFrame without deleting it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-17T12:01:23.013423Z",
     "start_time": "2021-06-17T12:01:22.990963Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Before set index: \n",
      "     Name  Age  Marks\n",
      "0    Joe   20  85.10\n",
      "1    Nat   21  77.80\n",
      "2  Harry   19  91.54\n",
      "\n",
      "After set index: \n",
      "         Name  Age  Marks\n",
      "Name                    \n",
      "Joe      Joe   20  85.10\n",
      "Nat      Nat   21  77.80\n",
      "Harry  Harry   19  91.54\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "student_dict = {'Name': ['Joe', 'Nat', 'Harry'], 'Age': [20, 21, 19], 'Marks': [85.10, 77.80, 91.54]}\n",
    "\n",
    "# create DataFrame from dict\n",
    "student_df = pd.DataFrame(student_dict)\n",
    "print(\"Before set index: \\n\", student_df)\n",
    "\n",
    "# set index, keep column\n",
    "student_df = student_df.set_index('Name', drop=False)\n",
    "print(\"\\nAfter set index: \\n\", student_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set index by keeping old index\n",
    "\n",
    "**`DataFrame.set_index()`** is used to set a new index to the DataFrame. It is also used to extend the existing DataFrame, i.e., we can update the index by append to the existing index.\n",
    "\n",
    "We need to use the **`append`** parameter of the **`DataFrame.set_index()`** function to append the new index to the existing one. By default, the value of **`append`** is **`False`**.\n",
    "\n",
    "**Example:**\n",
    "\n",
    "In the below example, each row has an index like s1, s2, and so on, but we update the index by appending values in the **`Name`** column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-17T12:01:29.680824Z",
     "start_time": "2021-06-17T12:01:29.656902Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Before set index: \n",
      "      Name  Age  Marks\n",
      "s1    Joe   20  85.10\n",
      "s2    Nat   21  77.80\n",
      "s3  Harry   19  91.54\n",
      "\n",
      "After set index: \n",
      "           Age  Marks\n",
      "   Name             \n",
      "s1 Joe     20  85.10\n",
      "s2 Nat     21  77.80\n",
      "s3 Harry   19  91.54\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "student_dict = {'Name':['Joe','Nat','Harry'], 'Age':[20,21,19], 'Marks':[85.10, 77.80, 91.54]}\n",
    "\n",
    "# create DataFrame from dict\n",
    "student_df = pd.DataFrame(student_dict, index = ['s1','s2','s3'])\n",
    "print(\"Before set index: \\n\", student_df)\n",
    "\n",
    "# set index by append\n",
    "student_df = student_df.set_index('Name', append=True)\n",
    "print(\"\\nAfter set index: \\n\", student_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set index in place\n",
    "\n",
    "In the above examples, whenever we executed **`DataFrame.set_index()`** operation, pandas created a new copy of DataFrame because the modification is not-in place.\n",
    "\n",
    "Specify **`inplace=True`** to set index in the existing DataFrame rather than creating a copy of it.\n",
    "\n",
    "If **`inplace=True`** then it updates the existing DataFrame and does not return anything.\n",
    "If **`inplace=False`** then it creates a new DataFrame with updated changes and returns it.\n",
    "\n",
    ">**Note:** You don’t need to assign the result back to a variable as we are performing modifications in place.\n",
    "\n",
    "**Example:**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-17T12:01:34.047473Z",
     "start_time": "2021-06-17T12:01:34.028918Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Before set index: \n",
      "     Name  Age  Marks\n",
      "0    Joe   20  85.10\n",
      "1    Nat   21  77.80\n",
      "2  Harry   19  91.54\n",
      "\n",
      "After set index: \n",
      "        Age  Marks\n",
      "Name             \n",
      "Joe     20  85.10\n",
      "Nat     21  77.80\n",
      "Harry   19  91.54\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "student_dict = {'Name': ['Joe', 'Nat', 'Harry'], 'Age': [20, 21, 19], 'Marks': [85.10, 77.80, 91.54]}\n",
    "\n",
    "# create DataFrame from dict\n",
    "student_df = pd.DataFrame(student_dict)\n",
    "print(\"Before set index: \\n\", student_df)\n",
    "\n",
    "# set index in place\n",
    "student_df.set_index('Name', inplace=True)\n",
    "print(\"\\nAfter set index: \\n\", student_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set index using a column with duplicates\n",
    "\n",
    "As we have discussed, we need to pass a column name that needs to be used to set row index in the DataFrame.\n",
    "\n",
    "But, the column may contain duplicate values. By default, **`DataFrame.set_index()`** allows duplicate index. If we want to change this behavior, then we can use the **`verify_integrity`** parameter of **`DataFrame.set_index()`**.\n",
    "\n",
    "* If **`verify_integrity=True`**, then it checks the new index for duplicates and throws **`ValueError`**.\n",
    "* If **`verify_integrity=False`**, then it defers the check until necessary.\n",
    "\n",
    ">**Note:** Setting to **`False`** will improve the performance of this method.\n",
    "\n",
    "**Example:**\n",
    "\n",
    "In the below example, we set **`verify_integrity=True`** and use the **`Name`** column to set an index that contains duplicate values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-17T12:01:40.485869Z",
     "start_time": "2021-06-17T12:01:40.048379Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Before set index: \n",
      "   Name  Age  Marks\n",
      "0  Joe   20  85.10\n",
      "1  Nat   21  77.80\n",
      "2  Joe   19  91.54\n"
     ]
    },
    {
     "ename": "ValueError",
     "evalue": "Index has duplicate keys: Index(['Joe'], dtype='object', name='Name')",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-10-f6b2789c4214>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m      8\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      9\u001b[0m \u001b[1;31m# set index error case\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 10\u001b[1;33m \u001b[0mstudent_df\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mstudent_df\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mset_index\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'Name'\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mverify_integrity\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;32mTrue\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     11\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"\\nAfter set index: \\n\"\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mstudent_df\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mC:\\ProgramData\\Anaconda3\\lib\\site-packages\\pandas\\core\\frame.py\u001b[0m in \u001b[0;36mset_index\u001b[1;34m(self, keys, drop, append, inplace, verify_integrity)\u001b[0m\n\u001b[0;32m   4777\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mverify_integrity\u001b[0m \u001b[1;32mand\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[0mindex\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mis_unique\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   4778\u001b[0m             \u001b[0mduplicates\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mindex\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mindex\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mduplicated\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0munique\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 4779\u001b[1;33m             \u001b[1;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34mf\"Index has duplicate keys: {duplicates}\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   4780\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   4781\u001b[0m         \u001b[1;31m# use set to handle duplicate column names gracefully in case of drop\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mValueError\u001b[0m: Index has duplicate keys: Index(['Joe'], dtype='object', name='Name')"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "student_dict = {'Name':['Joe','Nat','Joe'], 'Age':[20,21,19], 'Marks':[85.10, 77.80, 91.54]}\n",
    "\n",
    "# create DataFrame from dict\n",
    "student_df = pd.DataFrame(student_dict)\n",
    "print(\"Before set index: \\n\", student_df)\n",
    "\n",
    "# set index error case\n",
    "student_df = student_df.set_index('Name', verify_integrity=True)\n",
    "print(\"\\nAfter set index: \\n\", student_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set index by column number\n",
    "\n",
    "If we need to set single or multiple columns as an index of the DataFrame, but we do not know the column labels to pass to **`DataFrame.set_index()`** function. In such a case, we can use the columns parameter of the DataFrame to retrieve the column index position.\n",
    "\n",
    "We need to create a list of columns using column position **`df.columns[[0,1]]`** and pass it to the **`DataFrame.set_index()`** function.\n",
    "\n",
    "**Example:**\n",
    "\n",
    "In the below example, we set column 0 and 2 as an index of the student DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-17T12:01:46.411582Z",
     "start_time": "2021-06-17T12:01:46.385215Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Before set index: \n",
      "     Name  Age  Marks\n",
      "0    Joe   20  85.10\n",
      "1    Nat   21  77.80\n",
      "2  Harry   19  91.54\n",
      "\n",
      "After set index: \n",
      "              Age\n",
      "Name  Marks     \n",
      "Joe   85.10   20\n",
      "Nat   77.80   21\n",
      "Harry 91.54   19\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "student_dict = {'Name':['Joe','Nat','Harry'], 'Age':[20,21,19], 'Marks':[85.10, 77.80, 91.54]}\n",
    "\n",
    "# create DataFrame from dict\n",
    "student_df = pd.DataFrame(student_dict)\n",
    "print(\"Before set index: \\n\", student_df)\n",
    "\n",
    "# set index\n",
    "cols = list(student_df.columns[[0,2]])\n",
    "student_df = student_df.set_index(cols)\n",
    "print(\"\\nAfter set index: \\n\", student_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "hide_input": false,
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}