7.Creating DataFrames.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c3e96b69",
   "metadata": {},
   "source": [
    "# Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fd58b28b",
   "metadata": {},
   "source": [
    "DataFrame is a data structure provided by pandas library,apart from Series & Panel. It is a 2-\n",
    "dimensional structure & can be compared to a table of rows and columns.\n",
    "\n",
    "\n",
    "Each row can be identified by an integer index (0..N) or a label explicitly set when creating a\n",
    "DataFrame object. Each column can be of distinct type and is identified by a label.\n",
    "\n",
    "\n",
    "This topic covers various ways to construct/create a DataFrame object. Ex. from Numpy arrays,\n",
    "from list of tuples, from dictionary."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52c470d0",
   "metadata": {},
   "source": [
    "### Creating a sample dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "20742474",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>numbers</th>\n",
       "      <th>colors</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>red</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>white</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>blue</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   numbers colors\n",
       "0        1    red\n",
       "1        2  white\n",
       "2        3   blue"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.DataFrame({'numbers': [1, 2, 3], 'colors': ['red', 'white', 'blue']})\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "038a0d6f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>numbers</th>\n",
       "      <th>colors</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>red</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>white</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>blue</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   numbers colors\n",
       "0        1    red\n",
       "1        2  white\n",
       "2        3   blue"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame({'numbers': [1, 2, 3], 'colors': ['red', 'white', 'blue']},\n",
    " columns=['numbers', 'colors'])\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6d8fa951",
   "metadata": {},
   "source": [
    "#### Creating dataframe using random numbers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "e340c832",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.764052</td>\n",
       "      <td>0.400157</td>\n",
       "      <td>0.978738</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2.240893</td>\n",
       "      <td>1.867558</td>\n",
       "      <td>-0.977278</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.950088</td>\n",
       "      <td>-0.151357</td>\n",
       "      <td>-0.103219</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.410599</td>\n",
       "      <td>0.144044</td>\n",
       "      <td>1.454274</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.761038</td>\n",
       "      <td>0.121675</td>\n",
       "      <td>0.443863</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          A         B         C\n",
       "0  1.764052  0.400157  0.978738\n",
       "1  2.240893  1.867558 -0.977278\n",
       "2  0.950088 -0.151357 -0.103219\n",
       "3  0.410599  0.144044  1.454274\n",
       "4  0.761038  0.121675  0.443863"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "# Set the seed for a reproducible sample\n",
    "np.random.seed(0)\n",
    "df = pd.DataFrame(np.random.randn(5, 3), columns=list('ABC'))\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23da92f5",
   "metadata": {},
   "source": [
    "#### Create a DataFrame with integers:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "598f428e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>6</td>\n",
       "      <td>7</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>9</td>\n",
       "      <td>10</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>12</td>\n",
       "      <td>13</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    A   B   C\n",
       "0   0   1   2\n",
       "1   3   4   5\n",
       "2   6   7   8\n",
       "3   9  10  11\n",
       "4  12  13  14"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(np.arange(15).reshape(5,3),columns=list('ABC'))\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "15b3afc0",
   "metadata": {},
   "source": [
    "#### Create a DataFrame and include nans (NaT, NaN, 'nan', None) across columns and rows:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "41e46c7a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "      <th>E</th>\n",
       "      <th>F</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>6</td>\n",
       "      <td>7</td>\n",
       "      <td>8</td>\n",
       "      <td>9</td>\n",
       "      <td>10</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>12</td>\n",
       "      <td>13</td>\n",
       "      <td>14</td>\n",
       "      <td>15</td>\n",
       "      <td>16</td>\n",
       "      <td>17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>18</td>\n",
       "      <td>19</td>\n",
       "      <td>20</td>\n",
       "      <td>21</td>\n",
       "      <td>22</td>\n",
       "      <td>23</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>24</td>\n",
       "      <td>25</td>\n",
       "      <td>26</td>\n",
       "      <td>27</td>\n",
       "      <td>28</td>\n",
       "      <td>29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>30</td>\n",
       "      <td>31</td>\n",
       "      <td>32</td>\n",
       "      <td>33</td>\n",
       "      <td>34</td>\n",
       "      <td>35</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>36</td>\n",
       "      <td>37</td>\n",
       "      <td>38</td>\n",
       "      <td>39</td>\n",
       "      <td>40</td>\n",
       "      <td>41</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>42</td>\n",
       "      <td>43</td>\n",
       "      <td>44</td>\n",
       "      <td>45</td>\n",
       "      <td>46</td>\n",
       "      <td>47</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    A   B   C   D   E   F\n",
       "0   0   1   2   3   4   5\n",
       "1   6   7   8   9  10  11\n",
       "2  12  13  14  15  16  17\n",
       "3  18  19  20  21  22  23\n",
       "4  24  25  26  27  28  29\n",
       "5  30  31  32  33  34  35\n",
       "6  36  37  38  39  40  41\n",
       "7  42  43  44  45  46  47"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(np.arange(48).reshape(8,6),columns=list('ABCDEF'))\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "d2089a36",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.iloc[::2,0] = np.nan # in column 0, set elements with indices 0,2,4, ... to NaN\n",
    "df.iloc[::4,1] = pd.NaT # in column 1, set elements with indices 0,4, ... to np.NaT\n",
    "df.iloc[:3,2] = 'nan' # in column 2, set elements with index from 0 to 3 to 'nan'\n",
    "df.iloc[:,5] = None # in column 5, set all elements to None\n",
    "df.iloc[5,:] = None # in row 5, set all elements to None\n",
    "df.iloc[7,:] = np.nan # in row 7, set all elements to NaN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "44e18120",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "      <th>E</th>\n",
       "      <th>F</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaT</td>\n",
       "      <td>nan</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>6.0</td>\n",
       "      <td>7</td>\n",
       "      <td>nan</td>\n",
       "      <td>9.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NaN</td>\n",
       "      <td>13</td>\n",
       "      <td>nan</td>\n",
       "      <td>15.0</td>\n",
       "      <td>16.0</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>18.0</td>\n",
       "      <td>19</td>\n",
       "      <td>20</td>\n",
       "      <td>21.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaT</td>\n",
       "      <td>26</td>\n",
       "      <td>27.0</td>\n",
       "      <td>28.0</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>NaN</td>\n",
       "      <td>37</td>\n",
       "      <td>38</td>\n",
       "      <td>39.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      A     B     C     D     E     F\n",
       "0   NaN   NaT   nan   3.0   4.0  None\n",
       "1   6.0     7   nan   9.0  10.0  None\n",
       "2   NaN    13   nan  15.0  16.0  None\n",
       "3  18.0    19    20  21.0  22.0  None\n",
       "4   NaN   NaT    26  27.0  28.0  None\n",
       "5   NaN  None  None   NaN   NaN  None\n",
       "6   NaN    37    38  39.0  40.0  None\n",
       "7   NaN   NaN   NaN   NaN   NaN   NaN"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6fa5a49f",
   "metadata": {},
   "source": [
    "#### Create a sample DataFrame from multiple collections using Dictionary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "67f91d4f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>X</th>\n",
       "      <th>Y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-1.085631</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.997345</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.282978</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-1.506295</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          X  Y\n",
       "0 -1.085631  0\n",
       "1  0.997345  1\n",
       "2  0.282978  2\n",
       "3 -1.506295  3"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "np.random.seed(123)\n",
    "x = np.random.standard_normal(4)\n",
    "y = range(4)\n",
    "df = pd.DataFrame({'X':x, 'Y':y})\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2928fc7c",
   "metadata": {},
   "source": [
    "#### Create a DataFrame from a list of tuples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "05fc762b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>p1</td>\n",
       "      <td>t1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>p1</td>\n",
       "      <td>t2</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>p2</td>\n",
       "      <td>t1</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>p2</td>\n",
       "      <td>t2</td>\n",
       "      <td>7</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>p2</td>\n",
       "      <td>t3</td>\n",
       "      <td>2</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    0   1  2  3\n",
       "0  p1  t1  1  2\n",
       "1  p1  t2  3  4\n",
       "2  p2  t1  5  6\n",
       "3  p2  t2  7  8\n",
       "4  p2  t3  2  8"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "data = [\n",
    "('p1', 't1', 1, 2),\n",
    "('p1', 't2', 3, 4),\n",
    "('p2', 't1', 5, 6),\n",
    "('p2', 't2', 7, 8),\n",
    "('p2', 't3', 2, 8)\n",
    "]\n",
    "df = pd.DataFrame(data)\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cff6ee40",
   "metadata": {},
   "source": [
    "#### Create a DataFrame from a dictionary of lists"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "615f28ca",
   "metadata": {},
   "source": [
    "Create a DataFrame from multiple lists by passing a dict whose values lists. The keys of the\n",
    "dictionary are used as column labels. The lists can also be ndarrays. The lists/ndarrays must all be the same length.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "b55e4a53",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   A  B\n",
       "0  1  4\n",
       "1  2  3\n",
       "2  3  2\n",
       "3  4  1"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "# Create DF from dict of lists/ndarrays\n",
    "df = pd.DataFrame({'A' : [1, 2, 3, 4],\n",
    " 'B' : [4, 3, 2, 1]})\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca5ede61",
   "metadata": {},
   "source": [
    "If the arrays are not the same length an error is raised"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "e8ddb45f",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "All arrays must be of the same length",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[1;32m~\\AppData\\Local\\Temp/ipykernel_13364/1142886684.py\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mdf\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mpd\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mDataFrame\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m{\u001b[0m\u001b[1;34m'A'\u001b[0m \u001b[1;33m:\u001b[0m \u001b[1;33m[\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m2\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m3\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m4\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m'B'\u001b[0m \u001b[1;33m:\u001b[0m \u001b[1;33m[\u001b[0m\u001b[1;36m5\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m5\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m5\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m}\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;31m# a ValueError is raised\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      2\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      3\u001b[0m \u001b[1;31m#please click the box below to expand the output of this column\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\AppData\\Local\\Programs\\Python\\Python39\\lib\\site-packages\\pandas\\core\\frame.py\u001b[0m in \u001b[0;36m__init__\u001b[1;34m(self, data, index, columns, dtype, copy)\u001b[0m\n\u001b[0;32m    612\u001b[0m         \u001b[1;32melif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdict\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    613\u001b[0m             \u001b[1;31m# GH#38939 de facto copy defaults to False only in non-dict cases\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 614\u001b[1;33m             \u001b[0mmgr\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mdict_to_mgr\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mindex\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcolumns\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mdtype\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcopy\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mcopy\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtyp\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mmanager\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    615\u001b[0m         \u001b[1;32melif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mma\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mMaskedArray\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    616\u001b[0m             \u001b[1;32mimport\u001b[0m \u001b[0mnumpy\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mma\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmrecords\u001b[0m \u001b[1;32mas\u001b[0m \u001b[0mmrecords\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\AppData\\Local\\Programs\\Python\\Python39\\lib\\site-packages\\pandas\\core\\internals\\construction.py\u001b[0m in \u001b[0;36mdict_to_mgr\u001b[1;34m(data, index, columns, dtype, typ, copy)\u001b[0m\n\u001b[0;32m    462\u001b[0m         \u001b[1;31m# TODO: can we get rid of the dt64tz special case above?\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    463\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 464\u001b[1;33m     return arrays_to_mgr(\n\u001b[0m\u001b[0;32m    465\u001b[0m         \u001b[0marrays\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdata_names\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mindex\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcolumns\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mdtype\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtyp\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mtyp\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mconsolidate\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mcopy\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    466\u001b[0m     )\n",
      "\u001b[1;32m~\\AppData\\Local\\Programs\\Python\\Python39\\lib\\site-packages\\pandas\\core\\internals\\construction.py\u001b[0m in \u001b[0;36marrays_to_mgr\u001b[1;34m(arrays, arr_names, index, columns, dtype, verify_integrity, typ, consolidate)\u001b[0m\n\u001b[0;32m    117\u001b[0m         \u001b[1;31m# figure out the index, if necessary\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    118\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mindex\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 119\u001b[1;33m             \u001b[0mindex\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0m_extract_index\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0marrays\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    120\u001b[0m         \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    121\u001b[0m             \u001b[0mindex\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mensure_index\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mindex\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\AppData\\Local\\Programs\\Python\\Python39\\lib\\site-packages\\pandas\\core\\internals\\construction.py\u001b[0m in \u001b[0;36m_extract_index\u001b[1;34m(data)\u001b[0m\n\u001b[0;32m    633\u001b[0m             \u001b[0mlengths\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mlist\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mset\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mraw_lengths\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    634\u001b[0m             \u001b[1;32mif\u001b[0m \u001b[0mlen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mlengths\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;33m>\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 635\u001b[1;33m                 \u001b[1;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"All arrays must be of the same length\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    636\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    637\u001b[0m             \u001b[1;32mif\u001b[0m \u001b[0mhave_dicts\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mValueError\u001b[0m: All arrays must be of the same length"
     ]
    }
   ],
   "source": [
    "df = pd.DataFrame({'A' : [1, 2, 3, 4], 'B' : [5, 5, 5]}) # a ValueError is raised\n",
    "\n",
    "#please click the box below to expand the output of this column"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2247f8c4",
   "metadata": {},
   "source": [
    "#### Using ndarrays"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "f7964922",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>X</th>\n",
       "      <th>Y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-1.085631</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.997345</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.282978</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-1.506295</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          X  Y\n",
       "0 -1.085631  0\n",
       "1  0.997345  1\n",
       "2  0.282978  2\n",
       "3 -1.506295  3"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "np.random.seed(123)\n",
    "x = np.random.standard_normal(4)\n",
    "y = range(4)\n",
    "df = pd.DataFrame({'X':x, 'Y':y})\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f149de0d",
   "metadata": {},
   "source": [
    "####  Create a sample DataFrame with datetime\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "87dd2212",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Val</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2015-02-24 00:00:00</td>\n",
       "      <td>1.764052</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2015-02-24 00:01:00</td>\n",
       "      <td>0.400157</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2015-02-24 00:02:00</td>\n",
       "      <td>0.978738</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2015-02-24 00:03:00</td>\n",
       "      <td>2.240893</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2015-02-24 00:04:00</td>\n",
       "      <td>1.867558</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 Date       Val\n",
       "0 2015-02-24 00:00:00  1.764052\n",
       "1 2015-02-24 00:01:00  0.400157\n",
       "2 2015-02-24 00:02:00  0.978738\n",
       "3 2015-02-24 00:03:00  2.240893\n",
       "4 2015-02-24 00:04:00  1.867558"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "np.random.seed(0)\n",
    "\n",
    "# create an array of 5 dates starting at '2015-02-24', one per minute\n",
    "rng = pd.date_range('2015-02-24', periods=5, freq='T')\n",
    "df = pd.DataFrame({ 'Date': rng, 'Val': np.random.randn(len(rng)) })\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "357b9ca4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Val</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2015-02-24</td>\n",
       "      <td>-0.977278</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2015-02-25</td>\n",
       "      <td>0.950088</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2015-02-26</td>\n",
       "      <td>-0.151357</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2015-02-27</td>\n",
       "      <td>-0.103219</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2015-02-28</td>\n",
       "      <td>0.410599</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        Date       Val\n",
       "0 2015-02-24 -0.977278\n",
       "1 2015-02-25  0.950088\n",
       "2 2015-02-26 -0.151357\n",
       "3 2015-02-27 -0.103219\n",
       "4 2015-02-28  0.410599"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# create an array of 5 dates starting at '2015-02-24', one per day\n",
    "rng = pd.date_range('2015-02-24', periods=5, freq='D')\n",
    "df = pd.DataFrame({ 'Date': rng, 'Val' : np.random.randn(len(rng))})\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "4cda033f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Val</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2015-12-31</td>\n",
       "      <td>0.144044</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2018-12-31</td>\n",
       "      <td>1.454274</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2021-12-31</td>\n",
       "      <td>0.761038</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2024-12-31</td>\n",
       "      <td>0.121675</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2027-12-31</td>\n",
       "      <td>0.443863</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        Date       Val\n",
       "0 2015-12-31  0.144044\n",
       "1 2018-12-31  1.454274\n",
       "2 2021-12-31  0.761038\n",
       "3 2024-12-31  0.121675\n",
       "4 2027-12-31  0.443863"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# create an array of 5 dates starting at '2015-02-24', one every 3 years\n",
    "rng = pd.date_range('2015-02-24', periods=5, freq='3A')\n",
    "df = pd.DataFrame({ 'Date': rng, 'Val' : np.random.randn(len(rng))})\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c47f699",
   "metadata": {},
   "source": [
    "#### DataFrame with  - DatetimeIndex:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "99d0ef8c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Val</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2015-02-24 00:00:00</th>\n",
       "      <td>1.764052</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-02-24 00:01:00</th>\n",
       "      <td>0.400157</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-02-24 00:02:00</th>\n",
       "      <td>0.978738</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-02-24 00:03:00</th>\n",
       "      <td>2.240893</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-02-24 00:04:00</th>\n",
       "      <td>1.867558</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                          Val\n",
       "2015-02-24 00:00:00  1.764052\n",
       "2015-02-24 00:01:00  0.400157\n",
       "2015-02-24 00:02:00  0.978738\n",
       "2015-02-24 00:03:00  2.240893\n",
       "2015-02-24 00:04:00  1.867558"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "np.random.seed(0)\n",
    "rng = pd.date_range('2015-02-24', periods=5, freq='T')\n",
    "df = pd.DataFrame({ 'Val' : np.random.randn(len(rng)) }, index=rng)\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05b86ad4",
   "metadata": {},
   "source": [
    "#### Offset-aliases for parameter freq in date_range:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d2d5b093",
   "metadata": {},
   "source": [
    "Alias     Description\n",
    "B         business day frequency  \n",
    "C         custom business day frequency (experimental)  \n",
    "D         calendar day frequency  \n",
    "W         weekly frequency  \n",
    "M         month end frequency  \n",
    "BM        business month end frequency  \n",
    "CBM       custom business month end frequency  \n",
    "MS        month start frequency  \n",
    "BMS       business month start frequency  \n",
    "CBMS      custom business month start frequency  \n",
    "Q         quarter end frequency  \n",
    "BQ        business quarter endfrequency  \n",
    "QS        quarter start frequency  \n",
    "BQS       business quarter start frequency  \n",
    "A         year end frequency  \n",
    "BA        business year end frequency  \n",
    "AS        year start frequency  \n",
    "BAS       business year start frequency  \n",
    "BH        business hour frequency  \n",
    "H         hourly frequency  \n",
    "T, min    minutely frequency  \n",
    "S         secondly frequency  \n",
    "L, ms     milliseconds  \n",
    "U, us     microseconds  \n",
    "N         nanoseconds "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cae439b9",
   "metadata": {},
   "source": [
    "#### Create a sample DataFrame with MultiIndex"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "ab9fa51a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MultiIndex([('bar', 'one'),\n",
       "            ('bar', 'two'),\n",
       "            ('baz', 'one'),\n",
       "            ('baz', 'two'),\n",
       "            ('foo', 'one'),\n",
       "            ('foo', 'two'),\n",
       "            ('qux', 'one'),\n",
       "            ('qux', 'two')],\n",
       "           names=['first', 'second'])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.random.seed(0)\n",
    "tuples = list(zip(*[['bar', 'bar', 'baz', 'baz',\n",
    " 'foo', 'foo', 'qux', 'qux'],\n",
    " ['one', 'two', 'one', 'two',\n",
    " 'one', 'two', 'one', 'two']]))\n",
    "idx = pd.MultiIndex.from_tuples(tuples, names=['first', 'second'])\n",
    "idx"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a6534287",
   "metadata": {},
   "source": [
    "#### Using from_product:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "19cbfac8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MultiIndex([('bar', 'one'),\n",
       "            ('bar', 'two'),\n",
       "            ('baz', 'one'),\n",
       "            ('baz', 'two'),\n",
       "            ('foo', 'one'),\n",
       "            ('foo', 'two'),\n",
       "            ('qux', 'one'),\n",
       "            ('qux', 'two')],\n",
       "           )"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "idx = pd.MultiIndex.from_product([['bar', 'baz', 'foo', 'qux'],['one','two']])\n",
    "\n",
    "idx"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8782c71c",
   "metadata": {},
   "source": [
    "#### Then, use this MultiIndex:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "8018be3e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">bar</th>\n",
       "      <th>one</th>\n",
       "      <td>-0.887786</td>\n",
       "      <td>-1.980796</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>two</th>\n",
       "      <td>-0.347912</td>\n",
       "      <td>0.156349</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">baz</th>\n",
       "      <th>one</th>\n",
       "      <td>1.230291</td>\n",
       "      <td>1.202380</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>two</th>\n",
       "      <td>-0.387327</td>\n",
       "      <td>-0.302303</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">foo</th>\n",
       "      <th>one</th>\n",
       "      <td>-1.048553</td>\n",
       "      <td>-1.420018</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>two</th>\n",
       "      <td>-1.706270</td>\n",
       "      <td>1.950775</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">qux</th>\n",
       "      <th>one</th>\n",
       "      <td>-0.509652</td>\n",
       "      <td>-0.438074</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>two</th>\n",
       "      <td>-1.252795</td>\n",
       "      <td>0.777490</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                A         B\n",
       "bar one -0.887786 -1.980796\n",
       "    two -0.347912  0.156349\n",
       "baz one  1.230291  1.202380\n",
       "    two -0.387327 -0.302303\n",
       "foo one -1.048553 -1.420018\n",
       "    two -1.706270  1.950775\n",
       "qux one -0.509652 -0.438074\n",
       "    two -1.252795  0.777490"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(np.random.randn(8, 2), index=idx, columns=['A', 'B'])\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9eecf5f3",
   "metadata": {},
   "source": [
    "##### Save and Load a DataFrame in pickle (.plk) format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "c24d6d85",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">bar</th>\n",
       "      <th>one</th>\n",
       "      <td>-0.887786</td>\n",
       "      <td>-1.980796</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>two</th>\n",
       "      <td>-0.347912</td>\n",
       "      <td>0.156349</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">baz</th>\n",
       "      <th>one</th>\n",
       "      <td>1.230291</td>\n",
       "      <td>1.202380</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>two</th>\n",
       "      <td>-0.387327</td>\n",
       "      <td>-0.302303</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">foo</th>\n",
       "      <th>one</th>\n",
       "      <td>-1.048553</td>\n",
       "      <td>-1.420018</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>two</th>\n",
       "      <td>-1.706270</td>\n",
       "      <td>1.950775</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">qux</th>\n",
       "      <th>one</th>\n",
       "      <td>-0.509652</td>\n",
       "      <td>-0.438074</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>two</th>\n",
       "      <td>-1.252795</td>\n",
       "      <td>0.777490</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                A         B\n",
       "bar one -0.887786 -1.980796\n",
       "    two -0.347912  0.156349\n",
       "baz one  1.230291  1.202380\n",
       "    two -0.387327 -0.302303\n",
       "foo one -1.048553 -1.420018\n",
       "    two -1.706270  1.950775\n",
       "qux one -0.509652 -0.438074\n",
       "    two -1.252795  0.777490"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Save dataframe to pickled pandas object\n",
    "df.to_pickle('pwd') # where to save it usually as a .plk\n",
    "\n",
    "# Load dataframe from pickled pandas object\n",
    "df= pd.read_pickle('pwd')\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b79142cd",
   "metadata": {},
   "source": [
    "#### Create a DataFrame from a list of dictionaries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "288efdc5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Name</th>\n",
       "      <th>Last Name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>John</td>\n",
       "      <td>Smith</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Mary</td>\n",
       "      <td>Wood</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Name Last Name\n",
       "0  John     Smith\n",
       "1  Mary      Wood"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "L = [{'Name': 'John', 'Last Name': 'Smith'},\n",
    " {'Name': 'Mary', 'Last Name': 'Wood'}]\n",
    "pd.DataFrame(L)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "9ff288d8",
   "metadata": {},
   "outputs": [],
   "source": [
    "L = [{'Name': 'John', 'Last Name': 'Smith', 'Age': 37},\n",
    " {'Name': 'Mary', 'Last Name': 'Wood'}]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "e75bcd48",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Name</th>\n",
       "      <th>Last Name</th>\n",
       "      <th>Age</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>John</td>\n",
       "      <td>Smith</td>\n",
       "      <td>37.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Mary</td>\n",
       "      <td>Wood</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Name Last Name   Age\n",
       "0  John     Smith  37.0\n",
       "1  Mary      Wood   NaN"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(L)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "891b6308",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}