02_pandas_multiindex_examples.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Pandas MultiIndex examples\n",
    "\n",
    "When creating a DataFrame with different columns, the column names are accessible using `df.columns`, which will return an `Index` object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Index(['col1', 'col2'], dtype='object')\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>col1</th>\n",
       "      <th>col2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   col1  col2\n",
       "0     1     3\n",
       "1     2     4"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d = {'col1': [1, 2], 'col2': [3, 4]}\n",
    "df = pd.DataFrame(data=d)\n",
    "print(df.columns)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After adding a new column, it appears in the `Index` returned by `df.columns`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Index(['col1', 'col2', 'col3'], dtype='object')\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>col1</th>\n",
       "      <th>col2</th>\n",
       "      <th>col3</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   col1  col2  col3\n",
       "0     1     3     5\n",
       "1     2     4     6"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['col3'] = [5, 6]\n",
    "print(df.columns)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After dropping a column, using `axis=1`, it is no longer in the `Index` object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['col2', 'col3'], dtype='object')"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>col2</th>\n",
       "      <th>col3</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   col2  col3\n",
       "0     3     5\n",
       "1     4     6"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = df.drop(['col1'], axis=1)\n",
    "display(x.columns)\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But how does it work for a hierarchical MultiIndex?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th>bodypart</th>\n",
       "      <th colspan=\"2\" halign=\"left\">head</th>\n",
       "      <th colspan=\"2\" halign=\"left\">body</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>coordinates</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>9</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "bodypart    head    body   \n",
       "coordinates    x  y    x  y\n",
       "0              1  2    3  4\n",
       "1              9  2    3  4"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "columns = pd.MultiIndex.from_product([['head', 'body'], ['x', 'y']],\n",
    "                                     names=['bodypart', 'coordinates'])\n",
    "\n",
    "df = pd.DataFrame([[1, 2, 3, 4], [9, 2, 3, 4]], columns=columns)\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MultiIndex([('head', 'x'),\n",
       "            ('head', 'y'),\n",
       "            ('body', 'x'),\n",
       "            ('body', 'y')],\n",
       "           names=['bodypart', 'coordinates'])"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "FrozenList([['body', 'head'], ['x', 'y']])"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "['body', 'head']"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(df.columns)\n",
    "display(df.columns.levels)\n",
    "display(list(df.columns.levels[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What happens if we add another column? Will it appear in the column index?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th>bodypart</th>\n",
       "      <th colspan=\"2\" halign=\"left\">head</th>\n",
       "      <th colspan=\"2\" halign=\"left\">body</th>\n",
       "      <th colspan=\"2\" halign=\"left\">tail</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>coordinates</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>9</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "bodypart    head    body    tail   \n",
       "coordinates    x  y    x  y    x  y\n",
       "0              1  2    3  4    5  9\n",
       "1              9  2    3  4    5  9"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "MultiIndex([('head', 'x'),\n",
       "            ('head', 'y'),\n",
       "            ('body', 'x'),\n",
       "            ('body', 'y'),\n",
       "            ('tail', 'x'),\n",
       "            ('tail', 'y')],\n",
       "           names=['bodypart', 'coordinates'])"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "FrozenList([['body', 'head', 'tail'], ['x', 'y']])"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "['body', 'head', 'tail']"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df['tail', 'x'] = 5\n",
    "df['tail', 'y'] = 9\n",
    "\n",
    "display(df)\n",
    "\n",
    "display(df.columns)\n",
    "display(df.columns.levels)\n",
    "display(list(df.columns.levels[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It seems to be in columns level 0!\n",
    "And what happens if the drop on of the original columns? Is the columns `Index` updated accordingly?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/kwittek/.local/share/virtualenvs/winkie-pT7_bdXL/lib/python3.8/site-packages/pandas/core/generic.py:4150: PerformanceWarning: dropping on a non-lexsorted multi-index without a level parameter may impact performance.\n",
      "  obj = obj._drop_axis(labels, axis, level=level, errors=errors)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th>bodypart</th>\n",
       "      <th colspan=\"2\" halign=\"left\">head</th>\n",
       "      <th colspan=\"2\" halign=\"left\">tail</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>coordinates</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>9</td>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "bodypart    head    tail   \n",
       "coordinates    x  y    x  y\n",
       "0              1  2    5  9\n",
       "1              9  2    5  9"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "MultiIndex([('head', 'x'),\n",
       "            ('head', 'y'),\n",
       "            ('tail', 'x'),\n",
       "            ('tail', 'y')],\n",
       "           names=['bodypart', 'coordinates'])"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "FrozenList([['body', 'head', 'tail'], ['x', 'y']])"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "['body', 'head', 'tail']"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df_drop = df.drop(['body'], axis=1)\n",
    "\n",
    "display(df_drop)\n",
    "\n",
    "display(df_drop.columns)\n",
    "display(df_drop.columns.levels)\n",
    "display(list(df_drop.columns.levels[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So we can see, dropping a column does not remove it from the column index (which is backed by a `FrozenList`)! While some might consider this a bug, the [pandas developers think](https://github.com/pandas-dev/pandas/issues/3686) this is a philosophical question and actually works as intended.\n",
    "\n",
    "However, there is a good workarond:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['head', 'tail'], dtype='object', name='bodypart')"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(df_drop.columns.get_level_values(0).unique())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is another way, which means setting a new column index. While it seems this is a reasonable approach for some use cases, there might be unforseen (performance) implication, which are the reasons, that this is not the default behaviour."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "FrozenList([['head', 'tail'], ['x', 'y']])"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df_drop.columns = df_drop.columns.remove_unused_levels()\n",
    "display(df_drop.columns.levels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And how can we set values on certain values of a MutliIndexed DataFrame, if there is a non-MultiIndexed column?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th>bodypart</th>\n",
       "      <th colspan=\"2\" halign=\"left\">head</th>\n",
       "      <th colspan=\"2\" halign=\"left\">body</th>\n",
       "      <th colspan=\"2\" halign=\"left\">tail</th>\n",
       "      <th>behavior</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>coordinates</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>9</td>\n",
       "      <td>cute</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>9</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>9</td>\n",
       "      <td>cute</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "bodypart    head    body    tail    behavior\n",
       "coordinates    x  y    x  y    x  y         \n",
       "0              1  2    3  4    5  9     cute\n",
       "1              9  2    3  4    5  9     cute"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['behavior'] = 'cute'\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-12-5a1259736b5a>:1: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  df.behavior[df['head', 'x'] == 1] = 'foobar' # this does not work!\n",
      "/home/kwittek/.local/share/virtualenvs/winkie-pT7_bdXL/lib/python3.8/site-packages/pandas/core/series.py:963: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  self._where(~key, value, inplace=True)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th>bodypart</th>\n",
       "      <th colspan=\"2\" halign=\"left\">head</th>\n",
       "      <th colspan=\"2\" halign=\"left\">body</th>\n",
       "      <th colspan=\"2\" halign=\"left\">tail</th>\n",
       "      <th>behavior</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>coordinates</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>9</td>\n",
       "      <td>cute</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>9</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>9</td>\n",
       "      <td>cute</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "bodypart    head    body    tail    behavior\n",
       "coordinates    x  y    x  y    x  y         \n",
       "0              1  2    3  4    5  9     cute\n",
       "1              9  2    3  4    5  9     cute"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.behavior[df['head', 'x'] == 1] = 'foobar' # this does not work!\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, setting the value using chained indexing does not work. This comes from the fact how the Pandas DSL is translated into Python method calls. The [official docs](https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy) provide a detailed explanation of the reasons. You are actually getting a copy! (which is logged in a warning, that might be visible, depending on how you render the notebook)\n",
    "\n",
    "Instead, we should make use of the [`loc`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.loc.html) method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th>bodypart</th>\n",
       "      <th colspan=\"2\" halign=\"left\">head</th>\n",
       "      <th colspan=\"2\" halign=\"left\">body</th>\n",
       "      <th colspan=\"2\" halign=\"left\">tail</th>\n",
       "      <th>behavior</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>coordinates</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>9</td>\n",
       "      <td>buzzz</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>9</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>9</td>\n",
       "      <td>cute</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "bodypart    head    body    tail    behavior\n",
       "coordinates    x  y    x  y    x  y         \n",
       "0              1  2    3  4    5  9    buzzz\n",
       "1              9  2    3  4    5  9     cute"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.loc[df['head', 'x'] == 1, ['behavior']] = 'buzzz'\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Behind the scenes, the [operator overloading](https://docs.python.org/3/reference/datamodel.html#special-method-names) of the Python data model come into practice, which are used extensively by the Pandas DSL. What looks similar to a method call, will actually call the overloaded `__getitem__` method on an internal `_LocIndexer` object. While the internal code branches are a bit more involved, what will happen functionally in our case is using the first argument as a boolean Series to select specific rows and the second argument for specifying which column to access (and thereby override)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}