getting_started_tutorials/10min_to_cudf_colab.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "c6ed860b",
      "metadata": {
        "id": "c6ed860b"
      },
      "source": [
        "<a target=\"_blank\" href=\"https://colab.research.google.com/github/rapidsai-community/showcase/blob/main/getting_started_tutorials/10min_to_cudf_colab.ipynb\">\n",
        "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>\n",
        "</a>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "Td_alkbOv3Aj",
      "metadata": {
        "id": "Td_alkbOv3Aj"
      },
      "source": [
        "# 10 Minutes to cuDF\n",
        "\n",
        "Modeled after 10 Minutes to Pandas, this is a short introduction to cuDF, geared mainly towards new users.\n",
        "\n",
        "[cuDF](https://github.com/rapidsai/cudf) is a Python GPU DataFrame library to accelerate loading, joining, aggregating, filtering, and otherwise manipulating tabular data using a DataFrame style API in the style of [pandas](https://pandas.pydata.org).\n",
        "\n",
        "cuDF runs on a single GPU. If you want to distribute your workflow across multiple GPUs, have more data than you can fit in memory on a single GPU, or want to analyze data spread across many files at once, you may want to use [Dask-cuDF](https://github.com/rapidsai/cudf/tree/main/python/dask_cudf)."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "AhUsdz6jLdMi",
      "metadata": {
        "id": "AhUsdz6jLdMi"
      },
      "source": [
        "\n",
        "Before getting started - be sure to change your runtime to use a GPU Hardware accelerator! Use the Runtime -> \"Change runtime type\" menu option to add a GPU."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ZfNDlz0SM0DB",
      "metadata": {
        "id": "ZfNDlz0SM0DB"
      },
      "source": [
        "# Let's get started using RAPIDS"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "id": "PzW61-K04A1E",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PzW61-K04A1E",
        "outputId": "97252486-7715-4619-c31e-41d6e4c3d522"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Wed May 29 22:28:36 2024       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   57C    P8              12W /  70W |      0MiB / 15360MiB |      0%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "|  No running processes found                                                           |\n",
            "+---------------------------------------------------------------------------------------+\n"
          ]
        }
      ],
      "source": [
        "!nvidia-smi"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "id": "36307e42",
      "metadata": {
        "id": "36307e42"
      },
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "import cudf\n",
        "\n",
        "import cupy as cp\n",
        "import os\n",
        "\n",
        "cp.random.seed(0)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "eff5fc19",
      "metadata": {
        "id": "eff5fc19"
      },
      "source": [
        "## Creating Series and DataFrame objects\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "0a747886",
      "metadata": {
        "id": "0a747886"
      },
      "source": [
        "Creating a `cudf.Series`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "id": "f5e303df",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "f5e303df",
        "outputId": "ea0c5d3e-f358-4692-e336-d2d14df32475"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0       1\n",
              "1       2\n",
              "2       3\n",
              "3    <NA>\n",
              "4       4\n",
              "dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 3
        }
      ],
      "source": [
        "s = cudf.Series([1, 2, 3, None, 4])\n",
        "s"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "d934af4e",
      "metadata": {
        "id": "d934af4e"
      },
      "source": [
        "Creating a `cudf.DataFrame` by specifying values for each column."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "id": "3f53fb8b",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 676
        },
        "id": "3f53fb8b",
        "outputId": "e48e747e-ceac-429a-ba44-1109a5fa8bfa"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "     a   b   c\n",
              "0    0  19   0\n",
              "1    1  18   1\n",
              "2    2  17   2\n",
              "3    3  16   3\n",
              "4    4  15   4\n",
              "5    5  14   5\n",
              "6    6  13   6\n",
              "7    7  12   7\n",
              "8    8  11   8\n",
              "9    9  10   9\n",
              "10  10   9  10\n",
              "11  11   8  11\n",
              "12  12   7  12\n",
              "13  13   6  13\n",
              "14  14   5  14\n",
              "15  15   4  15\n",
              "16  16   3  16\n",
              "17  17   2  17\n",
              "18  18   1  18\n",
              "19  19   0  19"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>a</th>\n",
              "      <th>b</th>\n",
              "      <th>c</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>19</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>18</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "      <td>17</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3</td>\n",
              "      <td>16</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>4</td>\n",
              "      <td>15</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>5</td>\n",
              "      <td>14</td>\n",
              "      <td>5</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>6</td>\n",
              "      <td>13</td>\n",
              "      <td>6</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>7</td>\n",
              "      <td>12</td>\n",
              "      <td>7</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>8</td>\n",
              "      <td>11</td>\n",
              "      <td>8</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>9</td>\n",
              "      <td>10</td>\n",
              "      <td>9</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>10</td>\n",
              "      <td>9</td>\n",
              "      <td>10</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>11</td>\n",
              "      <td>8</td>\n",
              "      <td>11</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12</th>\n",
              "      <td>12</td>\n",
              "      <td>7</td>\n",
              "      <td>12</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13</th>\n",
              "      <td>13</td>\n",
              "      <td>6</td>\n",
              "      <td>13</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>14</td>\n",
              "      <td>5</td>\n",
              "      <td>14</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>15</td>\n",
              "      <td>4</td>\n",
              "      <td>15</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>16</td>\n",
              "      <td>3</td>\n",
              "      <td>16</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17</th>\n",
              "      <td>17</td>\n",
              "      <td>2</td>\n",
              "      <td>17</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>18</th>\n",
              "      <td>18</td>\n",
              "      <td>1</td>\n",
              "      <td>18</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19</th>\n",
              "      <td>19</td>\n",
              "      <td>0</td>\n",
              "      <td>19</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ],
      "source": [
        "df = cudf.DataFrame(\n",
        "    {\n",
        "        \"a\": list(range(20)),\n",
        "        \"b\": list(reversed(range(20))),\n",
        "        \"c\": list(range(20)),\n",
        "    }\n",
        ")\n",
        "df"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "0573b7bb",
      "metadata": {
        "id": "0573b7bb"
      },
      "source": [
        "Creating a `cudf.DataFrame` from a pandas `Dataframe`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "id": "06a42f3a",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 175
        },
        "id": "06a42f3a",
        "outputId": "43799344-b367-4c92-904b-5303921702b8"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   a     b\n",
              "0  0   0.1\n",
              "1  1   0.2\n",
              "2  2  <NA>\n",
              "3  3   0.3"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>a</th>\n",
              "      <th>b</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0.1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>0.2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "      <td>&lt;NA&gt;</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3</td>\n",
              "      <td>0.3</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ],
      "source": [
        "pdf = pd.DataFrame({\"a\": [0, 1, 2, 3], \"b\": [0.1, 0.2, None, 0.3]})\n",
        "gdf = cudf.DataFrame.from_pandas(pdf)\n",
        "gdf"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5820795f",
      "metadata": {
        "id": "5820795f"
      },
      "source": [
        "## Viewing Data"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "b3008757",
      "metadata": {
        "id": "b3008757"
      },
      "source": [
        "Viewing the top rows of a GPU dataframe."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "id": "0c329914",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 112
        },
        "id": "0c329914",
        "outputId": "6e382417-dfd6-4be7-c1e4-46eaa0250757"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   a   b  c\n",
              "0  0  19  0\n",
              "1  1  18  1"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>a</th>\n",
              "      <th>b</th>\n",
              "      <th>c</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>19</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>18</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ],
      "source": [
        "df.head(2)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "s4n0S0yjACsh",
      "metadata": {
        "id": "s4n0S0yjACsh"
      },
      "source": [
        "# Selecting Data"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "3302a647",
      "metadata": {
        "id": "3302a647"
      },
      "source": [
        "## Getting"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "aebc72ab",
      "metadata": {
        "id": "aebc72ab"
      },
      "source": [
        "Selecting a single column, which yields a `cudf.Series`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "id": "4dafb4ed",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4dafb4ed",
        "outputId": "fa830834-abc5-42e6-89d3-dce4c057e868"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0      0\n",
              "1      1\n",
              "2      2\n",
              "3      3\n",
              "4      4\n",
              "5      5\n",
              "6      6\n",
              "7      7\n",
              "8      8\n",
              "9      9\n",
              "10    10\n",
              "11    11\n",
              "12    12\n",
              "13    13\n",
              "14    14\n",
              "15    15\n",
              "16    16\n",
              "17    17\n",
              "18    18\n",
              "19    19\n",
              "Name: a, dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ],
      "source": [
        "df[\"a\"]"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "a5160dd1",
      "metadata": {
        "id": "a5160dd1"
      },
      "source": [
        "## Selection by Label"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "51ff2093",
      "metadata": {
        "id": "51ff2093"
      },
      "source": [
        "Selecting rows from index 2 to index 5 from columns 'a' and 'b'."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "id": "e8870657",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 175
        },
        "id": "e8870657",
        "outputId": "f71ca6a3-24fb-45d6-f6d0-2b3ba75b48ec"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   a   b\n",
              "2  2  17\n",
              "3  3  16\n",
              "4  4  15\n",
              "5  5  14"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>a</th>\n",
              "      <th>b</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "      <td>17</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3</td>\n",
              "      <td>16</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>4</td>\n",
              "      <td>15</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>5</td>\n",
              "      <td>14</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ],
      "source": [
        "df.loc[2:5, [\"a\", \"b\"]]"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "d8e07162",
      "metadata": {
        "id": "d8e07162"
      },
      "source": [
        "## Selection by Position"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "435eb76a",
      "metadata": {
        "id": "435eb76a"
      },
      "source": [
        "Selecting via integers and integer slices, like numpy/pandas."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "id": "bc337d5d",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bc337d5d",
        "outputId": "73e0cf23-39f6-4e66-8c6d-8659d0faa41f"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "a     0\n",
              "b    19\n",
              "c     0\n",
              "Name: 0, dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ],
      "source": [
        "df.iloc[0]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "id": "03671456",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 143
        },
        "id": "03671456",
        "outputId": "46a2dbdc-40d0-4d6b-afe8-f3a2cf783ff0"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   a   b\n",
              "0  0  19\n",
              "1  1  18\n",
              "2  2  17"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>a</th>\n",
              "      <th>b</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>19</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>18</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "      <td>17</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ],
      "source": [
        "df.iloc[0:3, 0:2]"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "aa935d5b",
      "metadata": {
        "id": "aa935d5b"
      },
      "source": [
        "You can also select elements of a `DataFrame` or `Series` with direct index access."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "id": "79883c37",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 112
        },
        "id": "79883c37",
        "outputId": "35fe5d8d-35e9-4cd3-9663-0ab1268e7d37"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   a   b  c\n",
              "3  3  16  3\n",
              "4  4  15  4"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>a</th>\n",
              "      <th>b</th>\n",
              "      <th>c</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3</td>\n",
              "      <td>16</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>4</td>\n",
              "      <td>15</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ],
      "source": [
        "df[3:5]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "id": "2f761695",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2f761695",
        "outputId": "cf6b95a4-46d0-40b3-c4b8-072a08e0d2e0"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3    <NA>\n",
              "4       4\n",
              "dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ],
      "source": [
        "s[3:5]"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "9bf9c0b0",
      "metadata": {
        "id": "9bf9c0b0"
      },
      "source": [
        "## Boolean Indexing"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "9b08b2b9",
      "metadata": {
        "id": "9b08b2b9"
      },
      "source": [
        "Selecting rows in a `DataFrame` or `Series` by direct Boolean indexing."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "id": "1eb08f0d",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 175
        },
        "id": "1eb08f0d",
        "outputId": "bb314fdb-3af4-498c-d1e0-4014630eeb93"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   a   b  c\n",
              "0  0  19  0\n",
              "1  1  18  1\n",
              "2  2  17  2\n",
              "3  3  16  3"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>a</th>\n",
              "      <th>b</th>\n",
              "      <th>c</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>19</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>18</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "      <td>17</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3</td>\n",
              "      <td>16</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ],
      "source": [
        "df[df.b > 15]"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "f95c9ab7",
      "metadata": {
        "id": "f95c9ab7"
      },
      "source": [
        "Selecting values from a `DataFrame` where a Boolean condition is met, via the `query` API."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "id": "fa643410",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 81
        },
        "id": "fa643410",
        "outputId": "61b8939c-47a4-4e91-c411-8063dfe556f2"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "     a  b   c\n",
              "16  16  3  16"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>a</th>\n",
              "      <th>b</th>\n",
              "      <th>c</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>16</td>\n",
              "      <td>3</td>\n",
              "      <td>16</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ],
      "source": [
        "df.query(\"b == 3\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "694dcbad",
      "metadata": {
        "id": "694dcbad"
      },
      "source": [
        "With standard cuDF, you may either use the `local_dict` keyword or directly pass the variable via the `@` keyword. Supported logical operators include `>`, `<`, `>=`, `<=`, `==`, and `!=`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "id": "353b0250",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 81
        },
        "id": "353b0250",
        "outputId": "8da14e18-120a-4a4b-dceb-56067a946b95"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "     a  b   c\n",
              "16  16  3  16"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>a</th>\n",
              "      <th>b</th>\n",
              "      <th>c</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>16</td>\n",
              "      <td>3</td>\n",
              "      <td>16</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ],
      "source": [
        "cudf_comparator = 3\n",
        "df.query(\"b == @cudf_comparator\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "8e004749",
      "metadata": {
        "id": "8e004749"
      },
      "source": [
        "Using the `isin` method for filtering."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "id": "20936418",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 112
        },
        "id": "20936418",
        "outputId": "a644a1c7-a039-496c-baa3-888878c18af0"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   a   b  c\n",
              "0  0  19  0\n",
              "5  5  14  5"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>a</th>\n",
              "      <th>b</th>\n",
              "      <th>c</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>19</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>5</td>\n",
              "      <td>14</td>\n",
              "      <td>5</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ],
      "source": [
        "df[df.a.isin([0, 5])]"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "8e456f03",
      "metadata": {
        "id": "8e456f03"
      },
      "source": [
        "## MultiIndex"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "e494bd0b",
      "metadata": {
        "id": "e494bd0b"
      },
      "source": [
        "cuDF supports hierarchical indexing of DataFrames using MultiIndex. Grouping hierarchically (see `Grouping` below) automatically produces a DataFrame with a MultiIndex."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "id": "4ae70724",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4ae70724",
        "outputId": "3400a304-c88e-4f7d-b0c9-572f91fd9c4f"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "MultiIndex([('a', 1),\n",
              "            ('a', 2),\n",
              "            ('b', 3),\n",
              "            ('b', 4)],\n",
              "           )"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ],
      "source": [
        "arrays = [[\"a\", \"a\", \"b\", \"b\"], [1, 2, 3, 4]]\n",
        "tuples = list(zip(*arrays))\n",
        "idx = cudf.MultiIndex.from_tuples(tuples)\n",
        "idx"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ab232727",
      "metadata": {
        "id": "ab232727"
      },
      "source": [
        "This index can back either axis of a DataFrame."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "id": "cb1d1633",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 175
        },
        "id": "cb1d1633",
        "outputId": "8ad2213a-7a0c-457c-deee-0b3a564678cc"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "        first    second\n",
              "a 1  0.438451  0.053011\n",
              "  2  0.460365  0.337699\n",
              "b 3  0.250215  0.396763\n",
              "  4  0.494744  0.874419"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th>first</th>\n",
              "      <th>second</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th rowspan=\"2\" valign=\"top\">a</th>\n",
              "      <th>1</th>\n",
              "      <td>0.438451</td>\n",
              "      <td>0.053011</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0.460365</td>\n",
              "      <td>0.337699</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th rowspan=\"2\" valign=\"top\">b</th>\n",
              "      <th>3</th>\n",
              "      <td>0.250215</td>\n",
              "      <td>0.396763</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0.494744</td>\n",
              "      <td>0.874419</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ],
      "source": [
        "gdf1 = cudf.DataFrame(\n",
        "    {\"first\": cp.random.rand(4), \"second\": cp.random.rand(4)}\n",
        ")\n",
        "gdf1.index = idx\n",
        "gdf1"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "id": "73ba31af",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 143
        },
        "id": "73ba31af",
        "outputId": "275000a7-173e-4175-b553-3b5d5c35730c"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "               a                  b          \n",
              "               1        2         3         4\n",
              "first   0.482167  0.04284  0.508414  0.654550\n",
              "second  0.512604  0.26430  0.051981  0.578997"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead tr th {\n",
              "        text-align: left;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr>\n",
              "      <th></th>\n",
              "      <th colspan=\"2\" halign=\"left\">a</th>\n",
              "      <th colspan=\"2\" halign=\"left\">b</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th></th>\n",
              "      <th>1</th>\n",
              "      <th>2</th>\n",
              "      <th>3</th>\n",
              "      <th>4</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>first</th>\n",
              "      <td>0.482167</td>\n",
              "      <td>0.04284</td>\n",
              "      <td>0.508414</td>\n",
              "      <td>0.654550</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>second</th>\n",
              "      <td>0.512604</td>\n",
              "      <td>0.26430</td>\n",
              "      <td>0.051981</td>\n",
              "      <td>0.578997</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ],
      "source": [
        "gdf2 = cudf.DataFrame(\n",
        "    {\"first\": cp.random.rand(4), \"second\": cp.random.rand(4)}\n",
        ").T\n",
        "gdf2.columns = idx\n",
        "gdf2"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "c5f33160",
      "metadata": {
        "id": "c5f33160"
      },
      "source": [
        "Accessing values of a DataFrame with a MultiIndex, both with `.loc`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "id": "1048b7cf",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1048b7cf",
        "outputId": "31dd73d4-4d58-4568-f82c-479e4d14e8c2"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "first     0.250215\n",
              "second    0.396763\n",
              "Name: ('b', 3), dtype: float64"
            ]
          },
          "metadata": {},
          "execution_count": 20
        }
      ],
      "source": [
        "gdf1.loc[(\"b\", 3)]"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5123f759",
      "metadata": {
        "id": "5123f759"
      },
      "source": [
        "And `.iloc`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "id": "369d164d",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 112
        },
        "id": "369d164d",
        "outputId": "52e02c07-12af-4cf4-b4cf-e67806126691"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "        first    second\n",
              "a 1  0.438451  0.053011\n",
              "  2  0.460365  0.337699"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th>first</th>\n",
              "      <th>second</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th rowspan=\"2\" valign=\"top\">a</th>\n",
              "      <th>1</th>\n",
              "      <td>0.438451</td>\n",
              "      <td>0.053011</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0.460365</td>\n",
              "      <td>0.337699</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 21
        }
      ],
      "source": [
        "gdf1.iloc[0:2]"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "8b3b96e9",
      "metadata": {
        "id": "8b3b96e9"
      },
      "source": [
        "Missing Data\n",
        "------------"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "d12141eb",
      "metadata": {
        "id": "d12141eb"
      },
      "source": [
        "Missing data can be replaced by using the `fillna` method."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "id": "913a7b5f",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "913a7b5f",
        "outputId": "b0164968-374b-42eb-a2eb-d878a2eb17a8"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0      1\n",
              "1      2\n",
              "2      3\n",
              "3    999\n",
              "4      4\n",
              "dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ],
      "source": [
        "s.fillna(999)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "22199029",
      "metadata": {
        "id": "22199029"
      },
      "source": [
        "# Operating on Data"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "d97605e6",
      "metadata": {
        "id": "d97605e6"
      },
      "source": [
        "## Stats"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "f29a5de0",
      "metadata": {
        "id": "f29a5de0"
      },
      "source": [
        "Calculating descriptive statistics for a `Series`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "id": "b1a1666e",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "b1a1666e",
        "outputId": "42b65b0c-c254-4c39-946f-b906df60ea09"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(2.5, 1.666666666666666)"
            ]
          },
          "metadata": {},
          "execution_count": 23
        }
      ],
      "source": [
        "s.mean(), s.var()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "af792a18",
      "metadata": {
        "id": "af792a18"
      },
      "source": [
        "## Applymap"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "f6094cbe",
      "metadata": {
        "id": "f6094cbe"
      },
      "source": [
        "Applying functions to a `Series`. Note that applying user defined functions directly with Dask-cuDF is not yet implemented. For now, you can use [map_partitions](http://docs.dask.org/en/stable/generated/dask.dataframe.DataFrame.map_partitions.html) to apply a function to each partition of the distributed dataframe."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "id": "5b154619",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5b154619",
        "outputId": "2e24ef5f-6649-4d45-c218-7228b5fbef4d"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0     10\n",
              "1     11\n",
              "2     12\n",
              "3     13\n",
              "4     14\n",
              "5     15\n",
              "6     16\n",
              "7     17\n",
              "8     18\n",
              "9     19\n",
              "10    20\n",
              "11    21\n",
              "12    22\n",
              "13    23\n",
              "14    24\n",
              "15    25\n",
              "16    26\n",
              "17    27\n",
              "18    28\n",
              "19    29\n",
              "Name: a, dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 24
        }
      ],
      "source": [
        "def add_ten(num):\n",
        "    return num + 10\n",
        "\n",
        "\n",
        "df[\"a\"].apply(add_ten)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "4d4fdde1",
      "metadata": {
        "id": "4d4fdde1"
      },
      "source": [
        "## Histogramming"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "b98a7daf",
      "metadata": {
        "id": "b98a7daf"
      },
      "source": [
        "Counting the number of occurrences of each unique value of variable."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "id": "c7b8ea5d",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "c7b8ea5d",
        "outputId": "a8a6ff62-0e16-48a1-d1fa-8ea243cf5839"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "a\n",
              "16    1\n",
              "11    1\n",
              "18    1\n",
              "2     1\n",
              "5     1\n",
              "3     1\n",
              "15    1\n",
              "10    1\n",
              "9     1\n",
              "1     1\n",
              "6     1\n",
              "13    1\n",
              "7     1\n",
              "8     1\n",
              "17    1\n",
              "4     1\n",
              "12    1\n",
              "14    1\n",
              "0     1\n",
              "19    1\n",
              "Name: count, dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 25
        }
      ],
      "source": [
        "df.a.value_counts()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "437172ba",
      "metadata": {
        "id": "437172ba"
      },
      "source": [
        "## String Methods"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "fd3fc4f3",
      "metadata": {
        "id": "fd3fc4f3"
      },
      "source": [
        "Like pandas, cuDF provides string processing methods in the `str` attribute of `Series`. Full documentation of string methods is a work in progress. Please see the [cuDF API documentation](https://docs.rapids.ai/api/cudf/stable/api_docs/series.html#string-handling) for more information."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "id": "86974041",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "86974041",
        "outputId": "4acb86a4-1f73-46bd-9e5b-14527df4f191"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0       a\n",
              "1       b\n",
              "2       c\n",
              "3    aaba\n",
              "4    baca\n",
              "5    <NA>\n",
              "6    caba\n",
              "7     dog\n",
              "8     cat\n",
              "dtype: object"
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ],
      "source": [
        "s = cudf.Series([\"A\", \"B\", \"C\", \"Aaba\", \"Baca\", None, \"CABA\", \"dog\", \"cat\"])\n",
        "s.str.lower()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "44fe1243",
      "metadata": {
        "id": "44fe1243"
      },
      "source": [
        "As well as simple manipulation, We can also match strings using [regular expressions](https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.core.column.string.StringMethods.match.html)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "id": "51158a24",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "51158a24",
        "outputId": "8527a5c2-1b11-4ded-d770-009ecc2f03d0"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0    False\n",
              "1    False\n",
              "2    False\n",
              "3     True\n",
              "4    False\n",
              "5     <NA>\n",
              "6    False\n",
              "7    False\n",
              "8     True\n",
              "dtype: bool"
            ]
          },
          "metadata": {},
          "execution_count": 27
        }
      ],
      "source": [
        "s.str.match(\"^[aAc].+\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5528afa8",
      "metadata": {
        "id": "5528afa8"
      },
      "source": [
        "## Concat"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "e05b1078",
      "metadata": {
        "id": "e05b1078"
      },
      "source": [
        "Concatenating `Series` and `DataFrames` row-wise."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "id": "6c6d10bc",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6c6d10bc",
        "outputId": "26f71657-affa-466f-db06-0405b929cecf"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0       1\n",
              "1       2\n",
              "2       3\n",
              "3    <NA>\n",
              "4       5\n",
              "0       1\n",
              "1       2\n",
              "2       3\n",
              "3    <NA>\n",
              "4       5\n",
              "dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 28
        }
      ],
      "source": [
        "s = cudf.Series([1, 2, 3, None, 5])\n",
        "cudf.concat([s, s])"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "df087d2f",
      "metadata": {
        "id": "df087d2f"
      },
      "source": [
        "## Join"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "89cf5022",
      "metadata": {
        "id": "89cf5022"
      },
      "source": [
        "Performing SQL style merges. Note that the dataframe order is **not maintained**, but may be restored post-merge by sorting by the index."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "id": "075c97a7",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "075c97a7",
        "outputId": "45ec2ae2-9942-4101-d612-dd7d49b98908"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "  key  vals_a vals_b\n",
              "0   a    10.0  100.0\n",
              "1   c    12.0  101.0\n",
              "2   e    14.0  102.0\n",
              "3   b    11.0   <NA>\n",
              "4   d    13.0   <NA>"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>key</th>\n",
              "      <th>vals_a</th>\n",
              "      <th>vals_b</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>a</td>\n",
              "      <td>10.0</td>\n",
              "      <td>100.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>c</td>\n",
              "      <td>12.0</td>\n",
              "      <td>101.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>e</td>\n",
              "      <td>14.0</td>\n",
              "      <td>102.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>b</td>\n",
              "      <td>11.0</td>\n",
              "      <td>&lt;NA&gt;</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>d</td>\n",
              "      <td>13.0</td>\n",
              "      <td>&lt;NA&gt;</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 29
        }
      ],
      "source": [
        "df_a = cudf.DataFrame()\n",
        "df_a[\"key\"] = [\"a\", \"b\", \"c\", \"d\", \"e\"]\n",
        "df_a[\"vals_a\"] = [float(i + 10) for i in range(5)]\n",
        "\n",
        "df_b = cudf.DataFrame()\n",
        "df_b[\"key\"] = [\"a\", \"c\", \"e\"]\n",
        "df_b[\"vals_b\"] = [float(i + 100) for i in range(3)]\n",
        "\n",
        "merged = df_a.merge(df_b, on=[\"key\"], how=\"left\")\n",
        "merged"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "e4695c6e",
      "metadata": {
        "id": "e4695c6e"
      },
      "source": [
        "## Grouping"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ecb27b06",
      "metadata": {
        "id": "ecb27b06"
      },
      "source": [
        "Like [pandas](https://pandas.pydata.org/docs/user_guide/groupby.html), cuDF and Dask-cuDF support the [Split-Apply-Combine groupby paradigm](https://doi.org/10.18637/jss.v040.i01)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "id": "d8db18d9",
      "metadata": {
        "id": "d8db18d9"
      },
      "outputs": [],
      "source": [
        "df[\"agg_col1\"] = [1 if x % 2 == 0 else 0 for x in range(len(df))]\n",
        "df[\"agg_col2\"] = [1 if x % 3 == 0 else 0 for x in range(len(df))]\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "8e2f0961",
      "metadata": {
        "id": "8e2f0961"
      },
      "source": [
        "Grouping and then applying the `sum` function to the grouped data."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "id": "e8a7f1f9",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 143
        },
        "id": "e8a7f1f9",
        "outputId": "6de09c35-52f0-4d4e-9d71-61a1120b2120"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "            a    b    c  agg_col2\n",
              "agg_col1                         \n",
              "1          90  100   90         4\n",
              "0         100   90  100         3"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>a</th>\n",
              "      <th>b</th>\n",
              "      <th>c</th>\n",
              "      <th>agg_col2</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>agg_col1</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>90</td>\n",
              "      <td>100</td>\n",
              "      <td>90</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>100</td>\n",
              "      <td>90</td>\n",
              "      <td>100</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 31
        }
      ],
      "source": [
        "df.groupby(\"agg_col1\").sum()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5ff1e8bf",
      "metadata": {
        "id": "5ff1e8bf"
      },
      "source": [
        "Grouping hierarchically then applying the `sum` function to grouped data."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 32,
      "id": "4738f0ef",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "4738f0ef",
        "outputId": "17fdfafe-88dd-46e3-8120-4dbc5f5b548a"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                    a   b   c\n",
              "agg_col1 agg_col2            \n",
              "1        1         36  40  36\n",
              "         0         54  60  54\n",
              "0        1         27  30  27\n",
              "         0         73  60  73"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th>a</th>\n",
              "      <th>b</th>\n",
              "      <th>c</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>agg_col1</th>\n",
              "      <th>agg_col2</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th rowspan=\"2\" valign=\"top\">1</th>\n",
              "      <th>1</th>\n",
              "      <td>36</td>\n",
              "      <td>40</td>\n",
              "      <td>36</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>54</td>\n",
              "      <td>60</td>\n",
              "      <td>54</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th rowspan=\"2\" valign=\"top\">0</th>\n",
              "      <th>1</th>\n",
              "      <td>27</td>\n",
              "      <td>30</td>\n",
              "      <td>27</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>73</td>\n",
              "      <td>60</td>\n",
              "      <td>73</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 32
        }
      ],
      "source": [
        "df.groupby([\"agg_col1\", \"agg_col2\"]).sum()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "443e179b",
      "metadata": {
        "id": "443e179b"
      },
      "source": [
        "Grouping and applying statistical functions to specific columns, using `agg`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 33,
      "id": "f196ad8b",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 143
        },
        "id": "f196ad8b",
        "outputId": "ff31c631-ec55-4ec9-e8ce-57b289dff2b6"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "           a     b    c\n",
              "agg_col1               \n",
              "1         18  10.0   90\n",
              "0         19   9.0  100"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>a</th>\n",
              "      <th>b</th>\n",
              "      <th>c</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>agg_col1</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>18</td>\n",
              "      <td>10.0</td>\n",
              "      <td>90</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>19</td>\n",
              "      <td>9.0</td>\n",
              "      <td>100</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 33
        }
      ],
      "source": [
        "df.groupby(\"agg_col1\").agg({\"a\": \"max\", \"b\": \"mean\", \"c\": \"sum\"})"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "SHQBiLOSBEz3",
      "metadata": {
        "id": "SHQBiLOSBEz3"
      },
      "source": [
        "## Sorting"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "16798a32",
      "metadata": {
        "id": "16798a32"
      },
      "source": [
        "Sorting by values."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 34,
      "id": "2190856d",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 676
        },
        "id": "2190856d",
        "outputId": "bd0f5dd2-706b-4342-f364-6054645238be"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "     a   b   c  agg_col1  agg_col2\n",
              "19  19   0  19         0         0\n",
              "18  18   1  18         1         1\n",
              "17  17   2  17         0         0\n",
              "16  16   3  16         1         0\n",
              "15  15   4  15         0         1\n",
              "14  14   5  14         1         0\n",
              "13  13   6  13         0         0\n",
              "12  12   7  12         1         1\n",
              "11  11   8  11         0         0\n",
              "10  10   9  10         1         0\n",
              "9    9  10   9         0         1\n",
              "8    8  11   8         1         0\n",
              "7    7  12   7         0         0\n",
              "6    6  13   6         1         1\n",
              "5    5  14   5         0         0\n",
              "4    4  15   4         1         0\n",
              "3    3  16   3         0         1\n",
              "2    2  17   2         1         0\n",
              "1    1  18   1         0         0\n",
              "0    0  19   0         1         1"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>a</th>\n",
              "      <th>b</th>\n",
              "      <th>c</th>\n",
              "      <th>agg_col1</th>\n",
              "      <th>agg_col2</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>19</th>\n",
              "      <td>19</td>\n",
              "      <td>0</td>\n",
              "      <td>19</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>18</th>\n",
              "      <td>18</td>\n",
              "      <td>1</td>\n",
              "      <td>18</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17</th>\n",
              "      <td>17</td>\n",
              "      <td>2</td>\n",
              "      <td>17</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>16</td>\n",
              "      <td>3</td>\n",
              "      <td>16</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>15</td>\n",
              "      <td>4</td>\n",
              "      <td>15</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>14</td>\n",
              "      <td>5</td>\n",
              "      <td>14</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13</th>\n",
              "      <td>13</td>\n",
              "      <td>6</td>\n",
              "      <td>13</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12</th>\n",
              "      <td>12</td>\n",
              "      <td>7</td>\n",
              "      <td>12</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>11</td>\n",
              "      <td>8</td>\n",
              "      <td>11</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>10</td>\n",
              "      <td>9</td>\n",
              "      <td>10</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>9</td>\n",
              "      <td>10</td>\n",
              "      <td>9</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>8</td>\n",
              "      <td>11</td>\n",
              "      <td>8</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>7</td>\n",
              "      <td>12</td>\n",
              "      <td>7</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>6</td>\n",
              "      <td>13</td>\n",
              "      <td>6</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>5</td>\n",
              "      <td>14</td>\n",
              "      <td>5</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>4</td>\n",
              "      <td>15</td>\n",
              "      <td>4</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3</td>\n",
              "      <td>16</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "      <td>17</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>18</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>19</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 34
        }
      ],
      "source": [
        "df.sort_values(by=\"b\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "a5bf30e1",
      "metadata": {
        "id": "a5bf30e1"
      },
      "source": [
        "## Transpose"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5ac3b004",
      "metadata": {
        "id": "5ac3b004"
      },
      "source": [
        "Transposing a dataframe, using either the `transpose` method or `T` property. Currently, all columns must have the same type. Transposing is not currently implemented in Dask-cuDF."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 35,
      "id": "c5fbdb50",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 143
        },
        "id": "c5fbdb50",
        "outputId": "b9d7c400-aec4-4f53-b122-6080b7c6a9e4"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   a  b\n",
              "0  1  4\n",
              "1  2  5\n",
              "2  3  6"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>a</th>\n",
              "      <th>b</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>5</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>6</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 35
        }
      ],
      "source": [
        "sample = cudf.DataFrame({\"a\": [1, 2, 3], \"b\": [4, 5, 6]})\n",
        "sample"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 36,
      "id": "733ed90c",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 112
        },
        "id": "733ed90c",
        "outputId": "8ca47dae-557b-4a37-c8ab-dfb8e5a4244a"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   0  1  2\n",
              "a  1  2  3\n",
              "b  4  5  6"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>0</th>\n",
              "      <th>1</th>\n",
              "      <th>2</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>a</th>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>b</th>\n",
              "      <td>4</td>\n",
              "      <td>5</td>\n",
              "      <td>6</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 36
        }
      ],
      "source": [
        "sample.transpose()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "e0915c46",
      "metadata": {
        "id": "e0915c46"
      },
      "source": [
        "Time Series\n",
        "------------"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ec7f0b81",
      "metadata": {
        "id": "ec7f0b81"
      },
      "source": [
        "`DataFrames` supports `datetime` typed columns, which allow users to interact with and filter data based on specific timestamps."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 37,
      "id": "a6d45607",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 175
        },
        "id": "a6d45607",
        "outputId": "2c5d4413-7091-47ec-a352-7b284070e7f4"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "        date     value\n",
              "0 2018-11-20  0.385556\n",
              "1 2018-11-21  0.908215\n",
              "2 2018-11-22  0.641620\n",
              "3 2018-11-23  0.283399"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>date</th>\n",
              "      <th>value</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>2018-11-20</td>\n",
              "      <td>0.385556</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2018-11-21</td>\n",
              "      <td>0.908215</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2018-11-22</td>\n",
              "      <td>0.641620</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2018-11-23</td>\n",
              "      <td>0.283399</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 37
        }
      ],
      "source": [
        "import datetime as dt\n",
        "\n",
        "date_df = cudf.DataFrame()\n",
        "date_df[\"date\"] = pd.date_range(\"11/20/2018\", periods=72, freq=\"D\")\n",
        "date_df[\"value\"] = cp.random.sample(len(date_df))\n",
        "\n",
        "search_date = dt.datetime.strptime(\"2018-11-23\", \"%Y-%m-%d\")\n",
        "date_df.query(\"date <= @search_date\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "45f9408b",
      "metadata": {
        "id": "45f9408b"
      },
      "source": [
        "Categoricals\n",
        "------------"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5eb96f98",
      "metadata": {
        "id": "5eb96f98"
      },
      "source": [
        "`DataFrames` support categorical columns."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 38,
      "id": "d735b5cb",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 237
        },
        "id": "d735b5cb",
        "outputId": "c66d2072-9f6e-4023-bde4-64bf9dbba81f"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   id grade\n",
              "0   1     a\n",
              "1   2     b\n",
              "2   3     b\n",
              "3   4     a\n",
              "4   5     a\n",
              "5   6     e"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>id</th>\n",
              "      <th>grade</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>a</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>b</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>b</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>a</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>a</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>6</td>\n",
              "      <td>e</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 38
        }
      ],
      "source": [
        "gdf = cudf.DataFrame(\n",
        "    {\"id\": [1, 2, 3, 4, 5, 6], \"grade\": [\"a\", \"b\", \"b\", \"a\", \"a\", \"e\"]}\n",
        ")\n",
        "gdf[\"grade\"] = gdf[\"grade\"].astype(\"category\")\n",
        "gdf"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "a9c2bcac",
      "metadata": {
        "id": "a9c2bcac"
      },
      "source": [
        "Accessing the categories of a column. Note that this is currently not supported in Dask-cuDF."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 39,
      "id": "a7135eda",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "a7135eda",
        "outputId": "9008d9cf-2c2c-4bf1-cdcf-00ab36a90674"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Index(['a', 'b', 'e'], dtype='object')"
            ]
          },
          "metadata": {},
          "execution_count": 39
        }
      ],
      "source": [
        "gdf.grade.cat.categories"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "466e1ed2",
      "metadata": {
        "id": "466e1ed2"
      },
      "source": [
        "Accessing the underlying code values of each categorical observation."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 40,
      "id": "f00c615a",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "f00c615a",
        "outputId": "574e4d06-21c1-44f4-f010-040c34e20f0c"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0    0\n",
              "1    1\n",
              "2    1\n",
              "3    0\n",
              "4    0\n",
              "5    2\n",
              "dtype: uint8"
            ]
          },
          "metadata": {},
          "execution_count": 40
        }
      ],
      "source": [
        "gdf.grade.cat.codes"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "4pr1V5J6_uZd",
      "metadata": {
        "id": "4pr1V5J6_uZd"
      },
      "source": [
        "# Converting Data Representation"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "1b391a0d",
      "metadata": {
        "id": "1b391a0d"
      },
      "source": [
        "## Pandas"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "cfdd172b",
      "metadata": {
        "id": "cfdd172b"
      },
      "source": [
        "Converting a cuDF `DataFrame` to a pandas `DataFrame`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 41,
      "id": "1fcd9c7f",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "1fcd9c7f",
        "outputId": "5de3a0a0-2133-4ab8-c4c9-73add9909a3b"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   a   b  c  agg_col1  agg_col2\n",
              "0  0  19  0         1         1\n",
              "1  1  18  1         0         0\n",
              "2  2  17  2         1         0\n",
              "3  3  16  3         0         1\n",
              "4  4  15  4         1         0"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-d59573ec-1bfa-4bd1-8fb7-2d229147ec5d\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>a</th>\n",
              "      <th>b</th>\n",
              "      <th>c</th>\n",
              "      <th>agg_col1</th>\n",
              "      <th>agg_col2</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>19</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>18</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "      <td>17</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3</td>\n",
              "      <td>16</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>4</td>\n",
              "      <td>15</td>\n",
              "      <td>4</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-d59573ec-1bfa-4bd1-8fb7-2d229147ec5d')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-d59573ec-1bfa-4bd1-8fb7-2d229147ec5d button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-d59573ec-1bfa-4bd1-8fb7-2d229147ec5d');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-e5def174-d2f5-4d76-87f1-bf57b628d9c3\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-e5def174-d2f5-4d76-87f1-bf57b628d9c3')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-e5def174-d2f5-4d76-87f1-bf57b628d9c3 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "summary": "{\n  \"name\": \"df\",\n  \"rows\": 5,\n  \"fields\": [\n    {\n      \"column\": \"a\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 1,\n        \"min\": 0,\n        \"max\": 4,\n        \"num_unique_values\": 5,\n        \"samples\": [\n          1,\n          4,\n          2\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"b\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 1,\n        \"min\": 15,\n        \"max\": 19,\n        \"num_unique_values\": 5,\n        \"samples\": [\n          18,\n          15,\n          17\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"c\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 1,\n        \"min\": 0,\n        \"max\": 4,\n        \"num_unique_values\": 5,\n        \"samples\": [\n          1,\n          4,\n          2\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"agg_col1\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0,\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"agg_col2\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0,\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 41
        }
      ],
      "source": [
        "df.head().to_pandas()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "a104294a",
      "metadata": {
        "id": "a104294a"
      },
      "source": [
        "## Numpy"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "c5d3e508",
      "metadata": {
        "id": "c5d3e508"
      },
      "source": [
        "Converting a cuDF or Dask-cuDF `DataFrame` to a numpy `ndarray`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 42,
      "id": "2948b577",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2948b577",
        "outputId": "dbf3ede0-e925-4df5-dab7-5ec45962431f"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 0, 19,  0,  1,  1],\n",
              "       [ 1, 18,  1,  0,  0],\n",
              "       [ 2, 17,  2,  1,  0],\n",
              "       [ 3, 16,  3,  0,  1],\n",
              "       [ 4, 15,  4,  1,  0],\n",
              "       [ 5, 14,  5,  0,  0],\n",
              "       [ 6, 13,  6,  1,  1],\n",
              "       [ 7, 12,  7,  0,  0],\n",
              "       [ 8, 11,  8,  1,  0],\n",
              "       [ 9, 10,  9,  0,  1],\n",
              "       [10,  9, 10,  1,  0],\n",
              "       [11,  8, 11,  0,  0],\n",
              "       [12,  7, 12,  1,  1],\n",
              "       [13,  6, 13,  0,  0],\n",
              "       [14,  5, 14,  1,  0],\n",
              "       [15,  4, 15,  0,  1],\n",
              "       [16,  3, 16,  1,  0],\n",
              "       [17,  2, 17,  0,  0],\n",
              "       [18,  1, 18,  1,  1],\n",
              "       [19,  0, 19,  0,  0]])"
            ]
          },
          "metadata": {},
          "execution_count": 42
        }
      ],
      "source": [
        "df.to_numpy()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "c1f09303",
      "metadata": {
        "id": "c1f09303"
      },
      "source": [
        "Converting a cuDF or Dask-cuDF `Series` to a numpy `ndarray`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 43,
      "id": "997c89ba",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "997c89ba",
        "outputId": "93fbbedf-efb2-465c-de19-c6dcfcf948a1"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16,\n",
              "       17, 18, 19])"
            ]
          },
          "metadata": {},
          "execution_count": 43
        }
      ],
      "source": [
        "df[\"a\"].to_numpy()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "b520acf7",
      "metadata": {
        "id": "b520acf7"
      },
      "source": [
        "## Arrow"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "050e67e5",
      "metadata": {
        "id": "050e67e5"
      },
      "source": [
        "Converting a cuDF or Dask-cuDF `DataFrame` to a PyArrow `Table`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 44,
      "id": "0ac9e740",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0ac9e740",
        "outputId": "7ece4322-d605-40ce-b1aa-7fa1dc5af5bd"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "pyarrow.Table\n",
              "a: int64\n",
              "b: int64\n",
              "c: int64\n",
              "agg_col1: int64\n",
              "agg_col2: int64\n",
              "----\n",
              "a: [[0,1,2,3,4,...,15,16,17,18,19]]\n",
              "b: [[19,18,17,16,15,...,4,3,2,1,0]]\n",
              "c: [[0,1,2,3,4,...,15,16,17,18,19]]\n",
              "agg_col1: [[1,0,1,0,1,...,0,1,0,1,0]]\n",
              "agg_col2: [[1,0,0,1,0,...,1,0,0,1,0]]"
            ]
          },
          "metadata": {},
          "execution_count": 44
        }
      ],
      "source": [
        "df.to_arrow()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "f0734690",
      "metadata": {
        "id": "f0734690"
      },
      "source": [
        "# Reading and Writing Data"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "6f0251c6",
      "metadata": {
        "id": "6f0251c6"
      },
      "source": [
        "## CSV"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "2d1935c6",
      "metadata": {
        "id": "2d1935c6"
      },
      "source": [
        "Writing to a CSV file."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 45,
      "id": "36f5039f",
      "metadata": {
        "id": "36f5039f"
      },
      "outputs": [],
      "source": [
        "if not os.path.exists(\"example_output\"):\n",
        "    os.mkdir(\"example_output\")\n",
        "\n",
        "df.to_csv(\"example_output/foo.csv\", index=False)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "320c3968",
      "metadata": {
        "id": "320c3968"
      },
      "source": [
        "Reading from a csv file."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 46,
      "id": "c110a80f",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 676
        },
        "id": "c110a80f",
        "outputId": "93aa58a0-612d-4043-8195-3f4a2d36cb93"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "     a   b   c  agg_col1  agg_col2\n",
              "0    0  19   0         1         1\n",
              "1    1  18   1         0         0\n",
              "2    2  17   2         1         0\n",
              "3    3  16   3         0         1\n",
              "4    4  15   4         1         0\n",
              "5    5  14   5         0         0\n",
              "6    6  13   6         1         1\n",
              "7    7  12   7         0         0\n",
              "8    8  11   8         1         0\n",
              "9    9  10   9         0         1\n",
              "10  10   9  10         1         0\n",
              "11  11   8  11         0         0\n",
              "12  12   7  12         1         1\n",
              "13  13   6  13         0         0\n",
              "14  14   5  14         1         0\n",
              "15  15   4  15         0         1\n",
              "16  16   3  16         1         0\n",
              "17  17   2  17         0         0\n",
              "18  18   1  18         1         1\n",
              "19  19   0  19         0         0"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>a</th>\n",
              "      <th>b</th>\n",
              "      <th>c</th>\n",
              "      <th>agg_col1</th>\n",
              "      <th>agg_col2</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>19</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>18</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "      <td>17</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3</td>\n",
              "      <td>16</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>4</td>\n",
              "      <td>15</td>\n",
              "      <td>4</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>5</td>\n",
              "      <td>14</td>\n",
              "      <td>5</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>6</td>\n",
              "      <td>13</td>\n",
              "      <td>6</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>7</td>\n",
              "      <td>12</td>\n",
              "      <td>7</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>8</td>\n",
              "      <td>11</td>\n",
              "      <td>8</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>9</td>\n",
              "      <td>10</td>\n",
              "      <td>9</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>10</td>\n",
              "      <td>9</td>\n",
              "      <td>10</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>11</td>\n",
              "      <td>8</td>\n",
              "      <td>11</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12</th>\n",
              "      <td>12</td>\n",
              "      <td>7</td>\n",
              "      <td>12</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13</th>\n",
              "      <td>13</td>\n",
              "      <td>6</td>\n",
              "      <td>13</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>14</td>\n",
              "      <td>5</td>\n",
              "      <td>14</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>15</td>\n",
              "      <td>4</td>\n",
              "      <td>15</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>16</td>\n",
              "      <td>3</td>\n",
              "      <td>16</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17</th>\n",
              "      <td>17</td>\n",
              "      <td>2</td>\n",
              "      <td>17</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>18</th>\n",
              "      <td>18</td>\n",
              "      <td>1</td>\n",
              "      <td>18</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19</th>\n",
              "      <td>19</td>\n",
              "      <td>0</td>\n",
              "      <td>19</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 46
        }
      ],
      "source": [
        "df = cudf.read_csv(\"example_output/foo.csv\")\n",
        "df"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "787eae14",
      "metadata": {
        "id": "787eae14"
      },
      "source": [
        "Note that for the dask-cuDF case, we use `dask_cudf.read_csv` in preference to `dask_cudf.from_cudf(cudf.read_csv)` since the former can parallelize across multiple GPUs and handle larger CSV files that would fit in memory on a single GPU."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "72857b2c",
      "metadata": {
        "id": "72857b2c"
      },
      "source": [
        "Reading all CSV files in a directory into a single `dask_cudf.DataFrame`, using the star wildcard."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "763c555b",
      "metadata": {
        "id": "763c555b"
      },
      "source": [
        "## Parquet"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "8766d4ac",
      "metadata": {
        "id": "8766d4ac"
      },
      "source": [
        "Writing to parquet files with cuDF's GPU-accelerated parquet writer"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 47,
      "id": "5038b284",
      "metadata": {
        "id": "5038b284"
      },
      "outputs": [],
      "source": [
        "df.to_parquet(\"example_output/temp_parquet\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "b4b49824",
      "metadata": {
        "id": "b4b49824"
      },
      "source": [
        "Reading parquet files with cuDF's GPU-accelerated parquet reader."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 48,
      "id": "bb657a69",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 676
        },
        "id": "bb657a69",
        "outputId": "02ceddfb-4246-4a86-937c-2a6ba85dbc69"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "     a   b   c  agg_col1  agg_col2\n",
              "0    0  19   0         1         1\n",
              "1    1  18   1         0         0\n",
              "2    2  17   2         1         0\n",
              "3    3  16   3         0         1\n",
              "4    4  15   4         1         0\n",
              "5    5  14   5         0         0\n",
              "6    6  13   6         1         1\n",
              "7    7  12   7         0         0\n",
              "8    8  11   8         1         0\n",
              "9    9  10   9         0         1\n",
              "10  10   9  10         1         0\n",
              "11  11   8  11         0         0\n",
              "12  12   7  12         1         1\n",
              "13  13   6  13         0         0\n",
              "14  14   5  14         1         0\n",
              "15  15   4  15         0         1\n",
              "16  16   3  16         1         0\n",
              "17  17   2  17         0         0\n",
              "18  18   1  18         1         1\n",
              "19  19   0  19         0         0"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>a</th>\n",
              "      <th>b</th>\n",
              "      <th>c</th>\n",
              "      <th>agg_col1</th>\n",
              "      <th>agg_col2</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>19</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>18</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "      <td>17</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3</td>\n",
              "      <td>16</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>4</td>\n",
              "      <td>15</td>\n",
              "      <td>4</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>5</td>\n",
              "      <td>14</td>\n",
              "      <td>5</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>6</td>\n",
              "      <td>13</td>\n",
              "      <td>6</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>7</td>\n",
              "      <td>12</td>\n",
              "      <td>7</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>8</td>\n",
              "      <td>11</td>\n",
              "      <td>8</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>9</td>\n",
              "      <td>10</td>\n",
              "      <td>9</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>10</td>\n",
              "      <td>9</td>\n",
              "      <td>10</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>11</td>\n",
              "      <td>8</td>\n",
              "      <td>11</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12</th>\n",
              "      <td>12</td>\n",
              "      <td>7</td>\n",
              "      <td>12</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13</th>\n",
              "      <td>13</td>\n",
              "      <td>6</td>\n",
              "      <td>13</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>14</td>\n",
              "      <td>5</td>\n",
              "      <td>14</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>15</td>\n",
              "      <td>4</td>\n",
              "      <td>15</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>16</td>\n",
              "      <td>3</td>\n",
              "      <td>16</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17</th>\n",
              "      <td>17</td>\n",
              "      <td>2</td>\n",
              "      <td>17</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>18</th>\n",
              "      <td>18</td>\n",
              "      <td>1</td>\n",
              "      <td>18</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19</th>\n",
              "      <td>19</td>\n",
              "      <td>0</td>\n",
              "      <td>19</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 48
        }
      ],
      "source": [
        "df = cudf.read_parquet(\"example_output/temp_parquet\")\n",
        "df"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "90a49967",
      "metadata": {
        "id": "90a49967"
      },
      "source": [
        "## ORC"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "de9d03fa",
      "metadata": {
        "id": "de9d03fa"
      },
      "source": [
        "Writing ORC files."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 49,
      "id": "c387f8f2",
      "metadata": {
        "id": "c387f8f2"
      },
      "outputs": [],
      "source": [
        "df.to_orc(\"example_output/temp_orc\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "242c32a2",
      "metadata": {
        "id": "242c32a2"
      },
      "source": [
        "And reading"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 50,
      "id": "d4bab6da",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 676
        },
        "id": "d4bab6da",
        "outputId": "636cd62b-e53c-440d-e264-b2d37ee2723b"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "     a   b   c  agg_col1  agg_col2\n",
              "0    0  19   0         1         1\n",
              "1    1  18   1         0         0\n",
              "2    2  17   2         1         0\n",
              "3    3  16   3         0         1\n",
              "4    4  15   4         1         0\n",
              "5    5  14   5         0         0\n",
              "6    6  13   6         1         1\n",
              "7    7  12   7         0         0\n",
              "8    8  11   8         1         0\n",
              "9    9  10   9         0         1\n",
              "10  10   9  10         1         0\n",
              "11  11   8  11         0         0\n",
              "12  12   7  12         1         1\n",
              "13  13   6  13         0         0\n",
              "14  14   5  14         1         0\n",
              "15  15   4  15         0         1\n",
              "16  16   3  16         1         0\n",
              "17  17   2  17         0         0\n",
              "18  18   1  18         1         1\n",
              "19  19   0  19         0         0"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>a</th>\n",
              "      <th>b</th>\n",
              "      <th>c</th>\n",
              "      <th>agg_col1</th>\n",
              "      <th>agg_col2</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>19</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>18</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "      <td>17</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3</td>\n",
              "      <td>16</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>4</td>\n",
              "      <td>15</td>\n",
              "      <td>4</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>5</td>\n",
              "      <td>14</td>\n",
              "      <td>5</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>6</td>\n",
              "      <td>13</td>\n",
              "      <td>6</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>7</td>\n",
              "      <td>12</td>\n",
              "      <td>7</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>8</td>\n",
              "      <td>11</td>\n",
              "      <td>8</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>9</td>\n",
              "      <td>10</td>\n",
              "      <td>9</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>10</td>\n",
              "      <td>9</td>\n",
              "      <td>10</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>11</td>\n",
              "      <td>8</td>\n",
              "      <td>11</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12</th>\n",
              "      <td>12</td>\n",
              "      <td>7</td>\n",
              "      <td>12</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13</th>\n",
              "      <td>13</td>\n",
              "      <td>6</td>\n",
              "      <td>13</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>14</td>\n",
              "      <td>5</td>\n",
              "      <td>14</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>15</td>\n",
              "      <td>4</td>\n",
              "      <td>15</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>16</td>\n",
              "      <td>3</td>\n",
              "      <td>16</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17</th>\n",
              "      <td>17</td>\n",
              "      <td>2</td>\n",
              "      <td>17</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>18</th>\n",
              "      <td>18</td>\n",
              "      <td>1</td>\n",
              "      <td>18</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19</th>\n",
              "      <td>19</td>\n",
              "      <td>0</td>\n",
              "      <td>19</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 50
        }
      ],
      "source": [
        "df2 = cudf.read_orc(\"example_output/temp_orc\")\n",
        "df2"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "oP0LQP96wqst",
      "metadata": {
        "id": "oP0LQP96wqst"
      },
      "source": [
        "# Performance with large dataframes"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "XKjgbxurw_rw",
      "metadata": {
        "id": "XKjgbxurw_rw"
      },
      "source": [
        "cuDF is great for handling large dataframes. In this example we aggregate values after grouping by a key:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 51,
      "id": "uA4yr08mwznw",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "uA4yr08mwznw",
        "outputId": "5f8a15c9-26dd-4122-ae26-60d402dd2f2f"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 42.4 ms, sys: 7.87 ms, total: 50.2 ms\n",
            "Wall time: 134 ms\n",
            "CPU times: user 2.46 s, sys: 368 ms, total: 2.83 s\n",
            "Wall time: 4.03 s\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "key\n",
              "0    0.500149\n",
              "1    0.499845\n",
              "2    0.499982\n",
              "3    0.499855\n",
              "4    0.499957\n",
              "5    0.499990\n",
              "6    0.500085\n",
              "7    0.499953\n",
              "8    0.500066\n",
              "9    0.500020\n",
              "Name: value, dtype: float64"
            ]
          },
          "metadata": {},
          "execution_count": 51
        }
      ],
      "source": [
        "nr = 100_000_000\n",
        "df = cudf.DataFrame({\n",
        "    'key': cp.random.randint(0, 10, nr),\n",
        "    'value': cp.random.random(nr)\n",
        "})\n",
        "\n",
        "%time df.groupby('key')['value'].mean()\n",
        "\n",
        "pdf = df.to_pandas()\n",
        "%time pdf.groupby('key')['value'].mean()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "w85kgVvBykvW",
      "metadata": {
        "id": "w85kgVvBykvW"
      },
      "source": [
        "cuDF also has efficient join algorithms. In this example we use a hash join to combine values from two dataframes based on a key:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 52,
      "id": "sHQk7h4XxM2w",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 494
        },
        "id": "sHQk7h4XxM2w",
        "outputId": "aa795fdf-7ac1-4ba9-8fcf-f5411759ffb2"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 109 ms, sys: 17 ms, total: 125 ms\n",
            "Wall time: 150 ms\n",
            "CPU times: user 3.03 s, sys: 1.08 s, total: 4.1 s\n",
            "Wall time: 4.51 s\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "          key     value    lookup\n",
              "0           6  0.576150  0.086387\n",
              "1           6  0.423739  0.086387\n",
              "2           6  0.884748  0.086387\n",
              "3           6  0.788128  0.086387\n",
              "4           6  0.800403  0.086387\n",
              "...       ...       ...       ...\n",
              "49999995    9  0.382200  0.262745\n",
              "49999996    9  0.293663  0.262745\n",
              "49999997    9  0.425066  0.262745\n",
              "49999998    9  0.432143  0.262745\n",
              "49999999    9  0.062610  0.262745\n",
              "\n",
              "[50000000 rows x 3 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-41a29f72-b1c4-4120-9739-26a8b492cc71\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>key</th>\n",
              "      <th>value</th>\n",
              "      <th>lookup</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>6</td>\n",
              "      <td>0.576150</td>\n",
              "      <td>0.086387</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>6</td>\n",
              "      <td>0.423739</td>\n",
              "      <td>0.086387</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>6</td>\n",
              "      <td>0.884748</td>\n",
              "      <td>0.086387</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>6</td>\n",
              "      <td>0.788128</td>\n",
              "      <td>0.086387</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>6</td>\n",
              "      <td>0.800403</td>\n",
              "      <td>0.086387</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49999995</th>\n",
              "      <td>9</td>\n",
              "      <td>0.382200</td>\n",
              "      <td>0.262745</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49999996</th>\n",
              "      <td>9</td>\n",
              "      <td>0.293663</td>\n",
              "      <td>0.262745</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49999997</th>\n",
              "      <td>9</td>\n",
              "      <td>0.425066</td>\n",
              "      <td>0.262745</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49999998</th>\n",
              "      <td>9</td>\n",
              "      <td>0.432143</td>\n",
              "      <td>0.262745</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49999999</th>\n",
              "      <td>9</td>\n",
              "      <td>0.062610</td>\n",
              "      <td>0.262745</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>50000000 rows × 3 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-41a29f72-b1c4-4120-9739-26a8b492cc71')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-41a29f72-b1c4-4120-9739-26a8b492cc71 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-41a29f72-b1c4-4120-9739-26a8b492cc71');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-9dc271e1-1682-499a-879d-e531393ef4d3\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-9dc271e1-1682-499a-879d-e531393ef4d3')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-9dc271e1-1682-499a-879d-e531393ef4d3 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe"
            }
          },
          "metadata": {},
          "execution_count": 52
        }
      ],
      "source": [
        "nr = 50_000_000\n",
        "df = cudf.DataFrame({\n",
        "        'key': cp.random.randint(0, 10, nr),\n",
        "        'value': cp.random.random(nr)\n",
        "})\n",
        "lookup = cudf.DataFrame({\n",
        "        'key': range(10),\n",
        "        'lookup': cp.random.random(10)\n",
        "})\n",
        "\n",
        "%time df.merge(lookup, on='key')\n",
        "\n",
        "pdf = df.to_pandas()\n",
        "plookup = lookup.to_pandas()\n",
        "%time pdf.merge(plookup, on='key')"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "lakM4rOf-oRt",
      "metadata": {
        "id": "lakM4rOf-oRt"
      },
      "source": [
        "Computing and applying filters to cuDF dataframes are also efficient operations."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 53,
      "id": "o12Z8bhbyVRX",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 494
        },
        "id": "o12Z8bhbyVRX",
        "outputId": "f878d5b5-83d4-47b1-ad46-086c97b10838"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 21.9 ms, sys: 6.78 ms, total: 28.7 ms\n",
            "Wall time: 37 ms\n",
            "CPU times: user 676 ms, sys: 329 ms, total: 1 s\n",
            "Wall time: 901 ms\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "          rating_a  rating_b  rating_c\n",
              "0              4.0       4.0       NaN\n",
              "1              NaN       NaN       NaN\n",
              "2              NaN       NaN       NaN\n",
              "3              NaN       NaN       4.0\n",
              "4              NaN       NaN       NaN\n",
              "...            ...       ...       ...\n",
              "19999995       NaN       NaN       4.0\n",
              "19999996       NaN       NaN       NaN\n",
              "19999997       NaN       4.0       4.0\n",
              "19999998       NaN       4.0       4.0\n",
              "19999999       NaN       NaN       NaN\n",
              "\n",
              "[20000000 rows x 3 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-03095eaf-6fcc-4db1-a357-13f2a0f113b4\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>rating_a</th>\n",
              "      <th>rating_b</th>\n",
              "      <th>rating_c</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>4.0</td>\n",
              "      <td>4.0</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>4.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19999995</th>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>4.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19999996</th>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19999997</th>\n",
              "      <td>NaN</td>\n",
              "      <td>4.0</td>\n",
              "      <td>4.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19999998</th>\n",
              "      <td>NaN</td>\n",
              "      <td>4.0</td>\n",
              "      <td>4.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19999999</th>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>20000000 rows × 3 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-03095eaf-6fcc-4db1-a357-13f2a0f113b4')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-03095eaf-6fcc-4db1-a357-13f2a0f113b4 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-03095eaf-6fcc-4db1-a357-13f2a0f113b4');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-c8b3197c-311c-4d80-af3e-7957e172f83f\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-c8b3197c-311c-4d80-af3e-7957e172f83f')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-c8b3197c-311c-4d80-af3e-7957e172f83f button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe"
            }
          },
          "metadata": {},
          "execution_count": 53
        }
      ],
      "source": [
        "nr = 20_000_000\n",
        "df = cudf.DataFrame({\n",
        "    'rating_a': cp.random.randint(1, 5, nr),\n",
        "    'rating_b': cp.random.randint(1, 5, nr),\n",
        "    'rating_c': cp.random.randint(1, 5, nr),\n",
        "})\n",
        "\n",
        "%time df.where(df>3)\n",
        "\n",
        "pdf = df.to_pandas()\n",
        "%time pdf.where(pdf>3)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "Jho2FyMP-2FZ",
      "metadata": {
        "id": "Jho2FyMP-2FZ"
      },
      "source": [
        "And sorting is another task where cuDF shows great acceleration."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 54,
      "id": "u9SmpVLp8kJx",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 494
        },
        "id": "u9SmpVLp8kJx",
        "outputId": "9972be20-abb8-4169-b9cc-73f274bd3cd0"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 135 ms, sys: 7.64 ms, total: 143 ms\n",
            "Wall time: 148 ms\n",
            "CPU times: user 2.44 s, sys: 191 ms, total: 2.63 s\n",
            "Wall time: 2.61 s\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                    a         b         c\n",
              "7327996  5.919987e-08  0.103876  0.362604\n",
              "2624195  4.288834e-07  0.952080  0.597901\n",
              "7367962  6.171877e-07  0.592560  0.187291\n",
              "5549822  7.305300e-07  0.384424  0.512523\n",
              "4596785  7.623830e-07  0.084830  0.753644\n",
              "...               ...       ...       ...\n",
              "1258033  9.999995e-01  0.267930  0.605742\n",
              "7004706  9.999996e-01  0.300102  0.418403\n",
              "1955702  9.999998e-01  0.428556  0.238820\n",
              "7634263  9.999999e-01  0.025396  0.385628\n",
              "6875674  1.000000e+00  0.257373  0.649026\n",
              "\n",
              "[10000000 rows x 3 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-632a4e22-4229-405d-bb1c-31ab0afca997\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>a</th>\n",
              "      <th>b</th>\n",
              "      <th>c</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>7327996</th>\n",
              "      <td>5.919987e-08</td>\n",
              "      <td>0.103876</td>\n",
              "      <td>0.362604</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2624195</th>\n",
              "      <td>4.288834e-07</td>\n",
              "      <td>0.952080</td>\n",
              "      <td>0.597901</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7367962</th>\n",
              "      <td>6.171877e-07</td>\n",
              "      <td>0.592560</td>\n",
              "      <td>0.187291</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5549822</th>\n",
              "      <td>7.305300e-07</td>\n",
              "      <td>0.384424</td>\n",
              "      <td>0.512523</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4596785</th>\n",
              "      <td>7.623830e-07</td>\n",
              "      <td>0.084830</td>\n",
              "      <td>0.753644</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1258033</th>\n",
              "      <td>9.999995e-01</td>\n",
              "      <td>0.267930</td>\n",
              "      <td>0.605742</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7004706</th>\n",
              "      <td>9.999996e-01</td>\n",
              "      <td>0.300102</td>\n",
              "      <td>0.418403</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1955702</th>\n",
              "      <td>9.999998e-01</td>\n",
              "      <td>0.428556</td>\n",
              "      <td>0.238820</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7634263</th>\n",
              "      <td>9.999999e-01</td>\n",
              "      <td>0.025396</td>\n",
              "      <td>0.385628</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6875674</th>\n",
              "      <td>1.000000e+00</td>\n",
              "      <td>0.257373</td>\n",
              "      <td>0.649026</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>10000000 rows × 3 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-632a4e22-4229-405d-bb1c-31ab0afca997')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-632a4e22-4229-405d-bb1c-31ab0afca997 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-632a4e22-4229-405d-bb1c-31ab0afca997');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-c4da05da-8424-4477-b4aa-5add92eec10d\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-c4da05da-8424-4477-b4aa-5add92eec10d')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-c4da05da-8424-4477-b4aa-5add92eec10d button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe"
            }
          },
          "metadata": {},
          "execution_count": 54
        }
      ],
      "source": [
        "nr = 10_000_000\n",
        "df = cudf.DataFrame({\n",
        "    'a': cp.random.rand(nr),\n",
        "    'b': cp.random.rand(nr),\n",
        "    'c': cp.random.rand(nr),\n",
        "})\n",
        "\n",
        "%time df.sort_values('a')\n",
        "\n",
        "pdf = df.to_pandas()\n",
        "%time pdf.sort_values('a')"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "provenance": []
    },
    "gpuClass": "standard",
    "kernelspec": {
      "display_name": "Python 3.9.12 ('base')",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.12"
    },
    "vscode": {
      "interpreter": {
        "hash": "5327a248d9883bedf47bfd9e608af95bf318797e621edcc550c6b5b3fdc820cc"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}