notebook/moving_average_timing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "3f4ab469",
   "metadata": {},
   "source": [
    "# How to add columns to Polars dataframes fast"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eca654d2",
   "metadata": {},
   "source": [
    "## The Conundrum"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7535e82",
   "metadata": {
    "cell_marker": "'''"
   },
   "source": [
    "I was recently writing a function to add columns of moving averages to a \n",
    "    [Polars](https://www.pola.rs/) dataframe.  For a particular problem, this is \n",
    "    a simple thing to do.  For example, we might have a dataframe like this \n",
    "    ([line 17 for a similar \n",
    "    example](https://github.com/afairless/polars_add_columns_dynamically_fast/blob/main/tests/test_moving_averages.py)):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "4e99a698",
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "import datetime\n",
    "import polars as pl\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "bf724770",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pl.DataFrame({\n",
    "    'timestamp': [\n",
    "        datetime.datetime(2022, 2, 1, 10, 0, 0),\n",
    "        datetime.datetime(2022, 2, 1, 10, 0, 1),\n",
    "        datetime.datetime(2022, 2, 1, 10, 0, 2),\n",
    "        datetime.datetime(2022, 2, 1, 10, 0, 3),\n",
    "        datetime.datetime(2022, 3, 1, 10, 0, 4)],\n",
    "    'data': [1, 2, 3, 4, 5]})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "6d2e2677",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><style>\n",
       ".dataframe > thead > tr > th,\n",
       ".dataframe > tbody > tr > td {\n",
       "  text-align: right;\n",
       "  white-space: pre-wrap;\n",
       "}\n",
       "</style>\n",
       "<small>shape: (5, 2)</small><table border=\"1\" class=\"dataframe\"><thead><tr><th>timestamp</th><th>data</th></tr><tr><td>datetime[μs]</td><td>i64</td></tr></thead><tbody><tr><td>2022-02-01 10:00:00</td><td>1</td></tr><tr><td>2022-02-01 10:00:01</td><td>2</td></tr><tr><td>2022-02-01 10:00:02</td><td>3</td></tr><tr><td>2022-02-01 10:00:03</td><td>4</td></tr><tr><td>2022-03-01 10:00:04</td><td>5</td></tr></tbody></table></div>"
      ],
      "text/plain": [
       "shape: (5, 2)\n",
       "┌─────────────────────┬──────┐\n",
       "│ timestamp           ┆ data │\n",
       "│ ---                 ┆ ---  │\n",
       "│ datetime[μs]        ┆ i64  │\n",
       "╞═════════════════════╪══════╡\n",
       "│ 2022-02-01 10:00:00 ┆ 1    │\n",
       "│ 2022-02-01 10:00:01 ┆ 2    │\n",
       "│ 2022-02-01 10:00:02 ┆ 3    │\n",
       "│ 2022-02-01 10:00:03 ┆ 4    │\n",
       "│ 2022-03-01 10:00:04 ┆ 5    │\n",
       "└─────────────────────┴──────┘"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a7dae34f",
   "metadata": {
    "cell_marker": "'''",
    "lines_to_end_of_cell_marker": 0,
    "lines_to_next_cell": 1
   },
   "source": [
    "Suppose we want to add three columns of [moving averages, one with a window \n",
    "    size](https://en.wikipedia.org/wiki/Moving_average) of 2, the second window\n",
    "    of size 3, and the final one of size 4.  That's a straightforward command\n",
    "    in Polars ([line \n",
    "    85](https://github.com/afairless/polars_add_columns_dynamically_fast/blob/main/src/s01_timing/moving_averages.py)):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "272e500d",
   "metadata": {
    "lines_to_next_cell": 1
   },
   "outputs": [],
   "source": [
    "def calculate_moving_averages_hard_coded(\n",
    "    df: pl.DataFrame, data_colname: str='data') -> pl.DataFrame:\n",
    "    \"\"\"\n",
    "    Calculates moving averages of 'data_colname' column in dataframe 'df' with\n",
    "        3 window sizes, so that each window size corresponds to a single column \n",
    "        of moving averages\n",
    "    \"\"\"\n",
    "\n",
    "    mas_df = df.with_columns(\n",
    "        ma_2=pl.col(data_colname).rolling_mean(window_size=2),\n",
    "        ma_3=pl.col(data_colname).rolling_mean(window_size=3),\n",
    "        ma_4=pl.col(data_colname).rolling_mean(window_size=4))\n",
    "\n",
    "    return mas_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "54c1b6a8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><style>\n",
       ".dataframe > thead > tr > th,\n",
       ".dataframe > tbody > tr > td {\n",
       "  text-align: right;\n",
       "  white-space: pre-wrap;\n",
       "}\n",
       "</style>\n",
       "<small>shape: (5, 5)</small><table border=\"1\" class=\"dataframe\"><thead><tr><th>timestamp</th><th>data</th><th>ma_2</th><th>ma_3</th><th>ma_4</th></tr><tr><td>datetime[μs]</td><td>i64</td><td>f64</td><td>f64</td><td>f64</td></tr></thead><tbody><tr><td>2022-02-01 10:00:00</td><td>1</td><td>null</td><td>null</td><td>null</td></tr><tr><td>2022-02-01 10:00:01</td><td>2</td><td>1.5</td><td>null</td><td>null</td></tr><tr><td>2022-02-01 10:00:02</td><td>3</td><td>2.5</td><td>2.0</td><td>null</td></tr><tr><td>2022-02-01 10:00:03</td><td>4</td><td>3.5</td><td>3.0</td><td>2.5</td></tr><tr><td>2022-03-01 10:00:04</td><td>5</td><td>4.5</td><td>4.0</td><td>3.5</td></tr></tbody></table></div>"
      ],
      "text/plain": [
       "shape: (5, 5)\n",
       "┌─────────────────────┬──────┬──────┬──────┬──────┐\n",
       "│ timestamp           ┆ data ┆ ma_2 ┆ ma_3 ┆ ma_4 │\n",
       "│ ---                 ┆ ---  ┆ ---  ┆ ---  ┆ ---  │\n",
       "│ datetime[μs]        ┆ i64  ┆ f64  ┆ f64  ┆ f64  │\n",
       "╞═════════════════════╪══════╪══════╪══════╪══════╡\n",
       "│ 2022-02-01 10:00:00 ┆ 1    ┆ null ┆ null ┆ null │\n",
       "│ 2022-02-01 10:00:01 ┆ 2    ┆ 1.5  ┆ null ┆ null │\n",
       "│ 2022-02-01 10:00:02 ┆ 3    ┆ 2.5  ┆ 2.0  ┆ null │\n",
       "│ 2022-02-01 10:00:03 ┆ 4    ┆ 3.5  ┆ 3.0  ┆ 2.5  │\n",
       "│ 2022-03-01 10:00:04 ┆ 5    ┆ 4.5  ┆ 4.0  ┆ 3.5  │\n",
       "└─────────────────────┴──────┴──────┴──────┴──────┘"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result = calculate_moving_averages_hard_coded(df) \n",
    "result"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4cb9b616",
   "metadata": {
    "cell_marker": "'''"
   },
   "source": [
    "But what if we don't know the window sizes in advance, or even how many columns\n",
    "    of moving averages we want to add?  It would be better to pass the window\n",
    "    sizes as a parameter to the function.  That way, our function can \n",
    "    accommodate any number and any sizes of windows.\n",
    "\n",
    "However, I found myself scratching my head about how to best do this in Polars. \n",
    "    It would be easy to add each column one-by-one in a loop.  But a big reason \n",
    "    to use Polars instead of [other dataframe \n",
    "    libraries](https://pandas.pydata.org/) is that Polars is fast, partly \n",
    "    because it can parallelize its workloads.  It can't parallelize creating our\n",
    "    moving-average columns if we're creating only one column at a time.\n",
    "\n",
    "I figured Polars had something that would handle this conundrum -- maybe an\n",
    "    *apply* method that would work across columns instead of rows.  But I \n",
    "    didn't find anything.  There might very well be something obvious that I\n",
    "    missed; and the Polars API is rapidly evolving, so if a feature doesn't \n",
    "    exist now, it likely will soon.  In any case, I found a good solution when\n",
    "    I started thinking beyond the Polars API."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18c4bc0a",
   "metadata": {},
   "source": [
    "## A Possible Solution"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf6b6ad9",
   "metadata": {
    "cell_marker": "'''"
   },
   "source": [
    "I started thinking about how I used to generate SQL queries as Python strings, \n",
    "    which saved me a lot of tedious typing when I had to specify a multitude of \n",
    "    conditions as [*CASE* \n",
    "    statements](https://www.w3schools.com/sql/sql_case.asp).  Could we do the \n",
    "    same thing here?\n",
    "    \n",
    "Yes, we can.  Python has \n",
    "    [*exec*](https://docs.python.org/3/library/functions.html#exec) and \n",
    "    [*eval*](https://docs.python.org/3/library/functions.html#eval) functions \n",
    "    that let us [execute a string as Python code](https://realpython.com/python-exec/).\n",
    "    For example, we can create a list like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "49689e6a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['meow', 'bark', 'oink', 'moo']"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a_list = eval('[\"meow\", \"bark\", \"oink\", \"moo\"]')\n",
    "a_list"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dbcccea3",
   "metadata": {
    "cell_marker": "'''"
   },
   "source": [
    "Or do some arithmetic:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "ebf7cf66",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a_sum = eval('4 + 3')\n",
    "a_sum"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2e35c7b",
   "metadata": {
    "cell_marker": "'''"
   },
   "source": [
    "Okay, that means that we can dynamically construct a Polars command to add as \n",
    "    many moving-average columns to our dataframe as we want.  But will that\n",
    "    approach actually be faster?  Let's test it and see."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8eb6d7ee",
   "metadata": {},
   "source": [
    "## Solution Code"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2745cc5",
   "metadata": {
    "cell_marker": "'''"
   },
   "source": [
    "First, we need to generate some data.  We can simply create a dataframe with a\n",
    "    series of timestamps and some Gaussian-distributed data.  We'll later \n",
    "    calculate the moving averages on that Gaussian data ([line \n",
    "    50](https://github.com/afairless/polars_add_columns_dynamically_fast/blob/main/src/s01_timing/moving_averages.py))."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "441e1b0b",
   "metadata": {
    "lines_to_next_cell": 1
   },
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "c2cbab4b",
   "metadata": {},
   "outputs": [],
   "source": [
    "def generate_data(\n",
    "    start_datetime: datetime.datetime=datetime.datetime(1970, 1, 1, 0, 0, 0), \n",
    "    end_datetime: datetime.datetime=datetime.datetime(1970, 1, 1, 0, 1, 0),\n",
    "    interval: str='1s',\n",
    "    column_n: int=1):\n",
    "    \"\"\"\n",
    "    Generates of Polars DataFrame with a 'timestamp' column and a provided\n",
    "        number ('column_n') of columns of standard-normally distributed data\n",
    "    For acceptable input values for 'interval', see:\n",
    "        https://pola-rs.github.io/polars/py-polars/html/reference/expressions/api/polars.date_range.html\n",
    "    \"\"\"\n",
    "\n",
    "    timestamp = pl.date_range(\n",
    "        start_datetime, end_datetime, interval, eager=True).alias('timestamp')\n",
    "\n",
    "    data = [\n",
    "        pl.Series(np.random.normal(size=len(timestamp))).alias(f'data{i}') \n",
    "        for i in range(column_n)]\n",
    "\n",
    "    df = pl.concat(\n",
    "        [pl.DataFrame(timestamp), pl.DataFrame(data)], how='horizontal')\n",
    "\n",
    "    return df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "2aeab02c",
   "metadata": {},
   "outputs": [],
   "source": [
    "start_datetime = datetime.datetime(2020, 1, 1, 0, 0, 0)\n",
    "end_datetime = datetime.datetime(2020, 2, 1, 0, 0, 0)\n",
    "interval = '1s'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "98514a61",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><style>\n",
       ".dataframe > thead > tr > th,\n",
       ".dataframe > tbody > tr > td {\n",
       "  text-align: right;\n",
       "  white-space: pre-wrap;\n",
       "}\n",
       "</style>\n",
       "<small>shape: (5, 2)</small><table border=\"1\" class=\"dataframe\"><thead><tr><th>timestamp</th><th>data0</th></tr><tr><td>datetime[μs]</td><td>f64</td></tr></thead><tbody><tr><td>2020-01-01 00:00:00</td><td>-1.401879</td></tr><tr><td>2020-01-01 00:00:01</td><td>0.23502</td></tr><tr><td>2020-01-01 00:00:02</td><td>-2.305322</td></tr><tr><td>2020-01-01 00:00:03</td><td>-1.373878</td></tr><tr><td>2020-01-01 00:00:04</td><td>-0.188347</td></tr></tbody></table></div>"
      ],
      "text/plain": [
       "shape: (5, 2)\n",
       "┌─────────────────────┬───────────┐\n",
       "│ timestamp           ┆ data0     │\n",
       "│ ---                 ┆ ---       │\n",
       "│ datetime[μs]        ┆ f64       │\n",
       "╞═════════════════════╪═══════════╡\n",
       "│ 2020-01-01 00:00:00 ┆ -1.401879 │\n",
       "│ 2020-01-01 00:00:01 ┆ 0.23502   │\n",
       "│ 2020-01-01 00:00:02 ┆ -2.305322 │\n",
       "│ 2020-01-01 00:00:03 ┆ -1.373878 │\n",
       "│ 2020-01-01 00:00:04 ┆ -0.188347 │\n",
       "└─────────────────────┴───────────┘"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_column_n = 1\n",
    "df = generate_data(start_datetime, end_datetime, interval, data_column_n)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb0c8462",
   "metadata": {
    "cell_marker": "'''"
   },
   "source": [
    "An example of the generated data is above.\n",
    "\n",
    "Next we need a function that will calculate each column of moving averages\n",
    "    one by one ([line \n",
    "    93](https://github.com/afairless/polars_add_columns_dynamically_fast/blob/main/src/s01_timing/moving_averages.py)).\n",
    "\n",
    "![image](./img/ma_loop.png)\n",
    "\n",
    "Notice that the columns are calculated sequentially in the list comprehension. \n",
    "    Then they're all assembled into the dataframe together.\n",
    "\n",
    "Now we need a function that implements our hopefully-faster solution: \n",
    "    constructing a Polars command that includes all our columns as a string, \n",
    "    then executing it all at once ([line \n",
    "    114](https://github.com/afairless/polars_add_columns_dynamically_fast/blob/main/src/s01_timing/moving_averages.py)).\n",
    "\n",
    "![image](./img/ma_eval.png)\n",
    "\n",
    "Notice that the loop now adds a string for each column, instead of actually\n",
    "    executing a command to create each column as in our prior function.  The\n",
    "    final execution is handled by Python's [*eval* \n",
    "    function](https://realpython.com/python-eval-function/).\n",
    "\n",
    "Finally, it'll be convenient to have a function that records how fast our two\n",
    "    approaches are ([line \n",
    "    157](https://github.com/afairless/polars_add_columns_dynamically_fast/blob/main/src/s01_timing/moving_averages.py)).\n",
    "\n",
    "![image](./img/time.png)\n",
    "\n",
    "We can call our timing function like this ([line \n",
    "    215](https://github.com/afairless/polars_add_columns_dynamically_fast/blob/main/src/s01_timing/moving_averages.py)):\n",
    "\n",
    "![image](./img/time_call.png)\n",
    "\n",
    "Notice that we send both the function that we want to time and the function\n",
    "    that does the timing (*perf_counter*, in this case) to our timing function."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7197b202",
   "metadata": {},
   "source": [
    "## Timing Conditions"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "daa7db42",
   "metadata": {
    "cell_marker": "'''"
   },
   "source": [
    "For our testing scenario, I set 200 different window sizes ([line \n",
    "    205](https://github.com/afairless/polars_add_columns_dynamically_fast/blob/main/src/s01_timing/moving_averages.py)), \n",
    "    so that 200 columns of moving averages would be added to the dataframe.  I \n",
    "    ran each test condition 400 times ([line \n",
    "    211](https://github.com/afairless/polars_add_columns_dynamically_fast/blob/main/src/s01_timing/moving_averages.py)) \n",
    "    and used the lowest recorded time.  There were two types of times \n",
    "    measured:  the [elapsed time, or wall-clock \n",
    "    time](https://en.wikipedia.org/wiki/Elapsed_real_time) and the *system+user \n",
    "    time*, that is, the [sum of the reported system time and the user \n",
    "    time](https://unix.stackexchange.com/questions/162115/why-does-the-user-and-sys-time-vary-on-multiple-executions).\n",
    "    During timing runs, my machine wasn't running any web browsers, multimedia\n",
    "    players, or other processing-intense applications, and I watched the system\n",
    "    monitor to verify that no unexpected processes were using substantial system\n",
    "    resources."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af98f0d0",
   "metadata": {},
   "source": [
    "## Timing Results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59b787a1",
   "metadata": {
    "cell_marker": "'''"
   },
   "source": [
    "The timing results clearly show that constructing the string query to be \n",
    "    executed all at once by Polars is faster than creating each moving average\n",
    "    column one-at-a-time.  Judging from the elapsed time, the *String Query, \n",
    "    Eval* approach is about 6 times faster than the *Loop One-by-One* method.\n",
    "\n",
    "![image](./img/elapsed_time_plot.png)\n",
    "\n",
    "Also, Polars is clearly accelerating the *String Query, Eval* approach by \n",
    "    parallelizing the workload.  Look at the next plot, which repeats the\n",
    "    elapsed times in the plot above and adds the *system+user time*.  The two \n",
    "    times for the *Loop One-by-One* approach (blue bars) are nearly the same,\n",
    "    suggesting that the processing is all serial.  But the *system+user time* \n",
    "    for the *String Query, Eval* approach is much higher than the elapsed time\n",
    "    (orange bars), suggesting that a lot more processing is getting crammed into \n",
    "    that small elapsed time -- that is, the workload is being parallelized. \n",
    "\n",
    "![image](./img/all_times_plot.png)\n",
    "\n",
    "This conclusion is consistent with my observations of my computer's CPU usage: \n",
    "    the *String Query, Eval* code was clearly engaging multiple cores \n",
    "    simultaneously, while the *Loop One-by-One* code did not.\n",
    "\n",
    "This is good news:  the *String Query, Eval* approach works faster, exactly as \n",
    "    intended.  However, it does have some disadvantages that we want to \n",
    "    consider before choosing it."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d0fa39be",
   "metadata": {},
   "source": [
    "## Disadvantages of the *String Query, Eval* Approach"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e202bd8",
   "metadata": {
    "cell_marker": "'''"
   },
   "source": [
    "One disadvantage of the *String Query, Eval* approach is that it doesn't play\n",
    "    nicely with [\n",
    "    IDEs](https://en.wikipedia.org/wiki/Integrated_development_environment) \n",
    "    like [NeoVim](https://neovim.io/) or [\n",
    "    VSCode](https://code.visualstudio.com/). Take a closer look at the signature \n",
    "    for our function that uses *eval*.\n",
    "\n",
    "![image](./img/ma_eval_signature.png)\n",
    "\n",
    "Notice that the [Language Server \n",
    "    Protocol (LSP)](https://en.wikipedia.org/wiki/Language_Server_Protocol) \n",
    "    claims that we didn't use the parameters *df* and *data_colname* in our \n",
    "    function. But we did.  We put them inside a string, so the LSP can't \n",
    "    detect them (I'm using [Pyright](https://github.com/microsoft/pyright) in an \n",
    "    [IDE built on Neovim](https://www.lunarvim.org/)).\n",
    "\n",
    "![image](./img/ma_eval_string.png)\n",
    "\n",
    "By contrast, notice that the LSP has no such difficulty detecting the same \n",
    "    variables in our *Loop One-by-One* function.\n",
    "\n",
    "![image](./img/ma_loop_signature.png)\n",
    "\n",
    "Another IDE-relevant disadvantage is that the *String Query, Eval* function is \n",
    "    less readable, because the code inside the string doesn't benefit from \n",
    "    syntax highlighting.\n",
    "\n",
    "A more serious issue is that using the Python *exec* and *eval* functions can\n",
    "    be a [security \n",
    "    vulnerability](https://realpython.com/python-exec/#uncovering-and-minimizing-the-security-risks-behind-exec). \n",
    "    If a nefarious actor could provide or alter our string query, they could\n",
    "    make our computer system execute any code that could compromise our \n",
    "    security, much like a [SQL injection \n",
    "    attack](https://en.wikipedia.org/wiki/SQL_injection).  \n",
    "\n",
    "In this particular case, though, that attack method isn't a great concern. \n",
    "    We're hard-coding our string (except for the values of our variables) within \n",
    "    our function, and we're not accepting any string input from outside parties. \n",
    "    If a nefarious actor could alter our string, they also have enough access to \n",
    "    alter any of our code; the query string is no more vulnerable than the rest \n",
    "    of our code. So while our code is quite safe for our timing exercise here, \n",
    "    this security vulnerability would be a serious concern in most production \n",
    "    systems."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "339530eb",
   "metadata": {},
   "source": [
    "## Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1e1f348",
   "metadata": {
    "cell_marker": "'''"
   },
   "source": [
    "To summarize, constructing a string query that is submitted to Polars so that \n",
    "    all our moving-average columns are created at once is a fast and viable \n",
    "    alternative to creating each column one-by-one.  So long as security issues \n",
    "    are not a concern, the only remaining trade-off against performance is code \n",
    "    readability in the IDE.  But the best alternative would be to find a way to\n",
    "    do this within the Polars API:  that way, we could enjoy both performance\n",
    "    and readability."
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "cell_metadata_filter": "-all",
   "main_language": "python",
   "notebook_metadata_filter": "-all"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}