machine-learning/glm.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Generalized Linear Models\n",
    "\n",
    "This notebook introduces the algorithms within [Dask-GLM](https://github.com/dask/dask-glm) for [Generalized Linear Models](https://en.wikipedia.org/wiki/Generalized_linear_model)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Start Dask Client for Dashboard\n",
    "\n",
    "Starting the Dask Client is optional.  It will provide a dashboard which \n",
    "is useful to gain insight on the computation.  \n",
    "\n",
    "The link to the dashboard will become visible when you create the client below.  We recommend having it open on one side of your screen while using your notebook on the other side.  This can take some effort to arrange your windows, but seeing them both at the same is very useful when learning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-27T19:21:46.509501Z",
     "iopub.status.busy": "2022-07-27T19:21:46.509058Z",
     "iopub.status.idle": "2022-07-27T19:21:47.686732Z",
     "shell.execute_reply": "2022-07-27T19:21:47.685976Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "    <div style=\"width: 24px; height: 24px; background-color: #e1e1e1; border: 3px solid #9D9D9D; border-radius: 5px; position: absolute;\"> </div>\n",
       "    <div style=\"margin-left: 48px;\">\n",
       "        <h3 style=\"margin-bottom: 0px;\">Client</h3>\n",
       "        <p style=\"color: #9D9D9D; margin-bottom: 0px;\">Client-5b571f45-0de1-11ed-a361-000d3a8f7959</p>\n",
       "        <table style=\"width: 100%; text-align: left;\">\n",
       "\n",
       "        <tr>\n",
       "        \n",
       "            <td style=\"text-align: left;\"><strong>Connection method:</strong> Cluster object</td>\n",
       "            <td style=\"text-align: left;\"><strong>Cluster type:</strong> distributed.LocalCluster</td>\n",
       "        \n",
       "        </tr>\n",
       "\n",
       "        \n",
       "            <tr>\n",
       "                <td style=\"text-align: left;\">\n",
       "                    <strong>Dashboard: </strong> <a href=\"http://10.1.1.64:8787/status\" target=\"_blank\">http://10.1.1.64:8787/status</a>\n",
       "                </td>\n",
       "                <td style=\"text-align: left;\"></td>\n",
       "            </tr>\n",
       "        \n",
       "\n",
       "        </table>\n",
       "\n",
       "        \n",
       "            <details>\n",
       "            <summary style=\"margin-bottom: 20px;\"><h3 style=\"display: inline;\">Cluster Info</h3></summary>\n",
       "            <div class=\"jp-RenderedHTMLCommon jp-RenderedHTML jp-mod-trusted jp-OutputArea-output\">\n",
       "    <div style=\"width: 24px; height: 24px; background-color: #e1e1e1; border: 3px solid #9D9D9D; border-radius: 5px; position: absolute;\">\n",
       "    </div>\n",
       "    <div style=\"margin-left: 48px;\">\n",
       "        <h3 style=\"margin-bottom: 0px; margin-top: 0px;\">LocalCluster</h3>\n",
       "        <p style=\"color: #9D9D9D; margin-bottom: 0px;\">94f79b63</p>\n",
       "        <table style=\"width: 100%; text-align: left;\">\n",
       "            <tr>\n",
       "                <td style=\"text-align: left;\">\n",
       "                    <strong>Dashboard:</strong> <a href=\"http://10.1.1.64:8787/status\" target=\"_blank\">http://10.1.1.64:8787/status</a>\n",
       "                </td>\n",
       "                <td style=\"text-align: left;\">\n",
       "                    <strong>Workers:</strong> 1\n",
       "                </td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                <td style=\"text-align: left;\">\n",
       "                    <strong>Total threads:</strong> 4\n",
       "                </td>\n",
       "                <td style=\"text-align: left;\">\n",
       "                    <strong>Total memory:</strong> 1.86 GiB\n",
       "                </td>\n",
       "            </tr>\n",
       "            \n",
       "            <tr>\n",
       "    <td style=\"text-align: left;\"><strong>Status:</strong> running</td>\n",
       "    <td style=\"text-align: left;\"><strong>Using processes:</strong> False</td>\n",
       "</tr>\n",
       "\n",
       "            \n",
       "        </table>\n",
       "\n",
       "        <details>\n",
       "            <summary style=\"margin-bottom: 20px;\">\n",
       "                <h3 style=\"display: inline;\">Scheduler Info</h3>\n",
       "            </summary>\n",
       "\n",
       "            <div style=\"\">\n",
       "    <div>\n",
       "        <div style=\"width: 24px; height: 24px; background-color: #FFF7E5; border: 3px solid #FF6132; border-radius: 5px; position: absolute;\"> </div>\n",
       "        <div style=\"margin-left: 48px;\">\n",
       "            <h3 style=\"margin-bottom: 0px;\">Scheduler</h3>\n",
       "            <p style=\"color: #9D9D9D; margin-bottom: 0px;\">Scheduler-079365e3-ef5b-4539-84b1-973599540812</p>\n",
       "            <table style=\"width: 100%; text-align: left;\">\n",
       "                <tr>\n",
       "                    <td style=\"text-align: left;\">\n",
       "                        <strong>Comm:</strong> inproc://10.1.1.64/9057/1\n",
       "                    </td>\n",
       "                    <td style=\"text-align: left;\">\n",
       "                        <strong>Workers:</strong> 1\n",
       "                    </td>\n",
       "                </tr>\n",
       "                <tr>\n",
       "                    <td style=\"text-align: left;\">\n",
       "                        <strong>Dashboard:</strong> <a href=\"http://10.1.1.64:8787/status\" target=\"_blank\">http://10.1.1.64:8787/status</a>\n",
       "                    </td>\n",
       "                    <td style=\"text-align: left;\">\n",
       "                        <strong>Total threads:</strong> 4\n",
       "                    </td>\n",
       "                </tr>\n",
       "                <tr>\n",
       "                    <td style=\"text-align: left;\">\n",
       "                        <strong>Started:</strong> Just now\n",
       "                    </td>\n",
       "                    <td style=\"text-align: left;\">\n",
       "                        <strong>Total memory:</strong> 1.86 GiB\n",
       "                    </td>\n",
       "                </tr>\n",
       "            </table>\n",
       "        </div>\n",
       "    </div>\n",
       "\n",
       "    <details style=\"margin-left: 48px;\">\n",
       "        <summary style=\"margin-bottom: 20px;\">\n",
       "            <h3 style=\"display: inline;\">Workers</h3>\n",
       "        </summary>\n",
       "\n",
       "        \n",
       "        <div style=\"margin-bottom: 20px;\">\n",
       "            <div style=\"width: 24px; height: 24px; background-color: #DBF5FF; border: 3px solid #4CC9FF; border-radius: 5px; position: absolute;\"> </div>\n",
       "            <div style=\"margin-left: 48px;\">\n",
       "            <details>\n",
       "                <summary>\n",
       "                    <h4 style=\"margin-bottom: 0px; display: inline;\">Worker: 0</h4>\n",
       "                </summary>\n",
       "                <table style=\"width: 100%; text-align: left;\">\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Comm: </strong> inproc://10.1.1.64/9057/4\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Total threads: </strong> 4\n",
       "                        </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Dashboard: </strong> <a href=\"http://10.1.1.64:39345/status\" target=\"_blank\">http://10.1.1.64:39345/status</a>\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Memory: </strong> 1.86 GiB\n",
       "                        </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Nanny: </strong> None\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\"></td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td colspan=\"2\" style=\"text-align: left;\">\n",
       "                            <strong>Local directory: </strong> /home/runner/work/dask-examples/dask-examples/machine-learning/dask-worker-space/worker-ptl8ho2_\n",
       "                        </td>\n",
       "                    </tr>\n",
       "\n",
       "                    \n",
       "\n",
       "                    \n",
       "\n",
       "                </table>\n",
       "            </details>\n",
       "            </div>\n",
       "        </div>\n",
       "        \n",
       "\n",
       "    </details>\n",
       "</div>\n",
       "\n",
       "        </details>\n",
       "    </div>\n",
       "</div>\n",
       "            </details>\n",
       "        \n",
       "\n",
       "    </div>\n",
       "</div>"
      ],
      "text/plain": [
       "<Client: 'inproc://10.1.1.64/9057/1' processes=1 threads=4, memory=1.86 GiB>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from dask.distributed import Client, progress\n",
    "client = Client(processes=False, threads_per_worker=4,\n",
    "                n_workers=1, memory_limit='2GB')\n",
    "client"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Make a random dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-27T19:21:47.690122Z",
     "iopub.status.busy": "2022-07-27T19:21:47.689737Z",
     "iopub.status.idle": "2022-07-27T19:21:47.926753Z",
     "shell.execute_reply": "2022-07-27T19:21:47.925891Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <tr>\n",
       "        <td>\n",
       "            <table>\n",
       "                <thead>\n",
       "                    <tr>\n",
       "                        <td> </td>\n",
       "                        <th> Array </th>\n",
       "                        <th> Chunk </th>\n",
       "                    </tr>\n",
       "                </thead>\n",
       "                <tbody>\n",
       "                    \n",
       "                    <tr>\n",
       "                        <th> Bytes </th>\n",
       "                        <td> 152.59 MiB </td>\n",
       "                        <td> 7.63 MiB </td>\n",
       "                    </tr>\n",
       "                    \n",
       "                    <tr>\n",
       "                        <th> Shape </th>\n",
       "                        <td> (200000, 100) </td>\n",
       "                        <td> (10000, 100) </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <th> Count </th>\n",
       "                        <td> 20 Tasks </td>\n",
       "                        <td> 20 Chunks </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                    <th> Type </th>\n",
       "                    <td> float64 </td>\n",
       "                    <td> numpy.ndarray </td>\n",
       "                    </tr>\n",
       "                </tbody>\n",
       "            </table>\n",
       "        </td>\n",
       "        <td>\n",
       "        <svg width=\"75\" height=\"170\" style=\"stroke:rgb(0,0,0);stroke-width:1\" >\n",
       "\n",
       "  <!-- Horizontal lines -->\n",
       "  <line x1=\"0\" y1=\"0\" x2=\"25\" y2=\"0\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"0\" y1=\"6\" x2=\"25\" y2=\"6\" />\n",
       "  <line x1=\"0\" y1=\"12\" x2=\"25\" y2=\"12\" />\n",
       "  <line x1=\"0\" y1=\"18\" x2=\"25\" y2=\"18\" />\n",
       "  <line x1=\"0\" y1=\"24\" x2=\"25\" y2=\"24\" />\n",
       "  <line x1=\"0\" y1=\"30\" x2=\"25\" y2=\"30\" />\n",
       "  <line x1=\"0\" y1=\"36\" x2=\"25\" y2=\"36\" />\n",
       "  <line x1=\"0\" y1=\"42\" x2=\"25\" y2=\"42\" />\n",
       "  <line x1=\"0\" y1=\"48\" x2=\"25\" y2=\"48\" />\n",
       "  <line x1=\"0\" y1=\"54\" x2=\"25\" y2=\"54\" />\n",
       "  <line x1=\"0\" y1=\"60\" x2=\"25\" y2=\"60\" />\n",
       "  <line x1=\"0\" y1=\"66\" x2=\"25\" y2=\"66\" />\n",
       "  <line x1=\"0\" y1=\"72\" x2=\"25\" y2=\"72\" />\n",
       "  <line x1=\"0\" y1=\"78\" x2=\"25\" y2=\"78\" />\n",
       "  <line x1=\"0\" y1=\"84\" x2=\"25\" y2=\"84\" />\n",
       "  <line x1=\"0\" y1=\"90\" x2=\"25\" y2=\"90\" />\n",
       "  <line x1=\"0\" y1=\"96\" x2=\"25\" y2=\"96\" />\n",
       "  <line x1=\"0\" y1=\"102\" x2=\"25\" y2=\"102\" />\n",
       "  <line x1=\"0\" y1=\"108\" x2=\"25\" y2=\"108\" />\n",
       "  <line x1=\"0\" y1=\"120\" x2=\"25\" y2=\"120\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Vertical lines -->\n",
       "  <line x1=\"0\" y1=\"0\" x2=\"0\" y2=\"120\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"25\" y1=\"0\" x2=\"25\" y2=\"120\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Colored Rectangle -->\n",
       "  <polygon points=\"0.0,0.0 25.41261651458249,0.0 25.41261651458249,120.0 0.0,120.0\" style=\"fill:#8B4903A0;stroke-width:0\"/>\n",
       "\n",
       "  <!-- Text -->\n",
       "  <text x=\"12.706308\" y=\"140.000000\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" >100</text>\n",
       "  <text x=\"45.412617\" y=\"60.000000\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(-90,45.412617,60.000000)\">200000</text>\n",
       "</svg>\n",
       "        </td>\n",
       "    </tr>\n",
       "</table>"
      ],
      "text/plain": [
       "dask.array<normal, shape=(200000, 100), dtype=float64, chunksize=(10000, 100), chunktype=numpy.ndarray>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from dask_glm.datasets import make_regression\n",
    "X, y = make_regression(n_samples=200000, n_features=100, n_informative=5, chunksize=10000)\n",
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-27T19:21:47.929938Z",
     "iopub.status.busy": "2022-07-27T19:21:47.929646Z",
     "iopub.status.idle": "2022-07-27T19:21:48.100130Z",
     "shell.execute_reply": "2022-07-27T19:21:48.096704Z"
    }
   },
   "outputs": [],
   "source": [
    "import dask\n",
    "X, y = dask.persist(X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Solve with a GLM algorithm\n",
    "\n",
    "*We also recommend looking at the \"Graph\" dashboard during execution if available*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-27T19:21:48.103783Z",
     "iopub.status.busy": "2022-07-27T19:21:48.103577Z",
     "iopub.status.idle": "2022-07-27T19:21:59.768184Z",
     "shell.execute_reply": "2022-07-27T19:21:59.767624Z"
    }
   },
   "outputs": [],
   "source": [
    "import dask_glm.algorithms\n",
    "\n",
    "b = dask_glm.algorithms.admm(X, y, max_iter=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Solve with a difference GLM algorithm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-27T19:21:59.772140Z",
     "iopub.status.busy": "2022-07-27T19:21:59.771482Z",
     "iopub.status.idle": "2022-07-27T19:22:04.451244Z",
     "shell.execute_reply": "2022-07-27T19:22:04.450652Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/share/miniconda3/envs/dask-examples/lib/python3.9/site-packages/dask/core.py:119: RuntimeWarning: overflow encountered in exp\n",
      "  return func(*(_execute_task(a, cache) for a in args))\n"
     ]
    }
   ],
   "source": [
    "b = dask_glm.algorithms.proximal_grad(X, y, max_iter=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Customizable with different families and regularizers\n",
    "\n",
    "The Dask-GLM project is nicely modular, allowing for different GLM families and regularizers, including a relatively straightforward interface for implementing custom ones."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-27T19:22:04.457858Z",
     "iopub.status.busy": "2022-07-27T19:22:04.455705Z",
     "iopub.status.idle": "2022-07-27T19:22:08.991631Z",
     "shell.execute_reply": "2022-07-27T19:22:08.991093Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/share/miniconda3/envs/dask-examples/lib/python3.9/site-packages/dask/core.py:119: RuntimeWarning: overflow encountered in exp\n",
      "  return func(*(_execute_task(a, cache) for a in args))\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/share/miniconda3/envs/dask-examples/lib/python3.9/site-packages/dask/core.py:119: RuntimeWarning: overflow encountered in exp\n",
      "  return func(*(_execute_task(a, cache) for a in args))\n"
     ]
    }
   ],
   "source": [
    "import dask_glm.families\n",
    "import dask_glm.regularizers\n",
    "\n",
    "family = dask_glm.families.Poisson()\n",
    "regularizer = dask_glm.regularizers.ElasticNet()\n",
    "\n",
    "b = dask_glm.algorithms.proximal_grad(\n",
    "    X, y, \n",
    "    max_iter=5, \n",
    "    family=family,\n",
    "    regularizer=regularizer,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-27T19:22:08.995266Z",
     "iopub.status.busy": "2022-07-27T19:22:08.994623Z",
     "iopub.status.idle": "2022-07-27T19:22:09.066066Z",
     "shell.execute_reply": "2022-07-27T19:22:09.065491Z"
    }
   },
   "outputs": [],
   "source": [
    "dask_glm.families.Poisson??"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-27T19:22:09.069331Z",
     "iopub.status.busy": "2022-07-27T19:22:09.068870Z",
     "iopub.status.idle": "2022-07-27T19:22:09.084687Z",
     "shell.execute_reply": "2022-07-27T19:22:09.084032Z"
    }
   },
   "outputs": [],
   "source": [
    "dask_glm.regularizers.ElasticNet??"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}