examples/cd_spot_the_diff_mnist_wine.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Interpretable drift detection with the spot-the-diff detector on MNIST and Wine-Quality datasets\n",
    "\n",
    "Under the hood drift detectors leverage a function of the data that is expected to be large when drift has occured and small when it hasn't. In the [Learned drift detectors on CIFAR-10](https://docs.seldon.io/projects/alibi-detect/en/latest/examples/cd_clf_cifar10.html) example notebook we note that we can **learn** a function satisfying this property by training a classifer to distinguish reference and test samples. However we now additionally note that if the classifier is specified in a certain way then when drift is detected we can inspect the weights of the classifier to shine light on exactly which features of the data were used to distinguish reference from test samples and therefore caused drift to be detected. \n",
    "\n",
    "The `SpotTheDiffDrift` detector is designed to make this process straightforward. Like the `ClassifierDrift` detector, it uses a portion of the available data to train a classifier to discriminate between reference and test instances. Letting $\\hat{p}_T(x)$ represent the probability assigned by the classifier that the instance $x$ is from the test set rather than reference set, the difference here is that we use a classifier of the form $$\\text{logit}(\\hat{p}_T) = b_0 + b_1 k(x,w_1) + ... + b_Jk(x,w_J),$$ where $k(\\cdot,\\cdot)$ is a kernel specifying a notion of similarity between instances, $w_i$ are learnable *test locations* and $b_i$ are learnable regression coefficients.\n",
    "\n",
    "The idea here is that if the detector flags drift and $b_i >0$ then we know that it reached its decision by considering how similar each instance is to the instance $w_i$, with those being more similar being more likely to be test instances than reference instances. Alternatively if $b_i < 0$ then instances more similar to $w_i$ were deemed more likely to be reference instances.\n",
    "\n",
    "In order to provide less noisy and therefore more interpretable results, we define each test location as $$w_i = \\bar{x}+d_i$$ where $\\bar{x}$ is the mean reference instance. We may then interpret $d_i$ as the additive transformation deemed to make the average reference more ($b_i>0$) or less ($b_i<0$) similar to a test instance. Defining the test locations in this way allows us to instead learn the difference $d_i$ and apply regularisation such that non-zero values must be justified by improved classification performance.  This allows us to more clearly identify which features any detected drift should be attributed to.\n",
    "\n",
    "This approach to interpretable drift detection is inspired by the work of [Jitkrittum et al. (2016)](https://arxiv.org/abs/1605.06796), however several major adaptations have been made.\n",
    "\n",
    "### Backend\n",
    "\n",
    "The method works with both the PyTorch and TensorFlow frameworks. Alibi Detect does however not install PyTorch for you. Check the PyTorch docs how to do this.\n",
    "\n",
    "### Dataset\n",
    "\n",
    "We start with an image example in order to provide a visual illustration of how the detector works. For this prupose we use the [MNIST dataset](https://en.wikipedia.org/wiki/MNIST_database) of 28 by 28 grayscale handwritten digits. To represent the common problem of new classes emerging during the deployment phase we consider a reference set of ~9,000 instances containing only the digits 1-9 and a test set of 10,000 instances containing all of the digits 0-9. We would like drift to be detected in this scenario because a model trained of the reference instances will not know how to process instances from the new class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import tensorflow as tf\n",
    "import torchvision\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from alibi_detect.cd import SpotTheDiffDrift\n",
    "\n",
    "np.random.seed(0)\n",
    "torch.manual_seed(0)\n",
    "tf.random.set_seed(0)\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "MNIST_PATH = 'my_path'\n",
    "DOWNLOAD = False\n",
    "MISSING_NUMBER = 0\n",
    "N = 10000\n",
    "\n",
    "# Load and shuffle data\n",
    "mnist_train_ds = torchvision.datasets.MNIST(MNIST_PATH, train=True, download=DOWNLOAD)\n",
    "all_x, all_y = mnist_train_ds.train_data, mnist_train_ds.train_labels\n",
    "perm = np.random.permutation(len(all_x))\n",
    "all_x, all_y = all_x[perm], all_y[perm]\n",
    "all_x = all_x[:, None, : , :].numpy().astype(np.float32)/255.\n",
    "\n",
    "# Create a reference and test set\n",
    "x_ref = all_x[:N]\n",
    "x = all_x[N:2*N]\n",
    "\n",
    "# Remove a class from reference set\n",
    "x_ref = x_ref[all_y[:10000] != MISSING_NUMBER]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When instantiating the detector we should specify the number of \"diffs\" we would like it to use to discriminate reference from test instances. Here there is a trade off. Using `n_diffs=1` is the simplest to interpret and seems to work well in practice. Using more diffs may result in stronger detection power but the diffs may be harder to interpret due to intereactions and conditional dependencies.\n",
    "\n",
    "The strength of the regularisation (`l1_reg`) to apply to the diffs should also be specified. Stronger regularisation results in sparser diffs as the classifier is encouraged to discriminate using fewer features. This may make the diff more interpretable but may again come at the cost of detection power.\n",
    "\n",
    "We should also specify how the classifier should be trained with standard arguments such as `learning_rate`, `epochs` and `batch_size`. By default a [Gaussian RBF](https://en.wikipedia.org/wiki/Radial_basis_function_kernel) is used for the kernel but alternatives can be specified via the `kernel` kwarg. Additionally the classifier can be initialised with any desired diffs by passing them with the `initial_diffs` kwarg -- by default they are initialised with Gaussian noise with standard deviation equal to that observed in the reference data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "cd = SpotTheDiffDrift(\n",
    "    x_ref,\n",
    "    n_diffs=1,\n",
    "    l1_reg=1e-4,\n",
    "    backend='tensorflow',\n",
    "    verbose=1,\n",
    "    learning_rate=1e-2, \n",
    "    epochs=5, \n",
    "    batch_size=64,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we then call the detector to detect drift on the deployment/test set it trains the classifier (thereby learning the diffs) and the usual `is_drift` and `p_val` properties can be inspected in the usual way:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "222/222 [=] - 2s 7ms/step - loss_ma: 0.6926\n",
      "222/222 [=] - 2s 7ms/step - loss_ma: 0.6905\n",
      "222/222 [=] - 2s 7ms/step - loss_ma: 0.6869\n",
      "222/222 [=] - 2s 7ms/step - loss_ma: 0.6831\n",
      "222/222 [=] - 2s 7ms/step - loss_ma: 0.6818\n",
      "Drift? Yes\n",
      "p-value: 6.911625415497573e-09\n"
     ]
    }
   ],
   "source": [
    "preds = cd.predict(x)\n",
    "\n",
    "print(f\"Drift? {'Yes' if preds['data']['is_drift'] else 'No'}\")\n",
    "print(f\"p-value: {preds['data']['p_val']}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As expected, the drift was detected. However we may now additionally look at the learned diffs and corresponding coefficients to determine how the detector reached this decision."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Diff coeff: [2.669408]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.colorbar.Colorbar at 0x7f31104da730>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATYAAAD4CAYAAABmKcrZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Z1A+gAAAACXBIWXMAAAsTAAALEwEAmpwYAAAd4ElEQVR4nO3df2wc53kn8O93l6RIiZRMmZJMS5R/KIobtb0qOtVJ66Rw6qQnB2gU9y6GfUDq3KVV/qhxDdA/6qZAExQ4wFc0yfWAIAclFuIUqZ0gic9qT1fHddOqSS85yToltqQ6phXJJiVLon5Yv0Vy97k/dpjbJTnPO+Qsd2aH3w+wELnvzsy7s8tHM/M+87w0M4iIFEkp6w6IiDSbApuIFI4Cm4gUjgKbiBSOApuIFE5HKzc2MDBg69evb+UmJYCBdo2Zt5fXX38dY2NjoY/VVVq+zjB5PdFr7drZ58xsW5rtLYRUgY3kNgB/AaAM4Mtm9rj3+vXr1+P73/9+mk1KkymwFcs999yTfiWVG+h8xwOJXjp+4MsD6TfYfPMObCTLAL4A4AMARgDsI7nbzA43q3Mikg2Wyll3IZU0R2x3Axg2s6MAQPJpANsBKLCJtDUu6sC2FsAbdb+PAHjX9BeR3AFgBwAMDQ2l2JyItAQXd2BLxMx2AtgJAFu2bNElG5GcI4lyZ1fW3UglTWAbBVB/CLYuek5E2txiPmLbB2AjyTtQC2gPAfj3TemViGRnMZ+KmtkkyUcBPIdauscuMzvUtJ7lzLXJ+LPojpKfNNGVMg36RsU/g/e2Xw7kcwRWnZq3/i5W062cKXas+du2wLqLnCZDACy1d+5+qmtsZrYHwJ4m9UVEcmERH7GJSEEt5lNRESkoEqVFPCoqIgVUu8amIzYRKRKdiopI8RAlBTYRKRTqVLSlvNyhUN5QKF+rI5CYtAw34hsD6VjVUrf/goAlmHDbWXE6UPU7Vy77XwGG8r06/Pc2WY3f8TcskP8XSMILFh2rjMcvG3hfKKX80wgs7+wWBNIiFxxBlDo0eCAiRaJrbCJSPO0f2Nr7vgkRaT4CLJcTPYKrIneRPE3y5Zh2kvxvJIdJ/pjklma8BQU2EWnA6IgtySOBrwDw5kS4H8DG6LEDwBdTvwHoVFREpmviNTYz20vyducl2wF81cwMwA9I3kRy0MxOptmuApuIzFDuSBwaBkjur/t9Z1RcNqnZKnGvBbCIApszRB8aIe9IU+IGgJWd4e/qpLssnbSDRAJ990rscPKqu2z52nl/3d77BmA9ft86SvHLL3TJJDeloxL4zALrDu2XYHqRxW/fmO2fJUkwec7JmJltXcj+zEd7BTYRaQmyZcl0C1KJW4MHIjJDqcREjybYDeC3o9HRdwN4K+31NUBHbCIyHTGXU1F/VeRTAO5F7VrcCIBPA+gEADP776gVqv0ggGEAVwH8h2ZsV4FNRBrUyhY1J7CZ2cOBdgPwe03ZWB0FNhFpRKJcbu+rVApsIjJDs47YsqLAJiINSDRrYCAz7RXYUuSihfKKQtPUeYIlcEICpYVCeXKuQPmcSt8at/2G+Rnolyf8vl+ZiC+5NB74UHoD8xb2dPjtXeUl8ct2Bso1TVx320P7tRyavi7jXLWQlGmfmcv33hWRTLQwj21BKLCJSAOSKAeOhvNOgU1EZtDggYgUC4GSTkVFpEiamaCbFQU2EZlmTtU9ckmBTUQaKY8tP8YDqWChPLVwnpszSjTp11ujBVYeyIOrdvf5yzs5VWevV9xFz73ltx89f9lt/8fhMbf98vX4HLxbbupxl7176Ca3fXWvXxNt0Gm3Tj8/b2lnuikTrwTy+7wcvKxjCgGU0iR25kCqwEbyGIBLACoAJvNYcE5E5khHbACA95mZ/9+2iLQVXWMTkYJh2995kDa92AB8h+SLJHfM9gKSO0juJ7l/bEwHdiJ5N3UTfIsq6C6ItEds7zGzUZKrATxP8l/MbG/9C6IZa3YCwJYtWxZ4+g4RaYZFfSpqZqPRv6dJPgPgbgB7/aVEJM9IoKvN7xWdd+9JLiPZN/UzgN8AMOs09iLSPgiiXEr2yKs0R2xrADwTXWTsAPBXZva3aToT3E1OXbJKoG5YaO0WyDWrOBdTu7qX+5se9+f2tLL/MVig9td1JwnvWiCf6sQlv+7Ya+f8vl+4Gl9vDQDGJ+O3f+aiv+3/fdyf87RS9T+zzWtXxLbdNbDMXXZgqb/PV3b737eeFH9ZgbeFsjMnaVNCDZHroJXEvHe/mR0F8EtN7IuI5ACxiAObiBQTCXQosIlIkZBs+8EDBTYRaVA7FVVgE5GC0TU2ESkULuZR0fnyygMFd6UzZVl34Mg5kPUQvDfOq+JyI1DzaEnXUrc9VHKpElj/VefNnbrip2OcvHTDbb+1zy/f07fB/wodOxufLjI+GSipdNnv29HTV9z2/T89F9v2kV9e5y773tv63fZQukdHJTB9n2OyHCiZFEj/SWsqj62d6YhNRGYot/lN8ApsItKgCLdUKbCJSAPlsYlI4egam4gUkgKbiBSK0j1EpHB0E3yThcrrptnVoc+pA34y2aRTui6UA5e2bPC5a36+1wVnirtqoBzT7f1+jl3Ipp5et33LYHxJp59euOYue8MpeQQAI+f85d84Fl/26IVlp91lVy9b4rav7Pb/dPoZSE50blkKzXw36XykzShRXYR7Rdu79yKyIJpZaJLkNpKvkBwm+dgs7R8jeYbkwejxO2n7n6sjNhHJXjOvsZEsA/gCgA8AGAGwj+RuMzs87aVfN7NHm7JRKLCJyDRNvsZ2N4DhqDAtSD4NYDuA6YGtqXQqKiKNOKdT0YGp6TWjx/RpONcCeKPu95Houen+Lckfk/wmyaG0b0FHbCLSgCA6k9djGzOzrSk3+dcAnjKzGyQ/AeBJAL+eZoU6YhORBkRtZDbJI4FRAPVHYOui537GzM6a2VQply8D+Ndp34MCm4g0au5M8PsAbCR5B8kuAA8B2N2wOXKw7tcPATiS9i209FTU4E8t1hkIs2lquXUEXsAb/jRz4+X46drGA/XSKoFcsrOBPLVTgbpkp6+Mx7ZNVPx8qlAe22Bvl9t+U6AuWX/lUmzbpgG/5tnoJb+W3PCdK912L4/t2BsX3GW/E3jfb1vp77eblgf+tMrO+s3/zLw1N+OSf+2IrTmDB2Y2SfJRAM8BKAPYZWaHSP4pgP1mthvAfyL5IQCTAM4B+Fja7eoam4jMUGpiPTYz2wNgz7Tn/qTu5z8C8EdN2yAU2ERkmqlrbO1MgU1EGpBER7m9L78rsInIDDpiE5FCIZp7jS0LCmwi0kj12ESkaHTENkclAF3ONck09diCy1bja5YBgHX4eUtXnMk/vXk9k7SPXvTnoBy7Gp+nFlr/+hX+HJW39vnve/0y/wveeeYnbnvlxGuxbeVqIMfutp932++/a9Bt/+5Lb8a2vXnsgrvsDw6edNu3BOYdXbdpldve47QZ/Qv3rHjfh2ZUZGv/a2zBoQ+Su0ieJvly3XMrST5P8tXoX/9TFpG2QRKd5VKiR14l6dlXAGyb9txjAF4ws40AXoh+F5ECqJ2KJnvkVTCwmdle1G5zqLcdtTvwEf374eZ2S0SyVCYTPfJqvtfY1pjZ1EWINwGsiXthVJ9pBwAMDaUusyQiC6wIgwepT5LNzOBcsTSznWa21cy2rhoYSLs5EVloBMqlZI+8mu8R2ymSg2Z2Mio54k/5IyJtY46FJnNpvr3fDeCR6OdHADzbnO6ISNamTkWTPPIqeMRG8ikA96JW23wEwKcBPA7gGyQ/DuA4gAeTbMw9Z0W4lpSXWxMoiYZyyX+rV73JGgFcuB5fG+zEJT8P7dWzgVpvgZppK3s63fbb++Ozorbc4s/72W+X3fby8I/c9isH/sltP3fkuNvuWfNe/0Tgrvf9R7e9oyu+Vtz5kWPustf7Yy8bAwAOHI+v9QYA/2bDzW57z1K/jp2nUorPPbRmVGRjvk8zkwgGNjN7OKbpvib3RURyoAiDB7qlSkRmaPO4psAmIjOVmlJkPDsKbCLSoDZhcta9SEeBTUQaUaeiIlIwBHUq2kyhgive1H0hTtUhAMC5a35ZozPOFHf/MnbFXfbQyFtu+8Zb+tz2wb4lbvtmJ6Vj4LxfVmjiJy+67Sf2/rPbfuZHr7vtl0/F75vOHv/r17nML7k08F5/v69y9lt5ib/uyvg1t/3sZb+UVCiFp2Lx6R6hm8tbcfO5jthEpHDyXLkjCQU2EWnQzAmTs6LAJiIztHlcU2ATkZnaPNtDgU1EGpG18uDtTIFNRGbQ4IGIFE6bH7C1NrAR4dJEnjT/i0wGkuCuBcoWnbx8I7YtlKd29Iyfb7Wy189Te/vNS932m986Gtt2fd933GVf+9Y/uu1nDp9x26+fj98vAPDKhfiSTr0d/pWcW7b4peQnuvySTOOTgeRFR0e3v+7ebv9PJ7Rl77vsT68HoBxftqgZ8UijoiJSPDmfgSoJBTYRmaHN45oCm4g0mppXtJ0psInIDO2e7tHueXgi0mTNngme5DaSr5AcJvnYLO1LSH49av8hydvTvgcFNhGZJtks8ElGTkmWAXwBwP0ANgF4mOSmaS/7OIDzZvY2AJ8H8F/SvgMFNhFpxKm7D8KPBO4GMGxmR81sHMDTALZPe812AE9GP38TwH1MeS6cq2tsaeqxeVPzAeHp+d508tQA4KUTF2PbDh/zp2KrBPKpbu6Nz0sCgL4u//+f6vBwbNvFV/16adfP+1MH3rjo51R5eWoA8BOnbtm7+v2aaD2r+t32M1f9GnpeHlu5w9/ny1b4fduw2s9z6wycp7mtgakiva9yipKFP0Mz0BKvaYDk/rrfd5rZzrrf1wJ4o+73EQDvmraOn73GzCZJvgXgZgBjc+p4nVwFNhHJCUuc3DxmZlsXsivzocAmIjMweWALGQVQfwvJuui52V4zQrIDwAoAZ9NsVNfYRGQaA6qVZI+wfQA2kryDZBeAhwDsnvaa3QAeiX7+dwD+3iz5ufBsdMQmIo3M5nIqGliVTZJ8FMBzAMoAdpnZIZJ/CmC/me0G8ASAvyQ5DOAcasEvFQU2EZmhiaeiMLM9APZMe+5P6n6+DuAjTdsgFNhEZDZNDGxZUGATkWmadyqalZYGNkNz8mxmX7n/QYTqrb169qrb/n9eix+kOXfqsrts701+TtSqZYE8toq//omfHoptuzzq11O7dt6fP/PK1Qm3PWTDss7Ytl9491p32VW/+Vtu+95ADt75C/HvbVn/SnfZvv4et311YK7X0By43texTH9Mz8uBa8odnoa2D2zBUVGSu0ieJvly3XOfITlK8mD0+ODCdlNEWsfAymSiR14lSff4CoBtszz/eTPbHD32zNIuIu3KqskeORU8FTWzvc24215E2oRZ7dHG0iToPkryx9GpauxNfSR3kNxPcv/Y2Lxv/RKRVmrzI7b5BrYvAtgAYDOAkwA+G/dCM9tpZlvNbOvAwMA8NycirUSrJnrk1bxGRc3s1NTPJL8E4G+a1iMRydgiTfcgOWhmJ6NfHwDwsvd6EWkjZkA1vyOeSQQDG8mnANyLWt2lEQCfBnAvyc2oZbwcA/CJZnQmVDOtoxqYb9FRpv9WzwTqsV06F58Tde2Sv+zArcvd9sFATlTn2Z+67edfPxXbNnnVz/XqCeRr9V/x89jGT/lzpt75i6tj2zZ99H3usjc2/Krbfvjgm277VacWXE+fnzt46y19bvvqQO5hyIST6FYOFRdcYERzb6nKQpJR0YdnefqJBeiLiORFteCBTUQWm/ZP91BgE5FGBbilSoFNRKYxsOiDByKyCOmITUQKxSxp2e/cylVg6wiMcnMiPnXBOv3SQKFZCkfP+WWLvHSPyet+WaFq5Wa3vbMUmF5viZ96sPztd7rtnt4hv2xRR89xt33pgJ8usvHD8RMYdWz7XXfZ74/6+/XAcX/awyXd8V/vUDrHfZvWuO3rlvvft2Wd/mfqtYZKHrUiG8Q0KioixaIjNhEpGoMCm4gUi5nBJtJVTs6aApuITKNTUREpGjOYApuIFI5GRUWkWHTENidVA65MxP9P0BvI/UHJ6a7XBqAy4ScHdXUE8o6cJLvxKxfdZcdv+LenlEp+YlJl+S1ue9fb/lVsW9+kX+rp+im/XPtt74+t+g4A6Bla57Z3vvs3Y9uOT/g5cIfPxE95CADXxv0/viGnXNR9m+LLKQHAXQO9bvvKHv/7tjTwXU6Ti+Z9k5ty67pGRUWkcDQqKiLFo1FRESka3SsqIkWke0VFpGB0xCYiBWNmsEkNHohIkSjdY25KDNep8ljZ6W6g4qfBTxy6bWCZ277i5qWxbVff8mt3TTq5ewDw8qlLbvvGlavc9tWrbott6+7y64Z1jL7mtpf6/W3bup9321+trIht+78n/Py/42f9qf36nHprAPDLd6yMbfulQD22lYF139Rddtu75v81D05DufB0KioiRWOAVRTYRKRQrCX3ipJcCeDrAG5HbeL1B81sRllkkhUAL0W/vm5mHwqtO8UBs4gUVrWS7JHOYwBeMLONAF6Ifp/NNTPbHD2CQQ3QEZuITGeGamtGRbcDuDf6+UkA/wDgD5uxYh2xiUgjM1ilmugBYIDk/rrHjjlsaY2ZnYx+fhNA3Chcd7TuH5D8cJIV64hNRBqYYSpoJTFmZrFTkZH8OwCzlaf548ZtmpGMGw++zcxGSd4J4O9JvmRm7nC+ApuITGNNu6XKzN4f10byFMlBMztJchDA6Zh1jEb/HiX5DwDeCaB9Ahsn4+cNBQDriM/JCi3bF8jnensgj23N6vj6XFcu3nCXvfqW3/7NH7zutt/St8Rt/9V18fXabl7t52uVbhp026vd8XloAPDKFf8r9KM343PVjp7153ItB+Zb/fW7/Jpqm5zPrDswie2KQJ5aOTBRbSgVjdX4Gn3+lgHjAv/Zzu2ILY3dAB4B8Hj077PTX0CyH8BVM7tBcgDAPQD+LLTi4DU2kkMkv0vyMMlDJH8/en4lyedJvhr961ckFJG2YGaojE8keqT0OIAPkHwVwPuj30FyK8kvR695B4D9JH8E4LsAHjezw6EVJwn9kwD+wMwOkOwD8CLJ5wF8DLWh2sdJPobaUG1TRjREJFutqO5hZmcB3DfL8/sB/E708z8D+MW5rjsY2KJRi5PRz5dIHgGwFgs4VCsiGYpGRdvZnE7WSd6O2oW7HyLhUG00/LsDAIaGhubdURFpnXYPbInz2Ej2AvgWgE+aWcMVYTMzxFwvNbOdZrbVzLYODAyk6qyILDyz2qhokkdeJTpiI9mJWlD7mpl9O3o60VCtiLSfapsfsQUDG0kCeALAETP7XF1TcKh2riwwhV6oNJFnSdVPB/m5QLrHr/1cfPmeC5f8dI7TI2+57edOXXbbd33vmNt+dsutsW1bb/XTNXo6/TSYil85CAdO+O+t7EwteOsKf9tLO/3Eh7sCn9nqpfHfp9D0eIEZEdOjs32vrRWqhuq4P2Vk3iU5YrsHwEcBvETyYPTcp1ALaN8g+XEAxwE8uCA9FJGWMiyCOQ/M7HtAbJXGGUO1ItLmFtuoqIgsDgpsIlIsBlSLfioqIouLQaeiIlI0ZqhOaPo9ESmS1lX3WDC5CmzBPDZPIPeHk+Nu+6qlfmmgLU4+WPev+PlWzx4YddtPB6ahO/H6Bbf9f3XEv/eDgWVD0w72LvE/k1BJpcHe+PblgXWHcs36A6WFlgZKE3lCZYeqaafIc76vmc++p1NRESkas0Vw54GILDbNq6CbFQU2EWlUBarjmjBZRArEYDoVFZGCMcBSj45kS4FNRGaoVhTYRKRA5jivaC61VWDzjo5L5S53WQZquS2ln2n9joGe2LbbVvi5XBtWxi8LAKOB6ftCOsvx+VqdgSnshgI10foCuWZezTPAz0Vzug0ASHvQ4C3uTX8HxJezmVIO5FyGuj7pvCCUfrfgx1JmMB2xiUihGFDRqKiIFIkBqGrwQEQKRaeiIlJEymMTkUKpjYrqiE1EikSBTUQKxwyVCY2Ktow312NwEKfDz9cK5S2tKMVfc1gR2IuDawPbXhOqJRfIc/Ny9Mp+zbLgxKGhuVyr/nuzanyOnwVyD8fN/1Q6A5N/uq1pav8lEMqbJPJbj82gOw9EpGg0/Z6IFJGusYlIodQq6CqwiUiRaPBARAqnAOke/nCciCw6hlrZoiSPNEh+hOQhklWSW53XbSP5Cslhko8lWbcCm4g0iq6xJXmk9DKA3wKwN+4FJMsAvgDgfgCbADxMclNoxcFTUZJDAL4KYA1qwXynmf0Fyc8A+F0AZ6KXfsrM9oTWt1BCtb0QqL9Vob8rKhb/f0BXoOZZ6ep5t53X/XlFQ3lsrDhzpgbyqWzJcr+9o9NvD9U1c8rcWSC3MJTRFcxd9L4Tofy8wDy1oU1fr/pfyMCUqBlrzU3wZnYEAEh3X90NYNjMjkavfRrAdgCHvYWSXGObBPAHZnaAZB+AF0k+H7V93sz+PME6RKRNmAFVSxzYBkjur/t9p5ntbGJ31gJ4o+73EQDvCi0UDGxmdhLAyejnSySPRBsTkQIyAOPJ67GNmZl3fezvANwyS9Mfm9mz8+heInMaFSV5O4B3AvghgHsAPErytwHsR+2obsY5F8kdAHYAwNDQUNr+ikgLVJIfsbnM7P0pVzEKoD5wrIuecyUePCDZC+BbAD5pZhcBfBHABgCbUTui++xsy5nZTjPbamZbBwYGkm5ORDJiqM03keTRAvsAbCR5B8kuAA8B2B1aKFFgI9mJWlD7mpl9GwDM7JSZVcysCuBLqF3kE5E2Z1Y7YkvySIPkAyRHAPwKgP9J8rno+VtJ7qn1xSYBPArgOQBHAHzDzA6F1p1kVJQAngBwxMw+V/f8YHT9DQAeQG3oVkQKoBVHY2b2DIBnZnn+BIAP1v2+B8CcMi6SXGO7B8BHAbxE8mD03KdQyyfZjNqR6zEAn5jLhpst+DmEytQEVuClk4wHMgfKPf1+e3efv4JQCZxKfMoFJ66lWjcCpYWC6SSd8WWLvCnoAKDspwGkTvFJg4F0kO5g5/LLkP5oLGtJRkW/h9kzgjLLWRORhVMbFc26F+noXlERaTB1ja2dKbCJyAxtfg+8ApuINKqle7R3ZFNgE5EGU3ls7UyBTUQamM3plqpcUmATkRl0KloQoY/R+w+sK21Vu9DGAzl4k6X4XDPrXOouG0q3YigXLJDP5ekIbDsw+15Qxfl6h3Z5G6ehpWYA2jzbQ4FNRKZbBAm6IrK4aPBARApH6R4iUjgaFRWRQtKpqIgUik5FRaRwijB4QGthZCZ5BsDxuqcGAIy1rANzk9e+5bVfgPo2X83s221mtirNCkj+LWp9SmLMzLal2d5CaGlgm7Fxcr83w02W8tq3vPYLUN/mK899a1eaCV5ECkeBTUQKJ+vA1swZo5str33La78A9W2+8ty3tpTpNTYRkYWQ9RGbiEjTKbCJSOFkEthIbiP5Cslhko9l0Yc4JI+RfInkQZL7M+7LLpKnSb5c99xKks+TfDX615+0tLV9+wzJ0WjfHST5QW8dC9i3IZLfJXmY5CGSvx89n+m+c/qVi/1WJC2/xkayDOAnAD4AYATAPgAPm9nhlnYkBsljALaaWebJnCR/DcBlAF81s1+InvszAOfM7PHoP4V+M/vDnPTtMwAum9mft7o/0/o2CGDQzA6Q7APwIoAPA/gYMtx3Tr8eRA72W5FkccR2N4BhMztqZuMAngawPYN+5J6Z7QVwbtrT2wE8Gf38JGp/GC0X07dcMLOTZnYg+vkSgCMA1iLjfef0S5osi8C2FsAbdb+PIF8frgH4DskXSe7IujOzWGNmJ6Of3wSwJsvOzOJRkj+OTlUzOU2uR/J2AO8E8EPkaN9N6xeQs/3W7jR4MNN7zGwLgPsB/F50ypVLVruOkKd8nS8C2ABgM4CTAD6bZWdI9gL4FoBPmtnF+rYs990s/crVfiuCLALbKIChut/XRc/lgpmNRv+eBvAMaqfOeXIqulYzdc3mdMb9+RkzO2VmFTOrAvgSMtx3JDtRCx5fM7NvR09nvu9m61ee9ltRZBHY9gHYSPIOkl0AHgKwO4N+zEByWXRRFySXAfgNAC/7S7XcbgCPRD8/AuDZDPvSYCpoRB5ARvuOJAE8AeCImX2urinTfRfXr7zstyLJ5M6DaDj7vwIoA9hlZv+55Z2YBck7UTtKA2q16v4qy76RfArAvaiVkDkF4NMA/geAbwBYj1oJqAfNrOUX8WP6di9qp1MG4BiAT9Rd02pl394D4J8AvIT/P5Pcp1C7npXZvnP69TBysN+KRLdUiUjhaPBARApHgU1ECkeBTUQKR4FNRApHgU1ECkeBTUQKR4FNRArn/wEqTJWMmrp/RAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(f\"Diff coeff: {preds['data']['diff_coeffs']}\")\n",
    "diff = preds['data']['diffs'][0,0]\n",
    "plt.imshow(diff, cmap='RdBu', vmin=-np.max(np.abs(diff)), vmax=np.max(np.abs(diff)))\n",
    "plt.colorbar()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The detector has identified the zero that was missing from the reference data -- it realised that test instances were on average **more** (coefficient > 0) simmilar to an instance with below average middle pixel values and above average zero-region pixel values than reference instances were. It used this information to determine that drift had occured. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Interpretable Drift Detection on the Wine Quality Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To provide an example on tabular data we consider the [Wine Quality Data Set](https://archive.ics.uci.edu/ml/datasets/wine+quality) consisting of 4898 and 1599 samples of white and red wine respectively. Each sample has an associated quality (as determined by experts) and 11 numeric features indicating its acidity, density, pH etc. To represent the problem of a model being trained on one distribution and deployed on a subtly different one, we take as a reference set the samples of white wine and consider the red wine samples to form a 'corrupted' deployment set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fixed acidity</th>\n",
       "      <th>volatile acidity</th>\n",
       "      <th>citric acid</th>\n",
       "      <th>residual sugar</th>\n",
       "      <th>chlorides</th>\n",
       "      <th>free sulfur dioxide</th>\n",
       "      <th>total sulfur dioxide</th>\n",
       "      <th>density</th>\n",
       "      <th>pH</th>\n",
       "      <th>sulphates</th>\n",
       "      <th>alcohol</th>\n",
       "      <th>quality</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>4898.000000</td>\n",
       "      <td>4898.000000</td>\n",
       "      <td>4898.000000</td>\n",
       "      <td>4898.000000</td>\n",
       "      <td>4898.000000</td>\n",
       "      <td>4898.000000</td>\n",
       "      <td>4898.000000</td>\n",
       "      <td>4898.000000</td>\n",
       "      <td>4898.000000</td>\n",
       "      <td>4898.000000</td>\n",
       "      <td>4898.000000</td>\n",
       "      <td>4898.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>6.854788</td>\n",
       "      <td>0.278241</td>\n",
       "      <td>0.334192</td>\n",
       "      <td>6.391415</td>\n",
       "      <td>0.045772</td>\n",
       "      <td>35.308085</td>\n",
       "      <td>138.360657</td>\n",
       "      <td>0.994027</td>\n",
       "      <td>3.188267</td>\n",
       "      <td>0.489847</td>\n",
       "      <td>10.514267</td>\n",
       "      <td>5.877909</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.843868</td>\n",
       "      <td>0.100795</td>\n",
       "      <td>0.121020</td>\n",
       "      <td>5.072058</td>\n",
       "      <td>0.021848</td>\n",
       "      <td>17.007137</td>\n",
       "      <td>42.498065</td>\n",
       "      <td>0.002991</td>\n",
       "      <td>0.151001</td>\n",
       "      <td>0.114126</td>\n",
       "      <td>1.230621</td>\n",
       "      <td>0.885639</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>3.800000</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.600000</td>\n",
       "      <td>0.009000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>0.987110</td>\n",
       "      <td>2.720000</td>\n",
       "      <td>0.220000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>3.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>6.300000</td>\n",
       "      <td>0.210000</td>\n",
       "      <td>0.270000</td>\n",
       "      <td>1.700000</td>\n",
       "      <td>0.036000</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>108.000000</td>\n",
       "      <td>0.991723</td>\n",
       "      <td>3.090000</td>\n",
       "      <td>0.410000</td>\n",
       "      <td>9.500000</td>\n",
       "      <td>5.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>6.800000</td>\n",
       "      <td>0.260000</td>\n",
       "      <td>0.320000</td>\n",
       "      <td>5.200000</td>\n",
       "      <td>0.043000</td>\n",
       "      <td>34.000000</td>\n",
       "      <td>134.000000</td>\n",
       "      <td>0.993740</td>\n",
       "      <td>3.180000</td>\n",
       "      <td>0.470000</td>\n",
       "      <td>10.400000</td>\n",
       "      <td>6.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>7.300000</td>\n",
       "      <td>0.320000</td>\n",
       "      <td>0.390000</td>\n",
       "      <td>9.900000</td>\n",
       "      <td>0.050000</td>\n",
       "      <td>46.000000</td>\n",
       "      <td>167.000000</td>\n",
       "      <td>0.996100</td>\n",
       "      <td>3.280000</td>\n",
       "      <td>0.550000</td>\n",
       "      <td>11.400000</td>\n",
       "      <td>6.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>14.200000</td>\n",
       "      <td>1.100000</td>\n",
       "      <td>1.660000</td>\n",
       "      <td>65.800000</td>\n",
       "      <td>0.346000</td>\n",
       "      <td>289.000000</td>\n",
       "      <td>440.000000</td>\n",
       "      <td>1.038980</td>\n",
       "      <td>3.820000</td>\n",
       "      <td>1.080000</td>\n",
       "      <td>14.200000</td>\n",
       "      <td>9.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       fixed acidity  volatile acidity  citric acid  residual sugar  \\\n",
       "count    4898.000000       4898.000000  4898.000000     4898.000000   \n",
       "mean        6.854788          0.278241     0.334192        6.391415   \n",
       "std         0.843868          0.100795     0.121020        5.072058   \n",
       "min         3.800000          0.080000     0.000000        0.600000   \n",
       "25%         6.300000          0.210000     0.270000        1.700000   \n",
       "50%         6.800000          0.260000     0.320000        5.200000   \n",
       "75%         7.300000          0.320000     0.390000        9.900000   \n",
       "max        14.200000          1.100000     1.660000       65.800000   \n",
       "\n",
       "         chlorides  free sulfur dioxide  total sulfur dioxide      density  \\\n",
       "count  4898.000000          4898.000000           4898.000000  4898.000000   \n",
       "mean      0.045772            35.308085            138.360657     0.994027   \n",
       "std       0.021848            17.007137             42.498065     0.002991   \n",
       "min       0.009000             2.000000              9.000000     0.987110   \n",
       "25%       0.036000            23.000000            108.000000     0.991723   \n",
       "50%       0.043000            34.000000            134.000000     0.993740   \n",
       "75%       0.050000            46.000000            167.000000     0.996100   \n",
       "max       0.346000           289.000000            440.000000     1.038980   \n",
       "\n",
       "                pH    sulphates      alcohol      quality  \n",
       "count  4898.000000  4898.000000  4898.000000  4898.000000  \n",
       "mean      3.188267     0.489847    10.514267     5.877909  \n",
       "std       0.151001     0.114126     1.230621     0.885639  \n",
       "min       2.720000     0.220000     8.000000     3.000000  \n",
       "25%       3.090000     0.410000     9.500000     5.000000  \n",
       "50%       3.180000     0.470000    10.400000     6.000000  \n",
       "75%       3.280000     0.550000    11.400000     6.000000  \n",
       "max       3.820000     1.080000    14.200000     9.000000  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "red_df = pd.read_csv(\n",
    "    \"https://storage.googleapis.com/seldon-datasets/wine_quality/winequality-red.csv\", sep=';'\n",
    ")\n",
    "white_df = pd.read_csv(\n",
    "    \"https://storage.googleapis.com/seldon-datasets/wine_quality/winequality-white.csv\", sep=';'\n",
    ")\n",
    "white_df.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the data for both red and white wine samples take the same format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fixed acidity</th>\n",
       "      <th>volatile acidity</th>\n",
       "      <th>citric acid</th>\n",
       "      <th>residual sugar</th>\n",
       "      <th>chlorides</th>\n",
       "      <th>free sulfur dioxide</th>\n",
       "      <th>total sulfur dioxide</th>\n",
       "      <th>density</th>\n",
       "      <th>pH</th>\n",
       "      <th>sulphates</th>\n",
       "      <th>alcohol</th>\n",
       "      <th>quality</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>8.319637</td>\n",
       "      <td>0.527821</td>\n",
       "      <td>0.270976</td>\n",
       "      <td>2.538806</td>\n",
       "      <td>0.087467</td>\n",
       "      <td>15.874922</td>\n",
       "      <td>46.467792</td>\n",
       "      <td>0.996747</td>\n",
       "      <td>3.311113</td>\n",
       "      <td>0.658149</td>\n",
       "      <td>10.422983</td>\n",
       "      <td>5.636023</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>1.741096</td>\n",
       "      <td>0.179060</td>\n",
       "      <td>0.194801</td>\n",
       "      <td>1.409928</td>\n",
       "      <td>0.047065</td>\n",
       "      <td>10.460157</td>\n",
       "      <td>32.895324</td>\n",
       "      <td>0.001887</td>\n",
       "      <td>0.154386</td>\n",
       "      <td>0.169507</td>\n",
       "      <td>1.065668</td>\n",
       "      <td>0.807569</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>4.600000</td>\n",
       "      <td>0.120000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.900000</td>\n",
       "      <td>0.012000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>0.990070</td>\n",
       "      <td>2.740000</td>\n",
       "      <td>0.330000</td>\n",
       "      <td>8.400000</td>\n",
       "      <td>3.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>7.100000</td>\n",
       "      <td>0.390000</td>\n",
       "      <td>0.090000</td>\n",
       "      <td>1.900000</td>\n",
       "      <td>0.070000</td>\n",
       "      <td>7.000000</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>0.995600</td>\n",
       "      <td>3.210000</td>\n",
       "      <td>0.550000</td>\n",
       "      <td>9.500000</td>\n",
       "      <td>5.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>7.900000</td>\n",
       "      <td>0.520000</td>\n",
       "      <td>0.260000</td>\n",
       "      <td>2.200000</td>\n",
       "      <td>0.079000</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>38.000000</td>\n",
       "      <td>0.996750</td>\n",
       "      <td>3.310000</td>\n",
       "      <td>0.620000</td>\n",
       "      <td>10.200000</td>\n",
       "      <td>6.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>9.200000</td>\n",
       "      <td>0.640000</td>\n",
       "      <td>0.420000</td>\n",
       "      <td>2.600000</td>\n",
       "      <td>0.090000</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>62.000000</td>\n",
       "      <td>0.997835</td>\n",
       "      <td>3.400000</td>\n",
       "      <td>0.730000</td>\n",
       "      <td>11.100000</td>\n",
       "      <td>6.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>15.900000</td>\n",
       "      <td>1.580000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>15.500000</td>\n",
       "      <td>0.611000</td>\n",
       "      <td>72.000000</td>\n",
       "      <td>289.000000</td>\n",
       "      <td>1.003690</td>\n",
       "      <td>4.010000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>14.900000</td>\n",
       "      <td>8.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       fixed acidity  volatile acidity  citric acid  residual sugar  \\\n",
       "count    1599.000000       1599.000000  1599.000000     1599.000000   \n",
       "mean        8.319637          0.527821     0.270976        2.538806   \n",
       "std         1.741096          0.179060     0.194801        1.409928   \n",
       "min         4.600000          0.120000     0.000000        0.900000   \n",
       "25%         7.100000          0.390000     0.090000        1.900000   \n",
       "50%         7.900000          0.520000     0.260000        2.200000   \n",
       "75%         9.200000          0.640000     0.420000        2.600000   \n",
       "max        15.900000          1.580000     1.000000       15.500000   \n",
       "\n",
       "         chlorides  free sulfur dioxide  total sulfur dioxide      density  \\\n",
       "count  1599.000000          1599.000000           1599.000000  1599.000000   \n",
       "mean      0.087467            15.874922             46.467792     0.996747   \n",
       "std       0.047065            10.460157             32.895324     0.001887   \n",
       "min       0.012000             1.000000              6.000000     0.990070   \n",
       "25%       0.070000             7.000000             22.000000     0.995600   \n",
       "50%       0.079000            14.000000             38.000000     0.996750   \n",
       "75%       0.090000            21.000000             62.000000     0.997835   \n",
       "max       0.611000            72.000000            289.000000     1.003690   \n",
       "\n",
       "                pH    sulphates      alcohol      quality  \n",
       "count  1599.000000  1599.000000  1599.000000  1599.000000  \n",
       "mean      3.311113     0.658149    10.422983     5.636023  \n",
       "std       0.154386     0.169507     1.065668     0.807569  \n",
       "min       2.740000     0.330000     8.400000     3.000000  \n",
       "25%       3.210000     0.550000     9.500000     5.000000  \n",
       "50%       3.310000     0.620000    10.200000     6.000000  \n",
       "75%       3.400000     0.730000    11.100000     6.000000  \n",
       "max       4.010000     2.000000    14.900000     8.000000  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "red_df.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We extract the features and shuffle and normalise them such that they take values in \\[0,1\\]."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "white, red = np.asarray(white_df, np.float32)[:, :-1], np.asarray(red_df, np.float32)[:, :-1]\n",
    "n_white, n_red = white.shape[0], red.shape[0]\n",
    "\n",
    "col_maxes = white.max(axis=0)\n",
    "white, red = white / col_maxes, red / col_maxes\n",
    "white, red = white[np.random.permutation(n_white)], red[np.random.permutation(n_red)]\n",
    "x, x_corr = white, red"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We then split off half of the reference set to act as an unseen sample from the same underlying distribution for which drift should *not* be detected."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_ref = x[:len(x)//2]\n",
    "x_h0 = x[len(x)//2:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We instantiate our detector in the same way as we do above, but this time using the *Pytorch* backend for the sake of variety. We then get the predictions of the detector on both the undrifted and corrupted test sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "No GPU detected, fall back on CPU.\n",
      "Epoch 1/5: 100%|██████████| 58/58 [00:00<00:00, 200.84it/s, loss_ma=0.694]\n",
      "Epoch 2/5: 100%|██████████| 58/58 [00:00<00:00, 201.83it/s, loss_ma=0.694]\n",
      "Epoch 3/5: 100%|██████████| 58/58 [00:00<00:00, 196.56it/s, loss_ma=0.694]\n",
      "Epoch 4/5: 100%|██████████| 58/58 [00:00<00:00, 201.99it/s, loss_ma=0.693]\n",
      "Epoch 5/5: 100%|██████████| 58/58 [00:00<00:00, 194.22it/s, loss_ma=0.694]\n",
      "Epoch 1/5: 100%|██████████| 48/48 [00:00<00:00, 267.09it/s, loss_ma=0.624]\n",
      "Epoch 2/5: 100%|██████████| 48/48 [00:00<00:00, 195.68it/s, loss_ma=0.533]\n",
      "Epoch 3/5: 100%|██████████| 48/48 [00:00<00:00, 193.16it/s, loss_ma=0.464]\n",
      "Epoch 4/5: 100%|██████████| 48/48 [00:00<00:00, 190.22it/s, loss_ma=0.412]\n",
      "Epoch 5/5: 100%|██████████| 48/48 [00:00<00:00, 231.42it/s, loss_ma=0.37] \n"
     ]
    }
   ],
   "source": [
    "cd = SpotTheDiffDrift(\n",
    "    x_ref,\n",
    "    n_diffs=1,\n",
    "    l1_reg=1e-4,\n",
    "    backend='pytorch',\n",
    "    verbose=1,\n",
    "    learning_rate=1e-2, \n",
    "    epochs=5, \n",
    "    batch_size=64,\n",
    ")\n",
    "\n",
    "preds_h0 = cd.predict(x_h0)\n",
    "preds_corr = cd.predict(x_corr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Drift on h0? No\n",
      "p-value on h0: 0.7351070494981216\n",
      "Drift on corrupted? Yes\n",
      "p-value on corrupted:: 3.825295341753544e-201\n"
     ]
    }
   ],
   "source": [
    "print(f\"Drift on h0? {'Yes' if preds_h0['data']['is_drift'] else 'No'}\")\n",
    "print(f\"p-value on h0: {preds_h0['data']['p_val']}\")\n",
    "print(f\"Drift on corrupted? {'Yes' if preds_corr['data']['is_drift'] else 'No'}\")\n",
    "print(f\"p-value on corrupted:: {preds_corr['data']['p_val']}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As expected drift is detected on the red wine samples but not the held out white wine samples from the same distribution. Now we can inspect the returned diff to determine how the detector reached its decision"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Diff coeff: [-1.905726]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcIAAAD4CAYAAAB/juY6AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Z1A+gAAAACXBIWXMAAAsTAAALEwEAmpwYAAAltElEQVR4nO3deZyVdd3/8ddb3MAFFCbTFCcsRU0dbVRcbywycSlzo1xR70itvF2oyKxb05JC7ywNjSwpckV/mrmLhpiICgi4IOqNdLsLuIu44Of3x/WdPE5nZs6cMzPXmTnv5+NxHlznWt/XMPCZ73Vdcz6KCMzMzGrVSnkHMDMzy5MLoZmZ1TQXQjMzq2kuhGZmVtNcCM3MrKatnHcAa58BAwZEfX193jHMPmbBggUAbLbZZjknMStu1qxZSyKirtgyF8Jupr6+npkzZ+Ydw+xjhg4dCsDUqVNzzWHWEkn/bGmZL42amVlN84jQzCp2+umn5x3BrGwuhGZWsWHDhuUdwaxsvjRqZhWbM2cOc+bMyTuGWVk8IjSzip100kmAH5ax7skjQjMzq2kuhGZmVtNcCM3MrKb5HqGZtal+zE2tLn9x4VKGDOrfRWnMOpYLoZlVrN/uR/HzE3bOO4ZZWXxp1MwqtvqGm7Pzzi6E1j25EJpZxZY/O5/p06fnHcOsLC6EbZC0SNKAMrabKOmgdqxfL+mR9h7HrBq8Nu1PnHbaaXnHMCuLC6GZmdU0F8ICkq6XNEvSo5JGFVl+pKR5kuZKmpTm1Uu6K82/U9LAgk12lzRd0sKm0aEy4yQ9IulhSSO66PTMzKwIPzX6ccdExCuSegMPSrq2aYGkLYHTgZ0jYomkddOiC4A/RcSfJB0D/AbYPy1bH9gVGAzcAFwDHAA0ANsAA9JxpnX6mZmZWVEeEX7ciZLmAjOAjYDPFiz7AjA5IpYARMQraf5OwOVpehJZ4WtyfUR8GBGPAeulebsCV0TEioh4Cbgb2L61UJJGSZopaebixYsrOD0zM2vOhTCRNBQYBuwUEdsADwGrV7jbdwsPUe5OImJCRDRGRGNdXV2Fkcw63rpfHMX555+fdwyzsrgQfqQv8GpELJM0GBjSbPldwMGS+gMUXBqdDnw9TR8G3NPGce4BRkjqJakO2B14oCNOwCwvq643iIaGhrxjmJXFhfAjtwIrS5oPjCW7PPovEfEo8DPg7nT59H/Sou8CR0uaBxwB/Fcbx7kOmAfMJSuu34+IFzvsLMxy8M6iOUyZMiXvGGZlUUTkncHaobGxMWbOnJl3DKsxbX7W6OVjGDKov/sRWtWSNCsiGost84jQzMxqmguhmZnVNP8eoZm1adHYfVpdPnTGuC5KYtbxPCI0M7Oa5hGhmVXsd7/7Xd4RzMrmQmhmFdtss83yjmBWNhdC6/HaevTfKrfsqfv5w1Hbs99+++UdxazdfI/QzCr2xgPXcd555+Udw6wsLoRmZlbTXAjNzKym1XwhlDRS0oWVrlNkm5Mk9aksnZmZdbaaL4Sd6CTAhdDMrMr1yEIoaQ1JN0maK+kRSSMkLZI0IC1vlDS1yHYTJV2cmuA+IWnfgsUbSLpV0pOSflmwzUVp/UclnZnmnQhsAPxd0t/TvD0l3SdptqTJktZM88dKekzSPEnndt5XxazzDNj3VCZNmpR3DLOy9NRfn9gLeD4i9gGQ1Bf4RYnb1gM7AJuQFbLPpPkNwLZkzXYXSLogIp4BfhQRr0jqBdwpaeuI+I2kU4A9ImJJKsCnA8Mi4m1JPwBOkfRb4GvA4IgISf2KBZI0ChgFMHDgwPZ9Jcy6wMpr17HRRhvlHcOsLD1yRAg8DHxJ0i8k7RYRr7dj26sj4sOIeBJYCAxO8++MiNcjYjnwGLBxmn+IpNlkHe23BLYoss8haf69kuYAR6XtXweWA3+QdACwrFggd6i3avf2/GlcddVVeccwK0uPLIQR8QSwHVlBPFvST4AP+Oh8V29t8xbev1swbwVZE99PA6OBL0bE1sBNLexbwB0R0ZBeW0TEsRHxAdno8xpgX7LmwGbdzpsP3cxFF12UdwyzsvTIQihpA2BZRPwFGEdWFBcBn0+rHNjK5gdLWknSJsAgYEEr664NvA28Lmk9YHjBsjeBtdL0DGCXpsus6R7mpuk+Yd+IuBk4GdimHadpZmYdoKfeI9wKGCfpQ+B94HigN9klyLOAqa1s+3/AA2RF7riIWC6p6IoRMVfSQ8DjwDPAvQWLJwC3Sno+IvaQNBK4QtJqafnpZMXyr5JWJxs1nlLOyZqZWfl6ZCGMiNuA24os2rTIuhOBiQWzpkTEca2tExH7FkyPbCHDBcAFBe/vArYvsuoOxbY3M7Ou0SMvjZqZmZWqR44Iy9XS6M66t7a6q1vllozeMe8IZmVzITSzig0YMCDvCGZl86VRM6vYxIkTmThxYt4xzMriQmhmFXMhtO7Ml0bNclA/5qa8I3SoFxcuZcig/nnHMCuLR4RmZlbTXAjNzKymuRCamVlNcyHMiaSpkhoL3tdLeiTPTGbl+sTBZ3DzzTfnHcOsLC6EZlaxlVZZnT59+uQdw6wsLoSdLI30Hpd0maT5kq6R5P8xrEd5c/ZNjB8/Pu8YZmVxIewamwHjI2Jz4A3ghDT/MklzUrPeFq8rSRolaaakmYsXL+78tGbt9Pbj93D11VfnHcOsLC6EXeOZiGhq0fQXYNc0fVhTs15g75Y2dod6M7PO40LYNVrqem9mZjlzIewaAyXtlKYPBf6RZxgzM/uIC2HXWAB8W9J8YB3gopzzmJlZ4s8a7RofRMThzeYNLXwTEYuAz3VVILOO9MlDxzLVfR+tm/KI0MzMapoi/NxGd9LY2BgzZ87MO4bZx5x77rkAjB49OuckZsVJmhURjcWWeURoZhW78cYbufHGG/OOYVYWF0IzM6tpLoRmZlbT/NSombWqfsxNba7jDvXWnXlEaGYV08qr0bt377xjmJXFhdDMKrbeIWdyyy235B3DrCwuhGZmVtNcCIuQdIakDvuFKEk3S+qXXie0vYVZ9/LavVdw1lln5R3DrCwuhF0gIvaOiNeAfnzUi9Csx1j+z7nceeedeccwK4sLYSLpR5KekPQPska6SNpE0q2SZkm6R9LgNH+ipN9Imi5poaSD0vz1JU1LzXYfkbRbmr9I0gBgLLBJWj5O0p8l7V+Q4TJJX+3qczczq2X+9QlA0ueBrwMNZF+T2cAsYAJwXEQ8KWlHYDzwhbTZ+mQNdgcDNwDXkLVYui0ifiapF9Cn2aHGAJ9LjXiR9B/AycD1kvoCOwNHFck3ChgFMHDgwI45aTMzA1wIm+wGXBcRywAk3QCsTlaYJktqWm+1gm2uj4gPgcckrZfmPQj8UdIqafmc1g4aEXdLGi+pDjgQuDYiPiiy3gSyokxjY6M/HNbMrAP50mjLVgJei4iGgtfmBcvfLZgWQERMA3YHngMmSjqyhOP8GTgcOBr4Y8dEN+tavXqvTf/+/oV6655cCDPTgP0l9Za0FrAfsAx4WtLBAMps09pOJG0MvBQRvwcuAbZrtsqbwFrN5k0ETgKIiMcqPA+zXNR97TSuvfbavGOYlcWFEIiI2cBVwFzgFrJLnACHAcdKmgs8CrT1IMtQYK6kh4ARwK+bHWcpcG96kGZcmvcSMB+4tGPOxszM2sP9CHMmqQ/wMLBdRLze1vruR2hdrZTPGn317omcMPQznHPOOV2QyKz93I+wSkkaRjYavKCUImhWrd597nHuu+++vGOYlcVPjeYoIqYAG+edw8yslrkQmlmrFo3dp811hs4Y1wVJzDqHL42amVlN84jQzCq24YYb5h3BrGx+arSb8VOjVq1KebrUrBKlXKZviZ8aNTMza4ELoZlV7KSTTuKVKRPyjmFWFt8jNLOKzZkzh/deXpp3DLOyeERoZmY1rdVCKKmfpDY7qkuql3Roies90p6ALeznDEmj0/Tg1Oj2IUmbVLrvtM+mRrpIml7mPo4r1n2io74GZmbWMdoaEfYD2iyEQD1ZU9o87A9cExHbRsT/lrKBpJIvCUfEzuWEioiLI+LP5WxrZmZdp61COBbYJI24xqVWRONS94SHJY0oWG+3tN7JadRzj6TZ6dVqMZG0vqRpaftHJO2W5r9VsM5BkiY2225vshZGx0v6e/PRlqTRks5I01MlnS9pJvBfzfbTX9Ltkh6VdAmpv2BhhpbOXdKvJf0kTX85ncdKzUatn5c0N3Wx+HbBvnulfT4oaZ6kb7Xx92FWlTbddFNWWfdTeccwK0tbI6MxwOciogFA0oFAA7ANMAB4UNK0tN7oiNg3rdcH+FJELJf0WeAKoOjvbySHArdFxM8k9QL6lBI+Im6WdDHwVkScK6m+jU1WbeH3SP4b+EdE/FTSPsCxRdY5gOLn/sM0fQ/wG2DviPiwoKs9ZC2WvhMR05raLyXHAq9HxPaSViNr0XR7RDxduLGkUcAogIEDB7ZximZdb8KECdzu3yO0bqq9D8vsClwREStSH727ge2LrLcK8HtJDwOTgS3a2O+DwNFp9LZVRLzZzlyluqqF+bsDfwGIiJuAV4usU/TcI2IZ8E3gDuDC5pdnJfUD+qXu9QCTChbvCRwpaQ5wP9Af+GzzA0fEhIhojIjGurq6kk7UzMxK01lPjZ4MvEQ2emoEVm1t5VQkdgeeAyYWPGRS+LE3q5dw3A/4+Dk13+btEvZRjq2ApcAG7dxOwHcjoiG9Ph0Rt3d8PLPONWrUKJbeekHeMczK0lYhfBNYq+D9PcCIdG+rjqx4PVBkvb7ACxHxIXAE0Ku1g0jaGHgpIn4PXAJslxa9JGlzSSsBXyvhfF4CPpHu+a0G7FvCNgDTSA/7SBoOrFNknaLnnrKfCmwLDJe0Y+FGEfEa8JqkXdOswwoW30Z2f3OVdOxNJa1RYmazqvHEE0/w/ivP5R3DrCyt3iOMiKWS7k0PoNwCfB/YCZhLNlr7fkS8KGkpsCI9DDIRGA9cm0Z2t9L2SGwo8D1J7wNvAU0jwjHAjcBiYCawZht535f0U7Li/BzweBvHbXImcIWkR4HpwP8VWec6mp07WeG9g+z+6POSjiUb0Ta/XHw08EdJARSO+C4he+J2trKbiovJnoI1M7Mu4g/d7mb8odtWjYYOHcqMhUv55KFj845iPZg/dNvMzKwT+LNGzaxiDQ0NNDTA+RX8xG6WFxdCM6vY+eefn3cEs7L50qiZmdU0F0Izq9jhhx/O4YcfnncMs7L40qiZVezZZ59lxsKl/KMbfcxaJU8gWs/iEaGZmdU0F0IzM6tpLoRmZlbT2lUIJZ0oab6kyzorUIk53KHerIrstNNOrPapwXnHMCtLex+WOQEYFhHPFs6UtHJEfNBxsdplf7IO9WeXukF78lbSob6c7cy6o3POOYcrutGDMmaFSh4Rpga4g4BbUhf6MyRNknQvMElSnaRrU7f1ByXtkrZbQ9IfJT2QRm1fLbJvd6h3h3ozs1yUXAgj4jjgeWCPiPhVmr0F2QjxG8CvgV9FxPbAgWSdFQB+BNwVETsAewDjirQaaupQ30DWw3BOiZluBi5Ox92jhE1WTQ1uz2s2v6lD/ZZkXSaKtYEv7FA/LJ3H+mQd6kdI2oOsQ/3Rqf1UoUvJ+g5u02z+vzrUkzU4/qakTzc/sKRRkmZKmrl48eISTtOsax144IEsvu7neccwK0ulv0d4Q0S8k6aHAVtk3YQAWFvSmmRd2L/SNDoia5Y7EJhfsJ8HydoUrQJcHxFzKszVktY61B8AWYd6Sa12qCfrk9jUof4GSd8k62l4cokd6oen6T2BrSUdlN73JetQ/3ThPiJiAjABsu4TpZ6sWVdZunQpK955I+8YZmWptBAW9hlcCRgSEcsLV0h99g6MiAUt7SQipknaHdiHrJ/f/0TEn6mdDvW3dXwkMzMrRUf++sTtwHeb3khqSJO3Ad9NBRFJ2zbfUO5Q7w71ZmY56ciPWDsR+K2keWm/04DjgLOA84F5qaA9zb8XqKG4Q7071JuZ5cAd6rsZd6i3anTWWWdx3u0L6LfLN/KOUjJ/1mhtkTvUm1ln+vGPf9ytiqBZIXefMLMO4RGWdVceEZpZxYYPH87w4cPbXtGsCnlEaGYVe+edd9peyaxKeURoZmY1zSNCqxn1/lDoTvPiwqUMGdQ/7xhmZfGI0MzMappHhGZWsd6b7MC++2yedwyzsrgQmlnF+u54AKNH+9cnrHuqyUujkiYWdHwonN/u7vGSNpB0TQvLpkoq+kkGZmZWHWqyEHaU1On++Yj4t6JqVktevHwMQ4cOzTuGWVlqohBKOjJ1gJ8raVKavbuk6ZIWtjA6XF3Spakb/UOp8S6SRkq6QdJdwJ2Fo0hJvSVdKWm+pOuA3gX721PSfZJmS5qcejUiaaykx1K+czv9i2FmZh/T4+8RStoSOB3YOSKWSFoX+B9gfbJmu4OBG4Dmlze/DUREbCVpMHC7pE3Tsu2ArSPiFUn1BdscDyyLiM0lbQ3MThkGpAzDIuJtST8ATpH0W7L2UoMjIlIT32LnMAoYBTBw4MBKvhxmZtZMLYwIvwBMjoglABHxSpp/fUR8GBGPAesV2W5X4C9pm8eBfwJNhfCOgv0U2r1gm3nAvDR/CLAFcK+kOcBRwMbA68By4A+SDgCWFTuBiJgQEY0R0VhXV1fyiZuZWdt6/IiwFe8WTKud27a3073Iiue/fTy/pB2ALwIHAd8hK9xmZtZFamFEeBdwsKT+AOnSaCnuIXWTT5dEBwIL2timsNP954Ct0/wZwC6SPpOWrZG60a8J9I2Im4GTgW1KPiuzKrLG4N045JBD8o5hVpYePyKMiEcl/Qy4W9IK4KESNx0PXCTpYeADYGREvJs1km/RRcClkuYD84FZKcNiSSOBKyStltY9HXgT+Kuk1clGjae07+zMqsNa2+3DCSf49wite3KH+m7GHerL588a7Twfvr+cx88aTp8+ffKOYlaUO9SbWad6efIZ7L333nnHMCuLC6GZmdW0Hn+P0KzJorG+h9VZhs4Yl3cEs7J5RGhmZjXNhdDMzGqaL42aWcVGjhzJ6Mlz/WRuC3xZvrp5RGhmFRs5ciRrbjUs7xhmZXEhNLOKLVmyhBXLXs87hllZXAjNrGIHHXQQi68/J+8YZmVxITQzs5rWbQuhpEskbVFk/khJF1aw37cqS2ZmZt1JVTw1quyTrBURH5a6TUT8ZydGypWkXhGxIu8cZma1ILcRoaR6SQsk/Rl4BNhI0vckPShpnqQz03prSLpJ0lxJj0gakeZPldSYpo+W9ISkB4BdCo4xUdJBBe/fSn+uKelOSbMlPSzpq21kbSnDotR9HkmNkqam6TpJd0h6NI1c/1mw3vWSZqVlowqzSTpP0lxgp4q/wGZmVpK8R4SfBY6KiBmS9kzvdyBrSXSDpN2BOuD5iNgHQFLfwh1IWh84E/g8Wcf3v9N2q6XlwNci4o1UoGZIuiFabsWxV2sZivhv4K6IOEfSXsCxBcuOiYhXJPUGHpR0bUQsBdYA7o+IU5vvLBXMUQADBw5s49BmXe/444/n0ctn5x3DrCx53yP8Z0TMSNN7ptdDwGxgMFlhfBj4kqRfSNotIpo/o70jMDUiFkfEe8BVJRxXwM8lzQOmAJ8C1mtl/bYyNLcrcCVARNwKvFqw7MQ06psBbJTOEWAFcG2xnUXEhIhojIjGurq6Ng5t1vVGjBjBGpvvnncMs7LkXQjfLpgWcE5ENKTXZyLiDxHxBLAdWTE6W9JP2rH/D0jnKGklYNU0/zCykebnI6IBeAlYvaWdtJLhX/tvbft/naA0FBgG7BQR25AV/abtlvu+oHVXzzzzDB+8sTjvGGZlybsQFroNOEbSmgCSPiXpE5I2AJZFxF+AcWQFqdD9wH9I6i9pFeDggmWLyC6ZAnwFWCVN9wVejoj3Je0BbNxasFYyFO7/wIJN7gUOSdvuCaxTcNxXI2KZpMHAkNaOa9ZdHHHEESy58by8Y5iVJe97hP8SEbdL2hy4L3uIlLeAw4HPAOMkfQi8DxzfbLsXJJ0B3Ae8BswpWPx74K/pUuStfDQCvQz4m6SHgZnA423E26qFDGcCf5B0FjC1YP0zgSskHZFyvQi8mTIcJ2k+sIDs8qiZmeVILT8fYuWStBqwIiI+kLQTcFG6BFuxxsbGmDlzZkfsyqzDDB06lBkLl/LJQ8fmHaUq+UO38ydpVkQ0FltWNSPCHmYgcHW6L/ke8M2c85iZWQtcCDtBRDwJbJt3DrOuNGRQf6Z65GPdkAuhmVXs1FP/7ddfzboNF0Izq9h+++2XdwSzslXTr0+YWTe1YMECFixYkHcMs7J4RGhmFfvWt77VqU+N+qlL60weEZqZWU1zITQzs5rmQmhmZjWtZgqhpOMkHZmmR6bPD21p3Z9KGtbZOZrNr5f0SGcc08zMWlYzD8tExMUFb0eSNQN+vvl6qTt8ezpcVJLDrEc4/fTTOeyS+/OOYVaWHjkilHRk6nI/V9KkNO8MSaNTx/pG4DJJcyT1Tp3mfyFpNnBwYWd7SdtLmp729YCktZodq8Vu963lSNOfT8vmAt/umq+OWccbNmwYvesb8o5hVpYeNyKUtCVwOrBzRCyRtG7h8oi4RtJ3gNERMTNtA7A0IrZL7/dKf65K1uh3REQ8KGlt4J1mhyza7R7YorUcyaXAdyJimqRxrZyTO9RbVZszZw7vvbSQVdcblHcUs3briSPCLwCTI2IJQES8UuJ2xTrbbwa8EBEPpn29EREfNFunpW73reaQ1A/oFxHT0qxJLQVzh3qrdieddBKv3Dkh7xhmZelxI8IKvN32KkUVdrt/X9IiSuhWb2Zm1aEnjgjvIrvP1x+ghUuSbwJrFZnf3AJgfUnbp32tJan5Dw8tdbtvNUdEvAa8JmnXNOuwEvKYmVkH63Ejwoh4VNLPgLslrQAeIntKtNBE4GJJ7wA7tbKv9ySNAC6Q1Jvs/uAw4K2C1Yp2uy8xx9HAHyUFcHsZp2tmZhVyh/puxh3qrRp1dod6f9aoVaq1DvU98dKomXWxn//85/Tb/ai8Y5iVpcddGjWzrrfzzjvz4s475x3DrCweEZpZxaZPn8706dPzjmFWFo8Izaxip512GgBTp07NN4hZGTwiNDOzmuYRoXWo+jE35R3BcvDiwqUMGdQ/7xhmZfGI0MzMapoLoZmZ1TRfGjWziq37xVGc/1+75R3DrCxVPyIspXN7WufQgveNkn6TpkdKurAT8xXtZi9pqKQb0/RXJI1J0/tL2qKz8pjlYdX1BtHQ0JB3DLOyVH0hLFE98K9CGBEzI+LErjhwRPwkIqa0sc4NEdH02VP7k/UqNOsx3lk0hylTWv1nYFa1urwQShor6dsF75s6x0vSOEmPpE7vI4psWy/pntQNfrakpo+yGAvsljrOn1w4Gmu2fZ2kayU9mF67tOMYSPpByjZX0tg0r7Cb/V6SHk+d7g8o2G6kpAvTvr4CjEtZN0nrNq332cL3Zt3F69Ov5Oyzz847hllZ8rhHeBVwPvDb9P4Q4MtkhaMB2AYYADwoaVqzbV8GvhQRyyV9FrgCaATGkHWc3xeyy5ItHPvXwK8i4h+SBgK3AZuXcgxJw4GvAjtGxLLmbZUkrQ78nqwh71MUafQbEdNT9/obI+KatN3rkhoiYg5ZN4pLm2/nDvVmZp2ny0eEEfEQ8AlJG0jaBng1Ip4BdgWuiIgVEfEScDewfbPNVwF+n1oeTab9lxiHARdKmgPcAKwtac0SjzEMuDQilqXzeKXZdoOBpyPiychaevylxEyXAEdL6gWMAC5vvoI71JuZdZ68nhqdDBwEfJIiI6dWnAy8RDZqXAlY3s7jrgQMiYjWtqv0GO11LfDfZI18Z0XE0k4+npmZFcjrYZmrgK+TFcPJad49wAhJvSTVAbsDDzTbri/wQkR8CBwB9ErzS+04fzvw3aY3khqKrNPSMe4gG7n1Sduu22y7x4F6SZuk999oIcPHsqaifBtwEUUui5qZWefKpRBGxKNkxeC5iHghzb4OmAfMJRsdfT8iXmy26XjgKElzyS5Fvp3mzwNWpIdYTm7l0CeS3e+bJ+kx4Lgi6xQ9RkTcSnY5dWa6tDq62TktJ7uPd1N64OXlFjJcCXxP0kMFRfMy4EPcpd66qf5f/g6/+93v8o5hVhZ3qK8CkkYDfSPix22tW+0d6v1Zo7XLXeStmrlDfRWTdB1wJNkTrWbd0rKn7udvf/tb3jHMyuJCmLOI+FpEbB0RS/LOYlauNx64jvPOOy/vGGZl8WeNWofy5bHaNHTGuLwjmJXNI0IzM6tpLoRmZlbTfGnUrAPV6lOz7lBv3ZlHhGZWsQH7nsqkSZPyjmFWFhdCM6vYymvXsdFGG+Udw6wsLoRmVrG350/jqqva87HBZtWjWxRCSSdKmi/pssJu7xXus2jPwjL24w71VvPefOhmLrroorxjmJWluzwscwIwLCKeTe9vyDNMoYj4SQnr3MBHmfcHbgQe68RYZmZWoqofEUq6GBgE3JK6z4+UdGFa9ldJR6bpb0m6LE3vKem+1GF+clPPwZY6yDc7njvUm5nVkKofEUbEcZL2AvaIiCWSRhYsHgXcK+lp4FRgiKQBwOlkI8i3Jf0AOEXSL2mjg3xSdR3qzcys81T9iLA1qZP9T4C/A6emrvFDyLrK35vaJR0FbEzpHeSrrkO9pFGSZkqauXjx4hJ3a2ZmpejWhTDZClgKbJDeC7gjIhrSa4uIOLYd+yvsUN8IrNqhaf/dtcBwYF9a6FAfERMiojEiGuvq6jo5jln71e3/Q6655pq8Y5iVpVsXQkk7kBWRbYHRkj4NzAB2kfSZtM4akjal9A7y7lBv1k69+vRlwIABeccwK0u3LYSSViO7J3dMRDxPdo/wj8ASYCRwhaR5wH3A4HZ0kHeHerN2euvhKUycODHvGGZlcYf6KtCTOtTXupr9rNHLxzBkUH+mTp2adxSzolrrUF/1T432dKlD/SZkT5uamVkXcyHMWUR8Le8MZma1zIXQrAMtGrtP3hFy4Q711p1124dlzMzMOoJHhGZWsZtvvjnvCGZlcyE0s4r16dMn7whmZfOlUTOr2Pjx4xk/fnzeMczK4kJoZhW7+uqrufrqq/OOYVYWF0IzM6tpLoRmZlbTXAjNzKymuRCamVlN84dudzOSFgP/rGAXA8g6dFQb52qfasxVjZnAudqrp+baOCKKNnR1Iawxkma29AnseXKu9qnGXNWYCZyrvWoxly+NmplZTXMhNDOzmuZCWHsm5B2gBc7VPtWYqxozgXO1V83l8j1CMzOraR4RmplZTXMhNDOzmuZC2ANJWlfSHZKeTH+u08J6R6V1npR0VMH8b0h6WNI8SbdKGlAluVaVNEHSE5Iel3RgNeQqWH6DpEfyziSpj6Sb0tfoUUljOyDPXpIWSHpK0pgiy1eTdFVafr+k+oJlP0zzF0j6cqVZOiKXpC9JmpW+z2dJ+kI15CpYPlDSW5JGV0suSVtLui99Tz0safU8M0laRdKfUpb5kn5YdoiI8KuHvYBfAmPS9BjgF0XWWRdYmP5cJ02vQ9aj8mVgQMG+zsg7V1p2JnB2ml6pKWPeudLyA4DLgUfyzgT0AfZI66wK3AMMryBLL+B/gUFpf3OBLZqtcwJwcZr+OnBVmt4irb8a8Om0n14d9DWqJNe2wAZp+nPAcx2RqdJcBcuvASYDo6shF9n/C/OAbdL7/h3x91hhpkOBK9N0H2ARUF9ODo8Ie6avAn9K038C9i+yzpeBOyLilYh4FbgD2AtQeq0hScDawPNVkAvgGOAcgIj4MCI66tMvKsolaU3gFODsDspTUaaIWBYRfweIiPeA2cCGFWTZAXgqIham/V2Z8rWU9xrgi+n756tk/1m9GxFPA0+l/XWEsnNFxEMR0fR9/SjQW9JqeecCkLQ/8HTK1ZEqybUnMC8i5gJExNKIWJFzpiD7f2ploDfwHvBGOSFcCHum9SLihTT9IrBekXU+BTxT8P5Z4FMR8T5wPPAwWQHcAvhD3rkk9Uvvz5I0W9JkScW279JcTZmA84BlHZSnIzIBkL5u+wF3VpClzeMUrhMRHwCvk40aStk2j1yFDgRmR8S7eedKP1T9gOzqR0er5Ou1KRCSbkv//r5fBZmuAd4GXgD+Dzg3Il4pJ8TK5Wxk+ZM0BfhkkUU/KnwTESGp5N+RkbQKWSHcluxS2wXADylxtNNZuci+VzcEpkfEKZJOAc4Fjsgzl6QGYJOIOLn5fZ68MhXsf2XgCuA3EbGwvdvXAklbAr8gG/FUgzOAX0XEW2mAWC1WBnYFtif7ge9OSbMiopIfsCq1A7AC2IDslsA9kqaU873uQthNRcSwlpZJeknS+hHxgqT1ye75NfccMLTg/YbAVKAh7f9/076uJrtHlXeupWT/AP9fmj8ZOLYKcu0ENEpaRPbv6ROSpkbEUNrQiZmaTACejIjz28rShueAjZod57kW1nk2FeC+ZH9npWybRy4kbQhcBxzZ9P1eBbl2BA6S9EugH/ChpOURcWHOuZ4FpjXdjpB0M7AdlV1pqDTTocCt6SrWy5LuBRrJfoBvn0pvdvpVfS9gHB9/0OKXRdZZl+w+xDrp9XSatwHZpYa6tN5ZwHl550rLrgS+kKZHApOrIVfBOvV03MMylX6tzgauBVbqgCwrp/9cPs1HDzRs2Wydb/PxBxquTtNb8vGHZRbScQ/LVJKrX1r/gI7I0lG5mq1zBh37sEwlX691yO4190n7mQLsk3OmHwCXpuk1gMeArcvK0dHfBH7l/yK7fn4n8GT6hm36z7ERuKRgvWPIHl54Cji6YP5xwHyyp8T+BvSvklwbA9NSrjuBgdWQq2B5PR1XCMvORPZTdaS/wznp9Z8V5tkbeILsCb8fpXk/Bb6SplcnG6U/BTwADCrY9kdpuwVU8PRqR+YCTie7vzSn4PWJvHM128cZdGAh7IC/x8PJHuB5hCI/mOXwd7hmmv8oWRH8XrkZ/BFrZmZW0/zUqJmZ1TQXQjMzq2kuhGZmVtNcCM3MrKa5EJqZWU1zITQzs5rmQmhmZjXt/wPMUvTAHvIP+AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "diff = preds_corr['data']['diffs'][0]\n",
    "print(f\"Diff coeff: {preds_corr['data']['diff_coeffs']}\")\n",
    "plt.barh(white_df.columns[:-1], diff)\n",
    "plt.xlim((-1.1*np.max(np.abs(diff)), 1.1*np.max(np.abs(diff))))\n",
    "plt.axvline(0, linestyle='--', color='black')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that the detector was able to discriminate the corrupted (red) wine samples from the reference (white) samples by noting that on average reference samples (coeff < 0) typically contain more sulfur dioxide and residual sugars but have less sulphates and chlorides and have lower pH and volatile and fixed acidity."
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "ffba93b5284319fb7a107c8eacae647f441487dcc7e0323a4c0d3feb66ea8c5e"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}