examples/05_Optimizing_AUROC_Loss_with_DenseNet121_on_CheXpert.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9Qkb6bYy3rOx"
      },
      "source": [
        "# **Optimizing AUROC loss on Chest X-Ray dataset (CheXpert)**\n",
        "\n",
        "**Author**: Zhuoning Yuan\n",
        "\n",
        "**Introduction**\n",
        "\n",
        "In this tutorial, you will learn how to quickly train a DenseNet121 model by optimizing **AUROC** using our novel **`AUCMLoss`** and **`PESG`** optimizer on Chest X-Ray dataset, e.g., [CheXpert](https://stanfordmlgroup.github.io/competitions/chexpert/). After completion of this tutorial, you should be able to use LibAUC to train your own models on your own datasets.\n",
        "\n",
        "**Useful Resources**:\n",
        "* Website: https://libauc.org\n",
        "* Github: https://github.com/Optimization-AI/LibAUC\n",
        "\n",
        "**Reference**:  \n",
        "\n",
        "If you find this tutorial helpful in your work,  please acknowledge our library and cite the following paper:\n",
        "\n",
        "<pre>\n",
        "@inproceedings{yuan2021large,\n",
        "  title={Large-scale robust deep auc maximization: A new surrogate loss and empirical studies on medical image classification},\n",
        "  author={Yuan, Zhuoning and Yan, Yan and Sonka, Milan and Yang, Tianbao},\n",
        "  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},\n",
        "  pages={3040--3049},\n",
        "  year={2021}\n",
        "  }\n",
        "</pre>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hTJ3ca0u4YQ4"
      },
      "source": [
        "# **Installing LibAUC**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "h8iVw1kU3guh"
      },
      "outputs": [],
      "source": [
        "!pip install libauc==1.2.0"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nlD-4SrE4dVW"
      },
      "source": [
        "# **Downloading CheXpert**\n",
        " \n",
        "*   To request dataset access, you need to apply from CheXpert website: https://stanfordmlgroup.github.io/competitions/chexpert/\n",
        "*   In this tutorial, we use the smaller version of dataset with lower image resolution, i.e., *CheXpert-v1.0-small.zip*\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "CcsJ4eoj3VST"
      },
      "outputs": [],
      "source": [
        "!cp /content/drive/MyDrive/chexpert-dataset/CheXpert-v1.0-small.zip /content/\n",
        "!mkdir CheXpert\n",
        "!unzip CheXpert-v1.0-small.zip -d /content/CheXpert/"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OVvrt3ku4qpq"
      },
      "source": [
        "\n",
        "# **Importing LibAUC**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "XGHWer3v4qJo"
      },
      "outputs": [],
      "source": [
        "from libauc.losses import AUCMLoss, CrossEntropyLoss\n",
        "from libauc.optimizers import PESG, Adam\n",
        "from libauc.models import densenet121 as DenseNet121\n",
        "from libauc.datasets import CheXpert\n",
        "\n",
        "import torch \n",
        "from PIL import Image\n",
        "import numpy as np\n",
        "import torchvision.transforms as transforms\n",
        "from torch.utils.data import Dataset\n",
        "from sklearn.metrics import roc_auc_score"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "l2swK5Mo7Kca"
      },
      "source": [
        "# **Reproducibility**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "OiiT5oEp7J3C"
      },
      "outputs": [],
      "source": [
        "def set_all_seeds(SEED):\n",
        "    # REPRODUCIBILITY\n",
        "    torch.manual_seed(SEED)\n",
        "    np.random.seed(SEED)\n",
        "    torch.backends.cudnn.deterministic = True\n",
        "    torch.backends.cudnn.benchmark = False"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G3Bge6KM7lBP"
      },
      "source": [
        "# **Pretraining**\n",
        "* Multi-label classification (5 tasks)   \n",
        "* Adam + CrossEntropy Loss \n",
        "* This step is optional\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 942,
          "referenced_widgets": [
            "ccc0acb27c3349e1a832f9bb352d5a83",
            "9f97ce48dd3e403d8bc7fd73ee2bdbe0",
            "eebacbbde22e41b7b399a4d8cde36a1a",
            "c3922304da024a25b792cdfb020ec909",
            "7d270228ca7b4c7c8e364fcab46e9a16",
            "70dc53c6f3004aa79d59db68fac2ecf4",
            "a3e652a8c7a44563a59d386dc97d27a3",
            "949eded9ab0e4bb28f2b7f5ba1165b91",
            "439218cb55504d499babf5e96d669db2",
            "a3d9fb897afa4af1b590c5f2b94c1a64",
            "042240cecf414a82a0e9aa1f818d2857"
          ]
        },
        "id": "XkKcegEh72VP",
        "outputId": "78293788-8197-4494-b8e5-c2f52c98f6f5"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Multi-label mode: True, Number of classes: [5]\n",
            "------------------------------\n",
            "Found 191027 images in total, 23385 positive images, 167642 negative images\n",
            "Cardiomegaly(C0): imbalance ratio is 0.1224\n",
            "\n",
            "Found 191027 images in total, 61493 positive images, 129534 negative images\n",
            "Edema(C1): imbalance ratio is 0.3219\n",
            "\n",
            "Found 191027 images in total, 12983 positive images, 178044 negative images\n",
            "Consolidation(C2): imbalance ratio is 0.0680\n",
            "\n",
            "Found 191027 images in total, 59583 positive images, 131444 negative images\n",
            "Atelectasis(C3): imbalance ratio is 0.3119\n",
            "\n",
            "Found 191027 images in total, 76899 positive images, 114128 negative images\n",
            "Pleural Effusion(C4): imbalance ratio is 0.4026\n",
            "\n",
            "Multi-label mode: True, Number of classes: [5]\n",
            "------------------------------\n",
            "Found 202 images in total, 66 positive images, 136 negative images\n",
            "Cardiomegaly(C0): imbalance ratio is 0.3267\n",
            "\n",
            "Found 202 images in total, 42 positive images, 160 negative images\n",
            "Edema(C1): imbalance ratio is 0.2079\n",
            "\n",
            "Found 202 images in total, 32 positive images, 170 negative images\n",
            "Consolidation(C2): imbalance ratio is 0.1584\n",
            "\n",
            "Found 202 images in total, 75 positive images, 127 negative images\n",
            "Atelectasis(C3): imbalance ratio is 0.3713\n",
            "\n",
            "Found 202 images in total, 64 positive images, 138 negative images\n",
            "Pleural Effusion(C4): imbalance ratio is 0.3168\n",
            "\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Downloading: \"https://download.pytorch.org/models/densenet121-a639ec97.pth\" to /root/.cache/torch/hub/checkpoints/densenet121-a639ec97.pth\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0.00/30.8M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "ccc0acb27c3349e1a832f9bb352d5a83"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch=0, BatchID=0, Val_AUC=0.5059, Best_Val_AUC=0.5059\n",
            "Epoch=0, BatchID=400, Val_AUC=0.8543, Best_Val_AUC=0.8543\n",
            "Epoch=0, BatchID=800, Val_AUC=0.8586, Best_Val_AUC=0.8586\n",
            "Epoch=0, BatchID=1200, Val_AUC=0.8715, Best_Val_AUC=0.8715\n",
            "Epoch=0, BatchID=1600, Val_AUC=0.8796, Best_Val_AUC=0.8796\n",
            "Epoch=0, BatchID=2000, Val_AUC=0.8680, Best_Val_AUC=0.8796\n",
            "Epoch=0, BatchID=2400, Val_AUC=0.8811, Best_Val_AUC=0.8811\n",
            "Epoch=0, BatchID=2800, Val_AUC=0.8747, Best_Val_AUC=0.8811\n",
            "Epoch=0, BatchID=3200, Val_AUC=0.8740, Best_Val_AUC=0.8811\n",
            "Epoch=0, BatchID=3600, Val_AUC=0.8842, Best_Val_AUC=0.8842\n",
            "Epoch=0, BatchID=4000, Val_AUC=0.8737, Best_Val_AUC=0.8842\n",
            "Epoch=0, BatchID=4400, Val_AUC=0.8550, Best_Val_AUC=0.8842\n",
            "Epoch=0, BatchID=4800, Val_AUC=0.8788, Best_Val_AUC=0.8842\n",
            "Epoch=0, BatchID=5200, Val_AUC=0.8655, Best_Val_AUC=0.8842\n",
            "Epoch=0, BatchID=5600, Val_AUC=0.8662, Best_Val_AUC=0.8842\n"
          ]
        }
      ],
      "source": [
        "# dataloader\n",
        "root = './CheXpert/CheXpert-v1.0-small/'\n",
        "# Index: -1 denotes multi-label mode including 5 diseases\n",
        "traindSet = CheXpert(csv_path=root+'train.csv', image_root_path=root, use_upsampling=False, use_frontal=True, image_size=224, mode='train', class_index=-1)\n",
        "testSet =  CheXpert(csv_path=root+'valid.csv',  image_root_path=root, use_upsampling=False, use_frontal=True, image_size=224, mode='valid', class_index=-1)\n",
        "trainloader =  torch.utils.data.DataLoader(traindSet, batch_size=32, num_workers=2, shuffle=True)\n",
        "testloader =  torch.utils.data.DataLoader(testSet, batch_size=32, num_workers=2, shuffle=False)\n",
        "\n",
        "# paramaters\n",
        "SEED = 123\n",
        "BATCH_SIZE = 32\n",
        "lr = 1e-4\n",
        "weight_decay = 1e-5\n",
        "\n",
        "# model\n",
        "set_all_seeds(SEED)\n",
        "model = DenseNet121(pretrained=True, last_activation=None, activations='relu', num_classes=5)\n",
        "model = model.cuda()\n",
        "\n",
        "# define loss & optimizer\n",
        "CELoss = CrossEntropyLoss()\n",
        "optimizer = Adam(model.parameters(), lr=lr, weight_decay=weight_decay)\n",
        "\n",
        "# training\n",
        "best_val_auc = 0 \n",
        "for epoch in range(1):\n",
        "    for idx, data in enumerate(trainloader):\n",
        "      train_data, train_labels = data\n",
        "      train_data, train_labels  = train_data.cuda(), train_labels.cuda()\n",
        "      y_pred = model(train_data)\n",
        "      loss = CELoss(y_pred, train_labels)\n",
        "      optimizer.zero_grad()\n",
        "      loss.backward()\n",
        "      optimizer.step()\n",
        "        \n",
        "      # validation  \n",
        "      if idx % 400 == 0:\n",
        "         model.eval()\n",
        "         with torch.no_grad():    \n",
        "              test_pred = []\n",
        "              test_true = [] \n",
        "              for jdx, data in enumerate(testloader):\n",
        "                  test_data, test_labels = data\n",
        "                  test_data = test_data.cuda()\n",
        "                  y_pred = model(test_data)\n",
        "                  test_pred.append(y_pred.cpu().detach().numpy())\n",
        "                  test_true.append(test_labels.numpy())\n",
        "            \n",
        "              test_true = np.concatenate(test_true)\n",
        "              test_pred = np.concatenate(test_pred)\n",
        "              val_auc_mean =  roc_auc_score(test_true, test_pred) \n",
        "              model.train()\n",
        "\n",
        "              if best_val_auc < val_auc_mean:\n",
        "                 best_val_auc = val_auc_mean\n",
        "                 torch.save(model.state_dict(), 'ce_pretrained_model.pth')\n",
        "\n",
        "              print ('Epoch=%s, BatchID=%s, Val_AUC=%.4f, Best_Val_AUC=%.4f'%(epoch, idx, val_auc_mean, best_val_auc ))\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fP4adNO97YBV"
      },
      "source": [
        "# **Optimizing AUCM Loss**\n",
        "*   Binary Classification\n",
        "*   PESG + AUCM Loss\n",
        "\n",
        "Alternatively, you can also try other losses in this task, e.g., [CompositionalAUCLoss](https://github.com/Optimization-AI/LibAUC/blob/main/examples/09_Optimizing_CompositionalAUC_Loss_with_ResNet20_on_CIFAR10.ipynb).\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oiOrKatkSCNJ",
        "outputId": "89d546e7-d016-4443-fed9-ac69073d3636"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Upsampling Cardiomegaly...\n",
            "Upsampling Consolidation...\n",
            "------------------------------\n",
            "Found 227395 images in total, 77866 positive images, 149529 negative images\n",
            "Edema(C1): imbalance ratio is 0.3424\n",
            "------------------------------\n",
            "------------------------------\n",
            "Found 202 images in total, 42 positive images, 160 negative images\n",
            "Edema(C1): imbalance ratio is 0.2079\n",
            "------------------------------\n",
            "Epoch=0, BatchID=0, Val_AUC=0.6918, lr=0.0500\n",
            "Epoch=0, BatchID=400, Val_AUC=0.9155, lr=0.0500\n",
            "Epoch=0, BatchID=800, Val_AUC=0.8869, lr=0.0500\n",
            "Epoch=0, BatchID=1200, Val_AUC=0.9144, lr=0.0500\n",
            "Epoch=0, BatchID=1600, Val_AUC=0.9165, lr=0.0500\n",
            "Epoch=0, BatchID=2000, Val_AUC=0.9052, lr=0.0500\n",
            "Epoch=0, BatchID=2400, Val_AUC=0.9237, lr=0.0500\n",
            "Epoch=0, BatchID=2800, Val_AUC=0.9055, lr=0.0500\n",
            "Epoch=0, BatchID=3200, Val_AUC=0.9350, lr=0.0500\n",
            "Epoch=0, BatchID=3600, Val_AUC=0.9231, lr=0.0500\n",
            "Epoch=0, BatchID=4000, Val_AUC=0.9080, lr=0.0500\n",
            "Epoch=0, BatchID=4400, Val_AUC=0.9174, lr=0.0500\n",
            "Epoch=0, BatchID=4800, Val_AUC=0.8854, lr=0.0500\n",
            "Epoch=0, BatchID=5200, Val_AUC=0.8899, lr=0.0500\n",
            "Epoch=0, BatchID=5600, Val_AUC=0.9067, lr=0.0500\n",
            "Epoch=0, BatchID=6000, Val_AUC=0.8842, lr=0.0500\n",
            "Epoch=0, BatchID=6400, Val_AUC=0.9214, lr=0.0500\n",
            "Epoch=0, BatchID=6800, Val_AUC=0.9003, lr=0.0500\n",
            "Reducing learning rate to 0.00500 @ T=7107!\n",
            "Updating regularizer @ T=7107!\n",
            "Epoch=1, BatchID=0, Val_AUC=0.9095, lr=0.0050\n",
            "Epoch=1, BatchID=400, Val_AUC=0.9295, lr=0.0050\n",
            "Epoch=1, BatchID=800, Val_AUC=0.9357, lr=0.0050\n",
            "Epoch=1, BatchID=1200, Val_AUC=0.9332, lr=0.0050\n",
            "Epoch=1, BatchID=1600, Val_AUC=0.9381, lr=0.0050\n",
            "Epoch=1, BatchID=2000, Val_AUC=0.9382, lr=0.0050\n",
            "Epoch=1, BatchID=2400, Val_AUC=0.9324, lr=0.0050\n",
            "Epoch=1, BatchID=2800, Val_AUC=0.9375, lr=0.0050\n",
            "Epoch=1, BatchID=3200, Val_AUC=0.9384, lr=0.0050\n",
            "Epoch=1, BatchID=3600, Val_AUC=0.9372, lr=0.0050\n",
            "Epoch=1, BatchID=4000, Val_AUC=0.9400, lr=0.0050\n",
            "Epoch=1, BatchID=4400, Val_AUC=0.9432, lr=0.0050\n",
            "Epoch=1, BatchID=4800, Val_AUC=0.9436, lr=0.0050\n",
            "Epoch=1, BatchID=5200, Val_AUC=0.9409, lr=0.0050\n",
            "Epoch=1, BatchID=5600, Val_AUC=0.9464, lr=0.0050\n",
            "Epoch=1, BatchID=6000, Val_AUC=0.9387, lr=0.0050\n",
            "Epoch=1, BatchID=6400, Val_AUC=0.9369, lr=0.0050\n",
            "Epoch=1, BatchID=6800, Val_AUC=0.9385, lr=0.0050\n",
            "Best Val_AUC is 0.9464\n"
          ]
        }
      ],
      "source": [
        "# parameters\n",
        "class_id = 1 # 0:Cardiomegaly, 1:Edema, 2:Consolidation, 3:Atelectasis, 4:Pleural Effusion \n",
        "root = './CheXpert/CheXpert-v1.0-small/'\n",
        "\n",
        "# You can set use_upsampling=True and pass the class name by upsampling_cols=['Cardiomegaly'] to do upsampling. This may improve the performance\n",
        "traindSet = CheXpert(csv_path=root+'train.csv', image_root_path=root, use_upsampling=True, use_frontal=True, image_size=224, mode='train', class_index=class_id)\n",
        "testSet =  CheXpert(csv_path=root+'valid.csv',  image_root_path=root, use_upsampling=False, use_frontal=True, image_size=224, mode='valid', class_index=class_id)\n",
        "trainloader =  torch.utils.data.DataLoader(traindSet, batch_size=32, num_workers=2, shuffle=True)\n",
        "testloader =  torch.utils.data.DataLoader(testSet, batch_size=32, num_workers=2, shuffle=False)\n",
        "\n",
        "# paramaters\n",
        "SEED = 123\n",
        "BATCH_SIZE = 32\n",
        "imratio = traindSet.imratio\n",
        "lr = 0.05 # using smaller learning rate is better\n",
        "epoch_decay = 2e-3\n",
        "weight_decay = 1e-5\n",
        "margin = 1.0\n",
        "\n",
        "# model\n",
        "set_all_seeds(SEED)\n",
        "model = DenseNet121(pretrained=False, last_activation=None, activations='relu', num_classes=1)\n",
        "model = model.cuda()\n",
        "\n",
        "\n",
        "# load pretrained model\n",
        "if True:\n",
        "  PATH = 'ce_pretrained_model.pth' \n",
        "  state_dict = torch.load(PATH)\n",
        "  state_dict.pop('classifier.weight', None)\n",
        "  state_dict.pop('classifier.bias', None) \n",
        "  model.load_state_dict(state_dict, strict=False)\n",
        "\n",
        "\n",
        "# define loss & optimizer\n",
        "loss_fn = AUCMLoss()\n",
        "optimizer = PESG(model, \n",
        "                 loss_fn=loss_fn, \n",
        "                 lr=lr, \n",
        "                 margin=margin, \n",
        "                 epoch_decay=epoch_decay, \n",
        "                 weight_decay=weight_decay)\n",
        "\n",
        "best_val_auc = 0\n",
        "for epoch in range(2):\n",
        "  if epoch > 0:\n",
        "     optimizer.update_regularizer(decay_factor=10)\n",
        "  for idx, data in enumerate(trainloader):\n",
        "      train_data, train_labels = data\n",
        "      train_data, train_labels = train_data.cuda(), train_labels.cuda()\n",
        "      y_pred = model(train_data)\n",
        "      y_pred = torch.sigmoid(y_pred)\n",
        "      loss = loss_fn(y_pred, train_labels)\n",
        "      optimizer.zero_grad()\n",
        "      loss.backward()\n",
        "      optimizer.step()\n",
        "\n",
        "      # validation\n",
        "      if idx % 400 == 0:\n",
        "        model.eval()\n",
        "        with torch.no_grad():    \n",
        "              test_pred = []\n",
        "              test_true = [] \n",
        "              for jdx, data in enumerate(testloader):\n",
        "                  test_data, test_label = data\n",
        "                  test_data = test_data.cuda()\n",
        "                  y_pred = model(test_data)\n",
        "                  test_pred.append(y_pred.cpu().detach().numpy())\n",
        "                  test_true.append(test_label.numpy())\n",
        "              \n",
        "              test_true = np.concatenate(test_true)\n",
        "              test_pred = np.concatenate(test_pred)\n",
        "              val_auc =  roc_auc_score(test_true, test_pred) \n",
        "              model.train()\n",
        "\n",
        "              if best_val_auc < val_auc:\n",
        "                 best_val_auc = val_auc\n",
        "              \n",
        "        print ('Epoch=%s, BatchID=%s, Val_AUC=%.4f, lr=%.4f'%(epoch, idx, val_auc,  optimizer.lr))\n",
        "\n",
        "print ('Best Val_AUC is %.4f'%best_val_auc)"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "background_execution": "on",
      "collapsed_sections": [],
      "name": "05_Optimizing_AUROC_Loss_with_DenseNet121_on_CheXpert.ipynb",
      "provenance": []
    },
    "gpuClass": "standard",
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "ccc0acb27c3349e1a832f9bb352d5a83": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_9f97ce48dd3e403d8bc7fd73ee2bdbe0",
              "IPY_MODEL_eebacbbde22e41b7b399a4d8cde36a1a",
              "IPY_MODEL_c3922304da024a25b792cdfb020ec909"
            ],
            "layout": "IPY_MODEL_7d270228ca7b4c7c8e364fcab46e9a16"
          }
        },
        "9f97ce48dd3e403d8bc7fd73ee2bdbe0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_70dc53c6f3004aa79d59db68fac2ecf4",
            "placeholder": "​",
            "style": "IPY_MODEL_a3e652a8c7a44563a59d386dc97d27a3",
            "value": "100%"
          }
        },
        "eebacbbde22e41b7b399a4d8cde36a1a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_949eded9ab0e4bb28f2b7f5ba1165b91",
            "max": 32342954,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_439218cb55504d499babf5e96d669db2",
            "value": 32342954
          }
        },
        "c3922304da024a25b792cdfb020ec909": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a3d9fb897afa4af1b590c5f2b94c1a64",
            "placeholder": "​",
            "style": "IPY_MODEL_042240cecf414a82a0e9aa1f818d2857",
            "value": " 30.8M/30.8M [00:00&lt;00:00, 154MB/s]"
          }
        },
        "7d270228ca7b4c7c8e364fcab46e9a16": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "70dc53c6f3004aa79d59db68fac2ecf4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a3e652a8c7a44563a59d386dc97d27a3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "949eded9ab0e4bb28f2b7f5ba1165b91": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "439218cb55504d499babf5e96d669db2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "a3d9fb897afa4af1b590c5f2b94c1a64": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "042240cecf414a82a0e9aa1f818d2857": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}