notebooks/unet_example.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "9d13341e",
   "metadata": {},
   "source": [
    "# U-Net example\n",
    "In this notebook we will give a minimal example on how to use U-Net to train/test/predict.\n",
    "\n",
    "First open up the shell/terminal you used for installation and activate the virtual environment you [installed](https://github.com/CSDGroup/aisegcell#installation). If you want to run this notebook interactively, you will have to manually `pip install jupyterlab` in your shell and check out [the documentation](https://docs.jupyter.org/en/latest/) on how to use it."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf97de71",
   "metadata": {},
   "source": [
    "## Generate sample data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4f61a2ae",
   "metadata": {},
   "outputs": [],
   "source": [
    "# install pooch if necessary\n",
    "!pip install pooch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "94e8013f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/Users/schidani/Library/Caches/aisegcell\n"
     ]
    }
   ],
   "source": [
    "import pooch\n",
    "\n",
    "import pandas as pd\n",
    "\n",
    "# download sample data\n",
    "THATCHER = pooch.create(\n",
    "    path=pooch.os_cache('aisegcell'),\n",
    "    base_url='https://github.com/CSDGroup/aisegcell/raw/521be0b66d497791d82e75c8211ac62cb31f6a2e/images/',\n",
    "    registry={\n",
    "        'bf1.png': (\n",
    "            '06047311573206ee8c31716ac0e25116a18159e48c1201c7d8b3cdbd398fc2b3'\n",
    "        ),\n",
    "        'mask1.png': (\n",
    "            'dae2999292a623626f900bdb1ed854bf9e33b342ac50dd5633eef7adb5021db1'\n",
    "        ),\n",
    "        'bf2.png': (\n",
    "            '90bf8dc0acb37f39141dd92be79d85f374e3929bb0aa0546e521980749b807d2'\n",
    "        ),\n",
    "        'mask2.png': (\n",
    "            'de488a87cee7c48fe32e8b86833498a25235a56d5ee8ced5d470d31a6c846cd7'\n",
    "        ),\n",
    "    }\n",
    ")\n",
    "\n",
    "_ = THATCHER.fetch('bf1.png')\n",
    "_ = THATCHER.fetch('bf2.png')\n",
    "_ = THATCHER.fetch('mask1.png')\n",
    "_ = THATCHER.fetch('mask2.png')\n",
    "\n",
    "# This directory will be different on every machine --> adapt subsequent commands with your path\n",
    "print(THATCHER.abspath)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "e10c55b7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bf</th>\n",
       "      <th>mask</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>/Users/schidani/Library/Caches/aisegcell/bf1.png</td>\n",
       "      <td>/Users/schidani/Library/Caches/aisegcell/mask1...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>/Users/schidani/Library/Caches/aisegcell/bf2.png</td>\n",
       "      <td>/Users/schidani/Library/Caches/aisegcell/mask2...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                 bf   \n",
       "0  /Users/schidani/Library/Caches/aisegcell/bf1.png  \\\n",
       "1  /Users/schidani/Library/Caches/aisegcell/bf2.png   \n",
       "\n",
       "                                                mask  \n",
       "0  /Users/schidani/Library/Caches/aisegcell/mask1...  \n",
       "1  /Users/schidani/Library/Caches/aisegcell/mask2...  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# we will use the aisegcell_generate_list command to generate an input file for U-Net with the downloaded files\n",
    "# we will search all stored images at `THATCHER.abspath` (see above, will be different on every system)\n",
    "# NOTE: prepending a command with \"!\" executes the command in the shell/terminal and not with python\n",
    "# NOTE: use linebreaks ('\\') for readability\n",
    "!aisegcell_generate_list \\\n",
    "--bf \"/Users/schidani/Library/Caches/aisegcell/*bf*.png\" \\\n",
    "--mask \"/Users/schidani/Library/Caches/aisegcell/*mask*.png\" \\\n",
    "--out /Users/schidani/Library/Caches/aisegcell/ \\\n",
    "--prefix example\n",
    "\n",
    "# let's have a look at the input file\n",
    "input_df = pd.read_csv('/Users/schidani/Library/Caches/aisegcell/example_paths.csv')\n",
    "\n",
    "input_df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1fd89fb4",
   "metadata": {},
   "source": [
    "## Training\n",
    "We have successfully generated the necessary input file and will now move on to train U-Net."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "a5f21726",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GPU available: False, used: False\n",
      "TPU available: False, using: 0 TPU cores\n",
      "IPU available: False, using: 0 IPUs\n",
      "\n",
      "  | Name | Type     | Params\n",
      "----------------------------------\n",
      "0 | unet | UNet_rec | 10.2 M\n",
      "----------------------------------\n",
      "10.2 M    Trainable params\n",
      "0         Non-trainable params\n",
      "10.2 M    Total params\n",
      "40.898    Total estimated model params size (MB)\n",
      "Epoch 0:  50%|████████        | 1/2 [00:07<00:07,  7.74s/it, loss=1.21, v_num=0]\n",
      "Validating: 0it [00:00, ?it/s]\u001b[A\n",
      "Validating:   0%|                                         | 0/1 [00:00<?, ?it/s]\u001b[A\n",
      "Epoch 0: 100%|████████████████| 2/2 [00:15<00:00,  7.56s/it, loss=1.21, v_num=0]\u001b[A\n",
      "Epoch 1:  50%|████████        | 1/2 [00:07<00:07,  7.74s/it, loss=1.19, v_num=0]\u001b[A\n",
      "Validating: 0it [00:00, ?it/s]\u001b[A\n",
      "Validating:   0%|                                         | 0/1 [00:00<?, ?it/s]\u001b[A\n",
      "Epoch 1: 100%|████████████████| 2/2 [00:15<00:00,  7.56s/it, loss=1.19, v_num=0]\u001b[A\n",
      "Epoch 1: 100%|████████████████| 2/2 [00:15<00:00,  7.74s/it, loss=1.19, v_num=0]\u001b[A\n"
     ]
    }
   ],
   "source": [
    "# Have a look at the README.md documentation for detailed explanations of the parameters\n",
    "!aisegcell_train \\\n",
    "--data /Users/schidani/Library/Caches/aisegcell/example_paths.csv \\\n",
    "--data_val /Users/schidani/Library/Caches/aisegcell/example_paths.csv \\\n",
    "--output_base_dir /Users/schidani/Library/Caches/aisegcell/training \\\n",
    "--devices cpu \\\n",
    "--epochs 2 \\\n",
    "--batch_size 2 \\\n",
    "--shape 128 \\\n",
    "--receptive_field 64 \\\n",
    "--log_frequency 1 \\\n",
    "--bilinear"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "aef51072",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m\u001b[34mcheckpoints\u001b[m\u001b[m  hparams.yaml metrics.csv\n",
      "best-f1-epoch=0-step=0.ckpt   best-loss-epoch=1-step=1.ckpt\n",
      "best-iou-epoch=0-step=0.ckpt  latest-epoch=1-step=1.ckpt\n"
     ]
    }
   ],
   "source": [
    "# let's have a look at the output folder (--output_base_dir)\n",
    "# NOTE: the folder {YYMMDD}_Unet_{ID} will be different for every training\n",
    "!ls /Users/schidani/Library/Caches/aisegcell/training/230414_Unet_19715/lightning_logs/version_0/\n",
    "\n",
    "# model weights are saved in the \"checkpoints\" directory\n",
    "!ls /Users/schidani/Library/Caches/aisegcell/training/230414_Unet_19715/lightning_logs/version_0/checkpoints/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc1bc5c8",
   "metadata": {},
   "source": [
    "## Testing\n",
    "We will now use one of the models we generated during training for testing. Testing requires ground truth segmentations for your input images to compare the predicted segmentation masks to. Testing will compute several metrics to quantify the (dis-)similarity between prediction and ground truth."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "8e5fd0b2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GPU available: False, used: False\n",
      "TPU available: False, using: 0 TPU cores\n",
      "IPU available: False, using: 0 IPUs\n",
      "Testing: 0it [00:00, ?it/s]/Users/schidani/.pyenv/versions/3.8.6/envs/aisegcell/lib/python3.8/site-packages/torchvision/utils.py:50: UserWarning: range will be deprecated, please use value_range instead.\n",
      "  warnings.warn(warning)\n",
      "Testing: 100%|████████████████████████████████████| 2/2 [00:12<00:00,  7.12s/it]--------------------------------------------------------------------------------\n",
      "DATALOADER:0 TEST RESULTS\n",
      "{'f1': 0.0,\n",
      " 'f1_epoch': 0.0,\n",
      " 'fn': 2.5,\n",
      " 'fn_epoch': 2.5,\n",
      " 'fp': 628.5,\n",
      " 'fp_epoch': 628.5,\n",
      " 'inaccurate_masks': 0.0,\n",
      " 'inaccurate_masks_epoch': 0.0,\n",
      " 'iou': 0.01586497575044632,\n",
      " 'iou_big': 0.0,\n",
      " 'iou_big_epoch': 0.0,\n",
      " 'iou_epoch': 0.01586497575044632,\n",
      " 'iou_small': 0.01586497575044632,\n",
      " 'iou_small_epoch': 0.01586497575044632,\n",
      " 'loss_test': 0.7539274096488953,\n",
      " 'loss_test_epoch': 0.7539274096488953,\n",
      " 'merges': 0.0,\n",
      " 'merges_epoch': 0.0,\n",
      " 'splits': 0.0,\n",
      " 'splits_epoch': 0.0,\n",
      " 'tp': 0.0,\n",
      " 'tp_epoch': 0.0}\n",
      "--------------------------------------------------------------------------------\n",
      "Testing: 100%|████████████████████████████████████| 2/2 [00:12<00:00,  6.44s/it]\n"
     ]
    }
   ],
   "source": [
    "# Have a look at the README.md documentation for detailed explanations of the parameters\n",
    "# NOTE: you must generate a trained model (.ckpt file, e.g. training above) for aisegcell_test to work\n",
    "!aisegcell_test \\\n",
    "--data /Users/schidani/Library/Caches/aisegcell/example_paths.csv \\\n",
    "--model /Users/schidani/Library/Caches/aisegcell/training/230414_Unet_19715/lightning_logs/version_0/checkpoints/latest-epoch\\=1-step\\=1.ckpt \\\n",
    "--output_base_dir /Users/schidani/Library/Caches/aisegcell/testing \\\n",
    "--suffix mask \\\n",
    "--devices cpu"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "34eab092",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "hparams.yaml metrics.csv  \u001b[1m\u001b[34mtest_masks\u001b[m\u001b[m\r\n"
     ]
    }
   ],
   "source": [
    "# let's have a look at the output of aisegcell_test\n",
    "!ls /Users/schidani/Library/Caches/aisegcell/testing/lightning_logs/version_0/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b5df67bd",
   "metadata": {},
   "source": [
    "## Predicting\n",
    "Similar to testing we can use trained models for predicting. In contrast to testing, predicting will only generate segmentation masks. Hence, ground truth segmentations are not required and no (dis-)similarity metrics between prediction and ground truth will be computed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "021f05a6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GPU available: False, used: False\n",
      "TPU available: False, using: 0 TPU cores\n",
      "IPU available: False, using: 0 IPUs\n",
      "/Users/schidani/.pyenv/versions/3.8.6/envs/aisegcell/lib/python3.8/site-packages/pytorch_lightning/trainer/data_loading.py:132: UserWarning: The dataloader, predict_dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 8 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.\n",
      "  rank_zero_warn(\n",
      "Predicting: 0it [00:00, ?it/s]/Users/schidani/.pyenv/versions/3.8.6/envs/aisegcell/lib/python3.8/site-packages/torchvision/utils.py:50: UserWarning: range will be deprecated, please use value_range instead.\n",
      "  warnings.warn(warning)\n",
      "/Users/schidani/.pyenv/versions/3.8.6/envs/aisegcell/lib/python3.8/site-packages/pytorch_lightning/loops/epoch/prediction_epoch_loop.py:141: UserWarning: predict returned None if it was on purpose, ignore this warning...\n",
      "  self._warning_cache.warn(\"predict returned None if it was on purpose, ignore this warning...\")\n",
      "Predicting: 100%|█████████████████████████████████| 2/2 [00:00<00:00,  3.34it/s]\n"
     ]
    }
   ],
   "source": [
    "# Have a look at the README.md documentation for detailed explanations of the parameters\n",
    "# NOTE: you must generate a trained model (.ckpt file, e.g. training above) for aisegcell_predict to work\n",
    "!aisegcell_predict \\\n",
    "--data /Users/schidani/Library/Caches/aisegcell/example_paths.csv \\\n",
    "--model /Users/schidani/Library/Caches/aisegcell/training/230414_Unet_19715/lightning_logs/version_0/checkpoints/latest-epoch\\=1-step\\=1.ckpt \\\n",
    "--output_base_dir /Users/schidani/Library/Caches/aisegcell/predicting \\\n",
    "--suffix mask \\\n",
    "--devices cpu"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "7da3e8d1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "hparams.yaml    \u001b[1m\u001b[34mpredicted_masks\u001b[m\u001b[m\n",
      "bf1mask.png bf2mask.png\n"
     ]
    }
   ],
   "source": [
    "# let's have a look at the output of aisegcell_predict\n",
    "# NOTE: no metrics.csv was computed\n",
    "!ls /Users/schidani/Library/Caches/aisegcell/predicting/lightning_logs/version_0/\n",
    "\n",
    "# content of predicted_masks directory\n",
    "!ls /Users/schidani/Library/Caches/aisegcell/predicting/lightning_logs/version_0/predicted_masks/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6302a15b",
   "metadata": {},
   "source": [
    "## (Optional) Clean-up\n",
    "Finally, let's clean up the `pooch` cache we have cluttered with train/test/predict output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "ed9a4c88",
   "metadata": {},
   "outputs": [],
   "source": [
    "!rm -r /Users/schidani/Library/Caches/aisegcell/training\n",
    "!rm -r /Users/schidani/Library/Caches/aisegcell/testing\n",
    "!rm -r /Users/schidani/Library/Caches/aisegcell/predicting"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}