docs/tutorials/benchmarking.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copyright (c) Microsoft Corporation. All rights reserved.\n",
    "\n",
    "Licensed under the MIT License."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OFXtoHmJClRf"
   },
   "source": [
    "# Benchmarking\n",
    "\n",
    "This tutorial benchmarks the performance of various sampling strategies, with and without caching.\n",
    "\n",
    "It's recommended to run this notebook on Google Colab if you don't have your own GPU. Click the \"Open in Colab\" button above to get started."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "First, we install TorchGeo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install torchgeo"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hC3pauOLChi4",
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "## Imports\n",
    "\n",
    "Next, we import TorchGeo and any other libraries we need."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "gather": {
     "logged": 1629238744113
    },
    "id": "gjFiws-PChi8"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import tempfile\n",
    "import time\n",
    "from typing import Tuple\n",
    "\n",
    "from torch.utils.data import DataLoader\n",
    "\n",
    "from torchgeo.datasets import NAIP, ChesapeakeDE\n",
    "from torchgeo.datasets.utils import download_url, stack_samples\n",
    "from torchgeo.samplers import RandomGeoSampler, GridGeoSampler, RandomBatchGeoSampler"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Datasets\n",
    "\n",
    "For this tutorial, we'll be using imagery from the [National Agriculture Imagery Program (NAIP)](https://www.fsa.usda.gov/programs-and-services/aerial-photography/imagery-programs/naip-imagery/) and labels from the [Chesapeake Bay High-Resolution Land Cover Project](https://www.chesapeakeconservancy.org/conservation-innovation-center/high-resolution-data/land-cover-data-project/). First, we manually download a few NAIP tiles."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_root = tempfile.gettempdir()\n",
    "naip_root = os.path.join(data_root, \"naip\")\n",
    "naip_url = \"https://naipblobs.blob.core.windows.net/naip/v002/de/2018/de_060cm_2018/38075/\"\n",
    "tiles = [\n",
    "    \"m_3807511_ne_18_060_20181104.tif\",\n",
    "    \"m_3807511_se_18_060_20181104.tif\",\n",
    "    \"m_3807512_nw_18_060_20180815.tif\",\n",
    "    \"m_3807512_sw_18_060_20180815.tif\",\n",
    "]\n",
    "for tile in tiles:\n",
    "    download_url(naip_url + tile, naip_root)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we tell TorchGeo to automatically download the corresponding Chesapeake labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "chesapeake_root = os.path.join(data_root, \"chesapeake\")\n",
    "\n",
    "chesapeake = ChesapeakeDE(chesapeake_root, download=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "n6HwpMz7Chi-",
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "## Timing function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "gather": {
     "logged": 1629238744228
    },
    "id": "8-z6_y2xChi-",
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def time_epoch(dataloader: DataLoader) -> Tuple[float, int]:\n",
    "    tic = time.time()\n",
    "    i = 0\n",
    "    for _ in dataloader:\n",
    "        i += 1\n",
    "    toc = time.time()\n",
    "    return toc - tic, i"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "I3pkKYoeChi_",
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "## RandomGeoSampler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "gather": {
     "logged": 1629248963725
    },
    "id": "jPjIZLF7Chi_",
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "outputId": "edcc8199-bd09-4832-e50c-7be8ac78995b",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "296.582683801651 74\n",
      "54.20210099220276 74\n"
     ]
    }
   ],
   "source": [
    "for cache in [False, True]:\n",
    "    chesapeake = ChesapeakeDE(chesapeake_root, cache=cache)\n",
    "    naip = NAIP(naip_root, crs=chesapeake.crs, res=chesapeake.res, cache=cache)\n",
    "    dataset = chesapeake & naip\n",
    "    sampler = RandomGeoSampler(naip, size=1000, length=888)\n",
    "    dataloader = DataLoader(dataset, batch_size=12, sampler=sampler, collate_fn=stack_samples)\n",
    "    duration, count = time_epoch(dataloader)\n",
    "    print(duration, count)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pHqLRDA_ChjB",
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "## GridGeoSampler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "gather": {
     "logged": 1629239313388
    },
    "id": "K67vnCK4ChjC",
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "outputId": "159ce99f-a438-4ecc-d218-9b9e28d02055",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "391.90197944641113 74\n",
      "118.0611424446106 74\n"
     ]
    }
   ],
   "source": [
    "for cache in [False, True]:\n",
    "    chesapeake = ChesapeakeDE(chesapeake_root, cache=cache)\n",
    "    naip = NAIP(naip_root, crs=chesapeake.crs, res=chesapeake.res, cache=cache)\n",
    "    dataset = chesapeake & naip\n",
    "    sampler = GridGeoSampler(naip, size=1000, stride=500)\n",
    "    dataloader = DataLoader(dataset, batch_size=12, sampler=sampler, collate_fn=stack_samples)\n",
    "    duration, count = time_epoch(dataloader)\n",
    "    print(duration, count)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8rwjrOD1ChjD",
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "## RandomBatchGeoSampler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "gather": {
     "logged": 1629249843438
    },
    "id": "v-N2fo6UChjE",
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "outputId": "497f6869-1ab7-4db7-bbce-e943b493ca41",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "230.51380324363708 74\n",
      "53.99923872947693 74\n"
     ]
    }
   ],
   "source": [
    "for cache in [False, True]:\n",
    "    chesapeake = ChesapeakeDE(chesapeake_root, cache=cache)\n",
    "    naip = NAIP(naip_root, crs=chesapeake.crs, res=chesapeake.res, cache=cache)\n",
    "    dataset = chesapeake & naip\n",
    "    sampler = RandomBatchGeoSampler(naip, size=1000, batch_size=12, length=888)\n",
    "    dataloader = DataLoader(dataset, batch_sampler=sampler, collate_fn=stack_samples)\n",
    "    duration, count = time_epoch(dataloader)\n",
    "    print(duration, count)"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "benchmarking.ipynb",
   "provenance": []
  },
  "execution": {
   "timeout": 1200
  },
  "kernel_info": {
   "name": "python38-azureml"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  },
  "nteract": {
   "version": "nteract-front-end@1.0.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}