a_Load_HyP3_Data.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0b333e27-cfbc-46f3-84b5-ee7505b64deb",
   "metadata": {},
   "source": [
    "# A. Load HyP3 SBAS Stack into MintPy\n",
    "\n",
    "*Author: Alex Lewandowski; Alaska Satellite Facility*\n",
    "\n",
    "**Load your previously downloaded HyP3 SBAS stack into MintPy.**\n",
    "\n",
    "- Load an SBAS stack from many GeoTiffs into two HDF5 datasets\n",
    "  - `geometryGeo.h5`\n",
    "  - `ifgramStack.h5`\n",
    " \n",
    "You only need to perform this step once. You will be able to configure and reconfigure the parameters of your time series analysis without having to reload the data.\n",
    "\n",
    "This notebook will provide the option to delete your GeoTiffs once they are loaded by MintPy\n",
    "\n",
    "---\n",
    "\n",
    "<div class=\"alert alert-info\" style=\"display: flex; align-items: center; font-family: 'Times New Roman', Times, serif; background-color: #d1ecf1;\">\n",
    "  <div style=\"display: flex; align-items: center; width: 10%;\">\n",
    "    <a href=\"https://github.com/ASFOpenSARlab/opensarlab_MintPy_Recipe_Book/issues\">\n",
    "      <img src=\"github_issues.png\" alt=\"GitHub logo over the word Issues\" style=\"width: 100%;\">\n",
    "    </a>\n",
    "  </div>\n",
    "  <div style=\"width: 95%;\">\n",
    "    <b>Did you find a bug? Do you have a feature request?</b>\n",
    "    <br/>\n",
    "    Explore GitHub Issues on this Jupyter Book's GitHub repository. Find solutions, add to the discussion, and start a new bug report or feature request: <a href=\"https://github.com/ASFOpenSARlab/opensarlab_MintPy_Recipe_Book/issues\">opensarlab_MintPy_Recipe_Book Issues</a>\n",
    "  </div>\n",
    "</div>\n",
    "\n",
    "<div class=\"alert alert-info\" style=\"display: flex; align-items: center; justify-content: space-between; font-family: 'Times New Roman', Times, serif; background-color: #d1ecf1;\">\n",
    "  <div style=\"display: flex; align-items: center; width: 10%; margin-right: 10px;\">\n",
    "    <a href=\"mailto:uso@asf.alaska.edu\">\n",
    "      <img src=\"ASF_support_logo.png\" alt=\"ASF logo\" style=\"width: 100%\">\n",
    "    </a>\n",
    "  </div>\n",
    "  <div style=\"width: 95%;\">\n",
    "    <b>Have a question related to SAR, ASF data access, or performing SBAS time series analyses with MintPy?</b>\n",
    "    <br/>\n",
    "    Contact ASF User Support: <a href=\"mailto:uso@asf.alaska.edu\">uso@asf.alaska.edu</a>\n",
    "  </div>\n",
    "</div>\n",
    "\n",
    "---\n",
    "\n",
    "## 0. Import Required Software"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4e2f708b-cdbc-4725-938c-eaff9d021848",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "from pprint import pprint\n",
    "import shutil\n",
    "\n",
    "import h5py\n",
    "from ipyfilechooser import FileChooser\n",
    "from mintpy.cli import plot_network\n",
    "import opensarlab_lib as osl"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "971a93c7-a567-4490-8faa-86c9c59086fd",
   "metadata": {},
   "source": [
    "---\n",
    "## 1. Create a Custom Config File Defining the Paths to the GeoTiffs\n",
    "\n",
    "**Select the directory holding your MintPy-ready HyP3 data stack and/or MintPy directory from a previously loaded MintPy SBAS stack**\n",
    "- Click the `Select` button\n",
    "- Navigate to your data directory\n",
    "- Click the `Select` button\n",
    "- Confirm that the desired path appears in green text\n",
    "- Click the `Change` button to alter your selection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6ea08a84-24b4-4222-9014-68fa52a4095a",
   "metadata": {},
   "outputs": [],
   "source": [
    "path = Path.home()\n",
    "fc = FileChooser(path)\n",
    "display(fc)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e814a21-c2ea-4d6f-9c37-9973892bafa3",
   "metadata": {},
   "source": [
    "**Define a project name and create a MintPy directory in which to store files output during our analysis**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a9847c45-1b91-48d5-ad1d-fdf31b646ea7",
   "metadata": {},
   "outputs": [],
   "source": [
    "# define the work directory\n",
    "data_path = Path(fc.selected_path)\n",
    "print(f\"Work directory: {data_path}\")\n",
    "\n",
    "# define the MintPy time-series directory\n",
    "mint_path = data_path / 'MintPy'\n",
    "mint_path.mkdir(exist_ok=True)\n",
    "print(f\"MintPy directory: {mint_path}\")\n",
    "\n",
    "# define a project name\n",
    "proj_name = input(\"Enter a project name: \")\n",
    "custom_config_path = mint_path / f'{proj_name}.txt'\n",
    "print(f\"Custom config path: {custom_config_path}\")\n",
    "\n",
    "#create a directory in which to store plots\n",
    "plot_path = mint_path / \"plots\"\n",
    "plot_path.mkdir(exist_ok=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b989baae-ae2a-4477-a417-600785ec46f0",
   "metadata": {},
   "source": [
    "**Define the paths to the GeoTiffs in a custom config**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6b6ee9eb-2bd4-49e5-b33b-dbd6ef1e18e7",
   "metadata": {},
   "outputs": [],
   "source": [
    "con_comp_path = list(data_path.glob('./*/*_conncomp*.tif'))\n",
    "burst = len(con_comp_path) > 0\n",
    "\n",
    "config = f\"\"\"\n",
    "mintpy.load.processor        = hyp3\n",
    "##---------geometry datasets:\n",
    "mintpy.load.demFile          = {data_path}/*/*_dem*.tif\n",
    "mintpy.load.incAngleFile     = {data_path}/*/*_lv_theta*.tif\n",
    "mintpy.load.azAngleFile      = {data_path}/*/*_lv_phi*.tif\n",
    "mintpy.load.waterMaskFile    = {data_path}/*/*_water_mask*.tif\n",
    "##---------interferogram datasets:\n",
    "mintpy.load.unwFile          = {data_path}/*/*_unw_phase*.tif\n",
    "mintpy.load.corFile          = {data_path}/*/*_corr*.tif\n",
    "\"\"\"\n",
    "\n",
    "if burst:\n",
    "    config += f\"mintpy.load.connCompFile     = {data_path}/*/*_conncomp*.tif\"\n",
    "    \n",
    "print(config)\n",
    "custom_config_path.write_text(config)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e959fd1c-2728-40ba-9f35-d9a4b829b245",
   "metadata": {},
   "source": [
    "---\n",
    "## 2. Load the Data with MintPy\n",
    "\n",
    "The output of the loading step is saved to the `MintPy/inputs` directory:\n",
    "- ifgramStack.h5: \n",
    "- geometryGeo.h5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "21624974-49f1-4b22-a885-b01d0b63c1f7",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!smallbaselineApp.py $custom_config_path --work-dir {mint_path} --dostep load_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f8560435-324a-4e4b-843d-c4b26d7e88fb",
   "metadata": {},
   "outputs": [],
   "source": [
    "inputs_path = mint_path/'inputs'\n",
    "!ls $inputs_path"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b3f47031-effa-418f-9feb-42f3bad303f4",
   "metadata": {},
   "source": [
    "---\n",
    "## 3. Run the Modify Network Step\n",
    "\n",
    "- Identifies and excludes interferograms (i.e. affected by coherence phase-unwrapping errors) before the network inversion"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2f4091e1-858d-40c8-875e-cc4834809cd0",
   "metadata": {},
   "outputs": [],
   "source": [
    "!smallbaselineApp.py $custom_config_path --work-dir {mint_path} --dostep modify_network"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c10dc30c-8599-4cd1-b86e-48f69a00ed01",
   "metadata": {},
   "source": [
    "---\n",
    "## 4. Plot the interferogram network\n",
    "\n",
    "Running **plot_network** gives an overview of the network and the average coherence of the stack. The program creates multiple files as follows:\n",
    "- `ifgramStack_coherence_spatialAvg.txt`: Contains interferogram dates, average coherence temporal and spatial baseline separation.\n",
    "- `Network.pdf`: Displays the network of interferograms on time-baseline coordinates, colorcoded by avergae coherence of the interferograms. \n",
    "- `CoherenceMatrix.pdf` shows the avergae coherence pairs between all available pairs in the stack."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "23dd70d7-ea6d-4249-8d86-2eb6e4e5d934",
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "with osl.work_dir(mint_path):\n",
    "    scp_args = f'{inputs_path}/ifgramStack.h5 --cmap-vlist 0.2 0.7 1.0'\n",
    "    plot_network.main(scp_args.split())\n",
    "    plots = ['pbaseHistory.pdf', 'coherenceHistory.pdf', 'coherenceMatrix.pdf', 'network.pdf']\n",
    "    for p in plots:\n",
    "        if (mint_path/p).exists():\n",
    "            (mint_path/p).rename(f'{plot_path}/{p}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c8769be-3d64-4b43-9032-2bdfd3d252d7",
   "metadata": {},
   "source": [
    "---\n",
    "## 5. Delete Input Data\n",
    "\n",
    "Once MintPy has loaded the data into HDF5 datasets, you may wish to delete the input data to conserve storage space. \n",
    "\n",
    "You can adjust configuration parameters and update your analysis without reloading the data from their original source."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "79e73bc8-e589-48f4-80b5-6132b4d5761e",
   "metadata": {},
   "outputs": [],
   "source": [
    "s1_paths = list(data_path.glob('S1*'))\n",
    "pprint(f'S1 directories: {s1_paths}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71595710-c0cf-4ae9-a98c-a85595939dfd",
   "metadata": {},
   "outputs": [],
   "source": [
    "cleanup_select = osl.select_parameter(\n",
    "    [\n",
    "        'Delete S1 directories',\n",
    "        'Save S1 directories'\n",
    "    ]\n",
    ")\n",
    "print(\"Save or delete input data:\")\n",
    "display(cleanup_select)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bf6c6631-2000-476d-9bdc-9be8ba816d19",
   "metadata": {},
   "outputs": [],
   "source": [
    "if 'Delete' in cleanup_select.value:\n",
    "    for d in s1_paths:\n",
    "        shutil.rmtree(d)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:.local-opensarlab_mintpy_recipe_book]",
   "language": "python",
   "name": "conda-env-.local-opensarlab_mintpy_recipe_book-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}