notebook3_How_to_download_UVIT_data_from_ISSDC_Astrobrowse.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ae6f2481-21ed-4c44-a130-19e9b62f49e9",
   "metadata": {},
   "source": [
    "The [ISSDC Astrobrowse website](https://astrobrowse.issdc.gov.in/astro_archive/archive/Home.jsp) is presently the only archive where AstroSat UVIT data can be accessed publicly. You need to have a username and password to log in to the Astrobrowse website. Click on the register link to create login credentials if you do not already possess one. Once you have logged in to the website, search and download your source of interest. For example, we will search for the data of NGC 7252 (also known as Atoms for Peace Galaxy). Please see the GIF movie below. We will find an Observation ID `G07_003T02_9000001564` (PI: Koshy) matching our search criteria. This data is now publicly available for download after the proprietary period. Two publications are associated with this data. See,\n",
    "* George, K., et al. \"Dissecting star formation in the \"Atoms-for-Peace\" galaxy-UVIT observations of the post-merger galaxy NGC7252.\" Astronomy & Astrophysics 614 (2018): A130. \n",
    "* George, K., et al. \"UVIT observations of the star-forming ring in NGC 7252: Evidence of possible AGN feedback suppressing central star formation.\" Astronomy & Astrophysics 613 (2018): L9.\n",
    "\n",
    "![](images/AstroBrowse-fast.gif)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9fd54a12-283d-4f18-83b0-f60b9e7f7396",
   "metadata": {},
   "source": [
    "The data download may take a while to complete (I get ~0.5 MB/s bandwidth from ISSDC, so a 2 GB dataset takes ~1 hour).\n",
    "\n",
    "After downloading the Level2 data (Level1: unprocessed, Level2: processed), we will have a zipped directory, `LEVL2AS1UVT20170927G07_003T02_9000001564.zip`. Once extracted, a directory named `20170927_G07_003T02_9000001564_level2` can be found. This directory has the following structure.\n",
    "```\n",
    "20170927_G07_003T02_9000001564_level2/\n",
    "└── uvit\n",
    "    ├── DISCLAIMER.txt\n",
    "    ├── LEVL1AS1UVT20170927G07_003T02_9000001564_10811_V2.2_dqr.xml\n",
    "    ├── RAS_NUV\n",
    "    │   ├── pipeline\n",
    "    │   ├── uvt_01\n",
    "    │   ├── uvt_02\n",
    "    │   ├── uvt_03\n",
    "    │   ├── uvt_04\n",
    "    │   ├── uvt_05\n",
    "    │   ├── uvt_06\n",
    "    │   └── uvt_ci\n",
    "    ├── RAS_VIS\n",
    "    │   ├── pipeline\n",
    "    │   ├── uvt_01\n",
    "    │   ├── uvt_02\n",
    "    │   ├── uvt_03\n",
    "    │   ├── uvt_04\n",
    "    │   ├── uvt_05\n",
    "    │   ├── uvt_06\n",
    "    │   └── uvt_ci\n",
    "    └── README.txt\n",
    "\n",
    "```\n",
    "\n",
    "![](images/Level2_directories-fast.gif)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16d91bc4-3c7f-4c35-bb31-feb715779a07",
   "metadata": {},
   "source": [
    "Please read the `README.txt` for details on Level2 directory structure and the files it contains. \n",
    "\n",
    "* `RAS_VIS` directory contains images corrected for satellite drift using the VIS (visible) channel. \n",
    "* `RAS_NUV` directory contains images corrected for satellite drift using the NUV (near-ultraviolet) channel. \n",
    "\n",
    "For most cases, the data from `RAS_VIS` would be suitable. Check the statistics inside `DISCLAIMER.txt` to decide what to use. \n",
    " \n",
    "For this case, data statistics inside the `DISCLAIMER.txt` is as follows. Note that the values are in seconds.  \n",
    "\n",
    "```\n",
    "        |\tVIS \t|\tNUV \t|\tFUV\n",
    "- - —  - - - —  - -  —  - - —  - - - —  - -  — \n",
    "ASV     |\t8300\t|\t8300\t|\t8300\n",
    "- - —  - - - —  - -  —  - - —  - - - —  - -  — \n",
    "L1      |\t8770\t|\t8268\t|\t8247\n",
    "- - —  - - - —  - -  —  - - —  - - - —  - -  — \n",
    "L2(NUV) |\t\t    |\t3180\t|\t3183\n",
    "- - —  - - - —  - -  —  - - —  - - - —  - -  — \n",
    "L2(VIS) |\t        |\t8267\t|\t8247\n",
    "```\n",
    "\n",
    "* ASV: This row lists the total duration of scheduled observations in each of the channels of UVIT. See [Astrosat Schedule Viewer (ASV)](https://webapps.issdc.gov.in/MCAP/) website.\n",
    "* L1: Lists the total unprocessed data found inside Level1 dataset.\n",
    "* L2 (NUV): Lists the total processed data found inside RAS_NUV orbits/episodes.\n",
    "* L2 (VIS): Lists the total processed data found inside RAS_VIS orbits/episodes.\n",
    "\n",
    "The GIF below shows how to access the AstroSat Schedule for `G07_003T02_9000001564` from the [ASV](https://webapps.issdc.gov.in/MCAP/) website.\n",
    "\n",
    "![](images/asv-fast.gif)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5051ce28-314c-4289-8cbc-92352b2bbcbb",
   "metadata": {},
   "source": [
    "\n",
    "We can see that the `RAS_VIS` directory contains more data than `RAS_NUV` from statistics provided in `DISCLAIMER.txt`. \n",
    "\n",
    "Our directory of interest, `RAS_VIS`, has the following contents.\n",
    "\n",
    "```\n",
    "RAS_VIS/\n",
    "├── pipeline\n",
    "│   └── LEVL2AS1UVT20170927G07_003_L2_DM_params.txt\n",
    "├── uvt_01\n",
    "│   ├── F_01\n",
    "│   ├── N_01\n",
    "│   └── V_01\n",
    "├── uvt_02\n",
    "│   ├── F_02\n",
    "│   ├── N_02\n",
    "│   └── V_02\n",
    "├── uvt_03\n",
    "│   ├── F_03\n",
    "│   ├── N_03\n",
    "│   └── V_03\n",
    "├── uvt_04\n",
    "│   ├── F_04\n",
    "│   ├── N_04\n",
    "│   └── V_04\n",
    "├── uvt_05\n",
    "│   ├── F_05\n",
    "│   ├── N_05\n",
    "│   └── V_05\n",
    "├── uvt_06\n",
    "│   ├── F_06\n",
    "│   ├── N_06\n",
    "│   └── V_06\n",
    "└── uvt_ci\n",
    "    ├── F_ci\n",
    "    └── N_ci\n",
    "\n",
    "```\n",
    "\n",
    "You will notice that there are directories named in a pattern, `uvt_01`, `uvt_02`, . . . , `uvt_06`. They correspond to different orbits/episodes of the UVIT observation. \n",
    "\n",
    "For example, `RAS_VIS/uvt_01` has the following contents.\n",
    "\n",
    "```\n",
    "uvt_01\n",
    "├── F_01\n",
    "│   ├── AS1G07_003T02_9000001564uvtFIIPC00F1A_l2err.fits\n",
    "│   ├── AS1G07_003T02_9000001564uvtFIIPC00F1A_l2exp.fits\n",
    "│   ├── AS1G07_003T02_9000001564uvtFIIPC00F1A_l2img.fits\n",
    "│   ├── AS1G07_003T02_9000001564uvtFIIPC00F1I_l2img.fits\n",
    "│   └── AS1G07_003T02_9000001564uvtFIIPC00F1_l2ce.fits\n",
    "├── N_01\n",
    "│   ├── AS1G07_003T02_9000001564uvtNIIPC00F1A_l2err.fits\n",
    "│   ├── AS1G07_003T02_9000001564uvtNIIPC00F1A_l2exp.fits\n",
    "│   ├── AS1G07_003T02_9000001564uvtNIIPC00F1A_l2img.fits\n",
    "│   ├── AS1G07_003T02_9000001564uvtNIIPC00F1I_l2img.fits\n",
    "│   └── AS1G07_003T02_9000001564uvtNIIPC00F1_l2ce.fits\n",
    "└── V_01\n",
    "    └── AS1G07_003T02_9000001564uvtVIIIM00F2_l2dr.fits\n",
    "```\n",
    "\n",
    "Inside the directory `uvt_01`, data are organized in separate folders, each corresponding to overlapping time ranges in UV and VIS channels, as available in Level1 dataset (`F_01`: FUV; `N_01`: NUV; `V_01`: VIS).\n",
    "\n",
    "The suffixes of the FITS files have the following meaning.\n",
    "\n",
    "`...A_l2img.fits`: Image file in astronomical coordinates.\n",
    "\n",
    "`...I_l2img.fits`: Image file in instrument coordinates.\n",
    "\n",
    "`...A_l2exp.fits`: Exposure map for `A_l2img.fits`.\n",
    "\n",
    "`...A_l2err.fits`: Error map for `A_l2img.fits`.\n",
    "\n",
    "`...l2ce.fits` : Corrected events list.\n",
    "\n",
    "`...l2dr.fits` : the Relative Aspects Series (RAS) file.\n",
    "\n",
    "This structure of subdirectories shall repeat for all sets - uvt_01, uvt_02, uvt_03, etc.\n",
    "\n",
    "The following GIF shows how to view an NUV image from one of the orbits using DS9.  \n",
    "\n",
    "![](images/orbit_data-fast.gif)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea53923d-b513-4060-834d-87d37baa609e",
   "metadata": {},
   "source": [
    "The directory `uvt_ci` contains the combined images. For example, `RAS_VIS/uvt_ci` has the following contents.\n",
    "\n",
    "```\n",
    "uvt_ci\n",
    "├── F_ci\n",
    "│   ├── AS1G07_003T02_9000001564uvtFIIPC00F1A_l2erb.fits\n",
    "│   ├── AS1G07_003T02_9000001564uvtFIIPC00F1A_l2exb.fits\n",
    "│   ├── AS1G07_003T02_9000001564uvtFIIPC00F1A_l2ima.fits\n",
    "│   └── AS1G07_003T02_9000001564uvtFIIPC00F1A_l2imb.fits\n",
    "└── N_ci\n",
    "    ├── AS1G07_003T02_9000001564uvtNIIPC00F1A_l2erb.fits\n",
    "    ├── AS1G07_003T02_9000001564uvtNIIPC00F1A_l2exb.fits\n",
    "    ├── AS1G07_003T02_9000001564uvtNIIPC00F1A_l2ima.fits\n",
    "    └── AS1G07_003T02_9000001564uvtNIIPC00F1A_l2imb.fits\n",
    "```\n",
    "\n",
    "The suffixes of the FITS files have the following meaning.\n",
    "\n",
    "\n",
    "`...A_l2imb.fits`: Image file in astronomical coordinates (WCS generated by catalogue matching).\n",
    "\n",
    "`...A_l2ima.fits`: Image file in astronomical coordinates (WCS generated from spacecraft aspect).\n",
    "\n",
    "`...A_l2exb.fits`: Exposure map for `A_l2imb.fits`.\n",
    "\n",
    "`...A_l2erb.fits`: Error map for `A_l2imb.fits`.\n",
    "\n",
    "\n",
    "The following GIF shows how to view the combined NUV image using DS9.  \n",
    "\n",
    "![](images/combined-fast.gif)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}