ACTNotebook_Section_1_Introduction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "6DA_syE9TrBh"
   },
   "source": [
    "# Overview and Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "BiWWr-eSpHLu"
   },
   "source": [
    "This notebook provides an overview of the ACT data releases 4 and 5 (DR4 and DR5) data products and an introduction to accessing and manipulating these products.  The data products (summarized in Table 1)  include temperature and polarization maps, a lensing map, and component seperated maps. The full DR4 and DR5 data releases on LAMBDA also include derived products such as angular power spectra, point source catalogs, masks and beams, and likelihoods.\n",
    "\n",
    "DR4 and DR5 contain raw maps at 98 and 150 GHz covering nearly half the sky as well as co-adds of the 98, 150 and 220 GHz channels.  Previous analyses used only ~650 sq deg of sky; this data release provides maps that cover ~18,000 sq degrees.  We provide these maps both independently and combined with data from the Planck satellite.  For more details on the raw and coadded data products and to understand the methods used to produce the maps please reference [Naess et al (2020)](https://phy-act1.princeton.edu/public/saiola/act_dr4_N20.pdf) for the coadded maps and [Aiola et al (2020)](https://phy-act1.princeton.edu/public/saiola/act_dr4_A20.pdf) for the raw maps. \n",
    "\n",
    "---\n",
    "Table 1: Overview of Map Products\n",
    "---\n",
    "\n",
    "Product type | Maps | Area Covered| Other products\n",
    "---|---|---|---\n",
    "Season 08 - 18 Coadded Maps| T, Q, U @ 98,150 and 220 GHz  | ~18,000 sq degrees | beams, inverse variance (hit count maps), ACT only as well as ACT+Planck maps\n",
    "Season 13 - 16 raw Maps| T, Q, U @ 98 and 150 GHz | ~ 17,000 sq degrees | Individual splits, beams, inverse variance, source free maps\n",
    "Lensing Maps| | ~2100 sq degrees |  Masks, simulations\n",
    "Component Separated Maps| Compton-y maps and CMB + kSZ | ~2100 sq degrees | Masks, Beams and Noise, ACT only as well as ACT+Planck maps\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "---\n",
    "\n",
    "We include raw maps that include night time data from Season 13 through 16 at 98 and 150 GHz.  The maps cover more than 17,000 sq degrees, with the deepest 600 sq degrees having noise levels below 10µK–arcmin.  These maps are discussed in more depth in [Aiola et al (2020)](https://phy-act1.princeton.edu/public/saiola/act_dr4_A20.pdf). As mentioned, we also include coadded maps, combining data from Season 08 through 18 across patches, and at higher frequencies (including 220 GHz) than the raw maps. In particular, the coadd maps include data from the MBAC instrument, ACTPol receiver, and AdvancedACT receiver, along with Planck data [Naess et al (2020)](https://phy-act1.princeton.edu/public/saiola/act_dr4_N20.pdf). Together, these data comprise the primary DR4 and DR5 map products, from which the following products are derived.\n",
    "\n",
    "DR4 includes the lensing maps discussed in [Darwish et al (2020)](https://arxiv.org/pdf/2004.01139.pdf).  The maps are constructed using two frequency bands, 98 GHz and 150 GHz, and data from seasons 2014 - 2015 in two regions of the sky, BOSS-North (\\~1633 sq. degrees of sky) and D56 (\\~456 sq. degrees). \n",
    "\n",
    "We also include component separated maps in the D56 and BOSS-North regions.  These maps include CMB+kSZ maps and Compton-$y$ maps along with versions of these maps that specifically deproject individual components.  These maps can be used to study the pressure profiles of galaxies, groups, and clusters, as well as for lensing reconstruction and kinematic SZ cross-correlations [Madhavacheril et al (2019)](https://arxiv.org/pdf/1911.05717.pdf). \n",
    "\n",
    "These data are stored in the CAR (or Plate Carée) format as ndmap objects, which are based off of the numpy array framework.   This is distinct from the [HEALPix](https://healpix.sourceforge.io/) format commonly used in cosmology.  Therefore we provide an introduction to the tools needed for manipulating these maps and for converting between HEALPix and CAR format.   The library that accomplishes this is called [Pixell](https://github.com/simonsobs/pixell).  \n",
    "\n",
    "Previously ACT maps were made with the cylindrical equal-area (CEA) projection.  This worked well for smaller areas but due to the new larger area of the ACT survey we have adopted the Plate Carée pixelization.  The CAR projection, as described above, is a rectangular projection, however, it is not a flat sky projection.  Instead, in each latitude ring, pixels are spaced such that there are equal number of pixels in each latitude ring.  This means that the physical space between pixels depends on the ring and pixels near the equator have a larger physical space difference between them than pixels near the poles.  [Pixell's](https://github.com/simonsobs/pixell) functionality handles this complexity, allowing us to run spherical harmonic, and thus curved-sky, analyses on the maps despite presenting them as 2D Numpy objects [Choi et al (2020)](https://phy-act1.princeton.edu/public/saiola/act_dr4_C20.pdf).\n",
    "\n",
    "The data themselves can be found on the [NASA Legacy Archive Microwave\n",
    "Background Data Analysis website](https://lambda.gsfc.nasa.gov/product/act/).  Users are welcome to use those data products and the examples shared in these notebooks in order to learn more about the new data set. In order to facilitate better understanding, we also provide auxilliary data (hosted on [here](https://phy-act1.princeton.edu/public/zatkins/)) which is used periodically in the notebooks. This data is for demonstrations purposes only.\n",
    "\n",
    "These Notebooks represent the work of our entire collaboration and include data products and processes that have been developed by numerous people within ACT.  As such we request that any work that uses these data products reference the *original work* produced by this collaboration."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "b09xD3x0d2Kl"
   },
   "source": [
    "## Data products discussed in these Notebooks\n",
    "DR4 includes a multitude of data products as discussed above.  These notebooks will focus on introducing the range of products and demonstrating how to access and examine the various maps. In particular, the first few notebooks will introduce users to the Pixell framework: demonstrating how to read in CAR maps and view objects in the maps. Instructions on how to access the data products for these notebooks can be found in more detail on our [GitHub page](https://github.com/ACTCollaboration/DR4_Notebooks)\n",
    " \n",
    "Notebook Info | Value\n",
    "---|---\n",
    "Dependencies | `numpy`, `scipy`, `healpy`, `matplotlib`, `pixell`\n",
    "Data products | None\n",
    "Memory usage | None\n",
    "Contributors | Maya Mallaby-Kay, Zach Atkins\n",
    "\n",
    "\n",
    "For the complete set of ACT notebooks, check out the ACT notebook repo https://github.com/ACTCollaboration/DR4_DR5_Notebooks\n",
    "\n",
    "\n",
    "## Table of contents\n",
    "\n",
    " --- \n",
    "> [Section 1](ACTNotebook_Section_1_Introduction.ipynb): Overview and Introduction \n",
    "\n",
    "\n",
    "> [Section 4](ACTNotebook_Section_4_visualize_objects.ipynb):  The DR5 Cluster Catalog from [Hilton et al (2020)](https://arxiv.org/abs/2009.11043) and Visualizing Objects in the Maps\n",
    "\n",
    "> [Section 5](ACTNotebook_Section_5_Lensing_maps.ipynb):  Introduction to the Lensing maps from [Darwish et al (2020)](https://arxiv.org/pdf/2004.01139.pdf)\n",
    " \n",
    "---\n",
    "Use the links in the table of contents to access the specific analysis topic you are interested in.  While many of the sections can be used independently it is useful to consult section 2 initially in order to gain an undestanding of the Pixell framework and the CAR projection."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "jvZURp-1z3LC"
   },
   "source": [
    "# Section 1: Importing packages"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "kxJg1HAylo9j"
   },
   "source": [
    "In order to work with the maps and data products discussed above, these notebooks rely on a few different publicly available packages.  If you are working within the Docker installation these will already be included in the container; if you're using these notebooks without the container then you will need to make sure you have the packages installed locally.\n",
    "\n",
    "For more information on Pixell and Healpy feel free to read the relevant documentation.  They can be found here: [Pixell](https://github.com/simonsobs/pixell) and [Healpy](https://healpy.readthedocs.io/en/latest/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "JuW6ZY2_PYjA"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import healpy as hp\n",
    "import time\n",
    "import sys\n",
    "from pixell import enmap, enplot, reproject, utils, curvedsky \n",
    "from matplotlib import cm\n",
    "from scipy.optimize import curve_fit\n",
    "from scipy.linalg import sqrtm\n",
    "\n",
    "# We also set the path for the data products you downloaded - if using this in Docker then the path\n",
    "# Should be correct as is (\"/usr/home/workspace/data/\"), if using these notebooks locally \n",
    "# you'll want to fill in the path to the directory containing the data products now \n",
    "\n",
    "path = \"/your/path/to/ACT/Data\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Continue to the next Section\n",
    "\n",
    "[Section 4](ACTNotebook_Section_4_visualize_objects.ipynb): The DR5 Cluster Catalog from [Hilton et al (2020)](https://arxiv.org/abs/2009.11043) and Visualizing Objects in the Mapsto Pixell"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [
    "7wuon2PcQDH3",
    "UqVls-28Fe6d",
    "hRBTzr5ghknb",
    "smN7aD_qfS0X",
    "Hdr6rmILNbUE",
    "lap_xzTkn5s5",
    "ydVE2Jvup42D",
    "LsvxgUOow66A",
    "sRfS65NaDKuX"
   ],
   "name": "Advanced ACT DR4 Combined data products ",
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}