docs/source/notebooks/tutorials/products.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introducing XGA Products"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this tutorial I will briefly touch on XGA's product classes, which are a key part of the internal makeup of the module. They act as an interface between XGA's functions and the data in different types of X-ray data (images, exposure maps, spectra etc.), and contain many useful and convenient methods for analysis and the extraction of information.\n",
    "\n",
    "I will not be demonstrating specific abilities of different product types in this tutorial, there are too many to go into here, but I will give an overview of the purpose and important abilities of all the product classes built into XGA."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from astropy.units import Quantity\n",
    "\n",
    "from xga.sources import GalaxyCluster"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What are products?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Most XGA products are what we use to wrap various types of data produced by XMM's SAS routines, as well as providing various product specific functionality. Generally, the user is unlikely to ever define a product instance themselves, internal methods in a source object and the SAS wrapper functions will define, check, and store the products. Those products that are generated by XGA's SAS interface have an added ability to parse the stderr output of the SAS routines used to generate them, then flag any recognised errors (by comparing to an archive of known SAS errors).\n",
    "\n",
    "There are some XGA products that are not wrappers for SAS generated data products, but are purely for storing and providing access to XGA generated data. These are not based on the same superclass as the other products, but are stored in the same way.\n",
    "\n",
    "Products instantiated by XGA are immediately 'associated' with a source object, though beyond storing the name of the source in the product's internal structure, they have no knowledge or awareness of the XGA source and its properties. This was by design, to make sure that any functionality built into a product would work regardless of the type of XGA source, and even if it was defined independently of any source at all."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What types of product are there?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* **BaseProduct** - The superclass for many of the standard XGA product classes, there isn't really any reason for a BaseProduct to be declared.\n",
    "\n",
    "    * **EventList** - A very simple product class which differs only slightly from the BaseProduct class, its only used to store path and header information for the XMM event lists that the configuration file points XGA at.\n",
    "\n",
    "    * **Image** - An extremely useful product with many extra features, it is used to wrap fits images. The data and header information are read into memory (when required), and can be accessed with properties and attributes of an Image instance. A view method (to look at the image and overlay extra information), and a coordinate transformation method are examples of the built-in functionality.\n",
    "\n",
    "        * **ExpMap** - A subclass of the XGA Image class, this is a very simple extension to Image that adds a method to easily retrieve an exposure time at a given angular or pixel coordinate.\n",
    "\n",
    "        * **RateMap** - The class that most photometric analyses are based around, also a subclass of Image. Instantiating this class requires the user pass matching Image and ExpMap instances (same ObsID, instrument etc.) A count rate map is then calculated from this information. This has several added methods, including the ability to retrieve a count rate at a given coordinate, and different peak finding methods.\n",
    "\n",
    "        * **PSF** - This wraps two-dimensional PSF images generated by a routine such as [psfgen](https://xmm-tools.cosmos.esa.int/external/sas/current/doc/psfgen/index.html), and can be used to PSF correct images and ratemaps. Few methods have been added, and it is unlikely a user will ever need to interact directly with this. The added functionality here is the ability to resample the PSF at a scale provided by a passed in Image object.\n",
    "    \n",
    "    * **Spectrum** - A complicated sub-class of BaseProduct, it wraps and stores a spectrum generated by a SAS routine, including storing paths to all of the other files necessary to analyse it (e.g. RMF, ARF, background spectrum). When a source that has had spectra associated with it is passed to an XGA XSPEC function, the fit results are added to the spectrum objects. This has methods to retrieve fit results, as well as view the fitted Spectrum.\n",
    "    \n",
    "    \n",
    "    \n",
    "* **BaseAggregateProduct** - Another base product class, this one is designed to store a group of related XGA products.\n",
    "    * **PSFGrid** - An XGA object used during the PSF correction of images and ratemaps, it stores a grid of PSF objects generated at different spatial positions on the XMM detectors. It provides access to those PSFs on request from the PSF correction function, and context as to which PSF was generated at which position.\n",
    "    \n",
    "    * **AnnularSpectra** - This holds sets of XGA Spectrum products generated in concentric annuli, and includes various methods for accessing the individual spectra, retrieving fit information (if a fit has been run), and viewing the spectra (both for individual annuli, and the whole set). This class is crucial for the measurement of galaxy cluster temperature profiles, and by extension the measurement of hydrostatic mass profiles.\n",
    "\n",
    "\n",
    "\n",
    "* **BaseProfile1D** - Here we move to the special products that don't wrap existing X-ray data products, these are entirely generated by XGA. This class is designed to store, fit, and generate plots of different 1D profiles. The user should never declare an instance of this class, only the specific subclass that they need for their analysis.\n",
    "    * **SurfaceBrightness1D** - Mostly meant for Galaxy Clusters, this class will store a 1D surface brightness profile, and enable the fitting of valid models such as a beta profile, or double beta profile.\n",
    "    \n",
    "    * **GasDensity1D** - This is meant to store a gas density profile as calculated by XGA, and includes methods to calculate a total gas mass within a given radius, as well as to generate a gas mass profile.\n",
    "    \n",
    "    * **GasMass1D** - A class for gas mass 1D profiles, which currently has no extra functionality over BaseProfile1D.\n",
    "    \n",
    "    * **ProjectedGasTemperature1D** - A class for the projected temperature profiles which are measured by fitting plasma emission models to annular spectra. They are 'projected' because they are a combination of temperatures of the 3D shells which are intersected along the line of sight by the annulus.\n",
    "    \n",
    "    * **APECNormalisation1D** - A class for storing profiles of the normalisation of the APEC plasma emission model, which is extracted from the same fitting process (run on an AnnularSpectra) that produces the projected temperature profiles. This profile can be used to measure the 3D density profile, and (when converted to an emission measure profile and combined with knowledge of the projected temperature profile) allows us to infer the 3D temperature profile.\n",
    "    \n",
    "    * **EmissionMeasure1D** - Calculated from an APECNormalisation1D, and knowledge of the cosmology and the redshift of the source. The emission measure profile can be used to help infer the 3D temperature profile of a cluster, when combined with the projected temperature profile and assumptions about the source geometry.\n",
    "    \n",
    "    * **ProjectedGasMetallicity1D** - Another profile that *can* be measured from the fitting of AnnularSpectra, though only if metallicity is allowed to vary as a free parameter. Again it is 'projected' because the metallicities are a combination of the metallicities of the 3D shells intersected along the line of sight by the annuli.\n",
    "    \n",
    "    * **GasTemperature3D** - A three-dimensional radial map of the plasma temperature of the intra-cluster medium of a galaxy cluster. This can be used, in combination with knowledge of the 3D gas density, to measure a mass profile for a galaxy cluster.\n",
    "    \n",
    "    * **HydrostaticMass** - Defined with a gas density profile and a 3D temperature profile, this type of profile describes the change of the total mass contained within a radius, and has methods to measure a mass at whatever radius the user wants to.\n",
    "\n",
    "    * **BaryonFraction** - Can be generated by a HydrostaticMass profile, this shows the change in total baryon fraction within a radius, with radius. Again the value at a specific point can be calculated using a method implemented in this class.\n",
    "        \n",
    "\n",
    "* **BaseAggregateProfile1D** - This is unlikely ever to actually have any specific subclasses, as awareness of the type of profile is not really necessary for its only job, which is to display multiple profiles on one axis. These can be generated just by adding multiple profile products together with the standard Python + operator\n",
    "\n",
    "\n",
    "* **ScalingRelation** - The scaling relation products are unique in XGA, in that they are the only products that cannot be stored within a source, as they concern multiple sources (or no sources at all if declared from the literature), it would not make sense. These are produced by the scaling relation generation functions built into XGA, and in that case would contain both data and a fitted model. There are also several scaling relations from literature defined in XGA, these only contain the fitted model. A view method capable of producing publication quality plots is provided, along with many other convenient methods.\n",
    "\n",
    "\n",
    "* **AggregateScalingRelation** - Just as the BaseAggregateProfile1D class was created purely to view multiple profiles on the same axes, this class is designed to enable scaling relations to be easily combined and viewed together. This class is instantiated by adding multiple ScalingRelation objects together.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generating Products"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This tutorial will not touch on how to generate the product objects, this will be left to more subject specific tutorials such as \"Photometry with XGA\" and \"Spectroscopy with XGA\".\n",
    "\n",
    "However, I do wish to say that any SAS data products generated by XGA are stored in the directory pointed to by the **xga_save_path** entry in the configuration file (which can be either an absolute or relative path). By default a source will load in any product previously generated for it, to save the computational expense of re-generating products that already exist, this behaviour can be turned off if the user sets the **load_products** keyword argument to False when defining a source object."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How products are stored in source objects"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here I will demonstrate how XGA stores product objects within a source, and how you can retrieve them if you wish to interact with the products directly. I will be demonstrate how to associate a product with a source object, as this is an internal mechanism with various checks and validation steps that the user should never have to interact with.\n",
    "\n",
    "I'll define a GalaxyCluster source object corresponding to my favourite cluster, Abell 907 (**please note that the overdensity radii and the redshift that I've used here are approximate and should not be used for a scientific analysis**). We can see that this cluster is quite well observed, with three ObsIDs associated, with all instruments valid for each observation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "-----------------------------------------------------\n",
      "Source Name - A907\n",
      "User Coordinates - (149.59209, -11.05972) degrees\n",
      "X-ray Peak - (149.59251340970866, -11.063958320861634) degrees\n",
      "nH - 0.0534 1e+22 / cm2\n",
      "Redshift - 0.16\n",
      "XMM ObsIDs - 3\n",
      "PN Observations - 3\n",
      "MOS1 Observations - 3\n",
      "MOS2 Observations - 3\n",
      "On-Axis - 3\n",
      "With regions - 3\n",
      "Total regions - 69\n",
      "Obs with one match - 3\n",
      "Obs with >1 matches - 0\n",
      "Images associated - 18\n",
      "Exposure maps associated - 18\n",
      "Combined Ratemaps associated - 1\n",
      "Spectra associated - 0\n",
      "R200 - 1700.0 kpc\n",
      "R200 SNR - 230.23\n",
      "R500 - 1200.0 kpc\n",
      "R500 SNR - 251.61\n",
      "-----------------------------------------------------\n",
      "\n"
     ]
    }
   ],
   "source": [
    "src = GalaxyCluster(149.59209, -11.05972, 0.16, r500=Quantity(1200, 'kpc'), r200=Quantity(1700, 'kpc'), \n",
    "                    name=\"A907\")\n",
    "src.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Products associatied with a source are stored in a protected attribute of the source, src.\\_products. This essentially consists of a series of nested dictionaries, with the top level keys being the associated ObsIDs (any products that are a combination of multiple ObsIDs are stored under the 'combined' key). The user should **never** interact directly with this protected attribute, but I display below the current contents of this source's storage structure - this will expand as further analyses are performed on the source, with things like spectra and PSF corrected images also being stored here.\n",
    "\n",
    "You will notice that ratemap objects are already present, that is because XGA checks for matching image-expmap pairs every time an image or expmap is assigned to it; if it finds a matching pair with no corresponding ratemap, it generates one automatically."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'0404910601': {'pn': {'events': <xga.products.misc.EventList at 0x7f309d948220>,\n",
       "   'bound_0.5-2.0': {'image': <xga.products.phot.Image at 0x7f309d948ca0>,\n",
       "    'expmap': <xga.products.phot.ExpMap at 0x7f309d948160>,\n",
       "    'ratemap': <xga.products.phot.RateMap at 0x7f309d948cd0>},\n",
       "   'bound_2.0-10.0': {'image': <xga.products.phot.Image at 0x7f309d948d60>,\n",
       "    'expmap': <xga.products.phot.ExpMap at 0x7f309d948d00>,\n",
       "    'ratemap': <xga.products.phot.RateMap at 0x7f309d948820>}},\n",
       "  'mos1': {'events': <xga.products.misc.EventList at 0x7f309d948bb0>,\n",
       "   'bound_0.5-2.0': {'image': <xga.products.phot.Image at 0x7f309d948e80>,\n",
       "    'expmap': <xga.products.phot.ExpMap at 0x7f309d948e50>,\n",
       "    'ratemap': <xga.products.phot.RateMap at 0x7f309d9487c0>},\n",
       "   'bound_2.0-10.0': {'image': <xga.products.phot.Image at 0x7f309d948f70>,\n",
       "    'expmap': <xga.products.phot.ExpMap at 0x7f309d948f40>,\n",
       "    'ratemap': <xga.products.phot.RateMap at 0x7f309d948e20>}},\n",
       "  'mos2': {'events': <xga.products.misc.EventList at 0x7f309d948af0>,\n",
       "   'bound_0.5-2.0': {'image': <xga.products.phot.Image at 0x7f309d948c10>,\n",
       "    'expmap': <xga.products.phot.ExpMap at 0x7f309d8e10d0>,\n",
       "    'ratemap': <xga.products.phot.RateMap at 0x7f309d8e1070>},\n",
       "   'bound_2.0-10.0': {'image': <xga.products.phot.Image at 0x7f309d8e11c0>,\n",
       "    'expmap': <xga.products.phot.ExpMap at 0x7f309d8e1190>,\n",
       "    'ratemap': <xga.products.phot.RateMap at 0x7f309d8e10a0>}}},\n",
       " '0201901401': {'pn': {'events': <xga.products.misc.EventList at 0x7f309d948be0>,\n",
       "   'bound_0.5-2.0': {'image': <xga.products.phot.Image at 0x7f309d8e12e0>,\n",
       "    'expmap': <xga.products.phot.ExpMap at 0x7f309d8e12b0>,\n",
       "    'ratemap': <xga.products.phot.RateMap at 0x7f309d8e1130>},\n",
       "   'bound_2.0-10.0': {'image': <xga.products.phot.Image at 0x7f309d8e13d0>,\n",
       "    'expmap': <xga.products.phot.ExpMap at 0x7f309d8e13a0>,\n",
       "    'ratemap': <xga.products.phot.RateMap at 0x7f309d8e1280>}},\n",
       "  'mos1': {'events': <xga.products.misc.EventList at 0x7f309d948fa0>,\n",
       "   'bound_0.5-2.0': {'image': <xga.products.phot.Image at 0x7f309d8e14f0>,\n",
       "    'expmap': <xga.products.phot.ExpMap at 0x7f309d8e14c0>,\n",
       "    'ratemap': <xga.products.phot.RateMap at 0x7f309d9481c0>},\n",
       "   'bound_2.0-10.0': {'image': <xga.products.phot.Image at 0x7f309d8e1580>,\n",
       "    'expmap': <xga.products.phot.ExpMap at 0x7f309d8e1490>,\n",
       "    'ratemap': <xga.products.phot.RateMap at 0x7f309d8e1340>}},\n",
       "  'mos2': {'events': <xga.products.misc.EventList at 0x7f309d948a60>,\n",
       "   'bound_0.5-2.0': {'image': <xga.products.phot.Image at 0x7f309d8e16a0>,\n",
       "    'expmap': <xga.products.phot.ExpMap at 0x7f309d8e1670>,\n",
       "    'ratemap': <xga.products.phot.RateMap at 0x7f309d8e1460>},\n",
       "   'bound_2.0-10.0': {'image': <xga.products.phot.Image at 0x7f309d8e1790>,\n",
       "    'expmap': <xga.products.phot.ExpMap at 0x7f309d8e1760>,\n",
       "    'ratemap': <xga.products.phot.RateMap at 0x7f309d8e1640>}}},\n",
       " '0201903501': {'pn': {'events': <xga.products.misc.EventList at 0x7f309de2a0a0>,\n",
       "   'bound_0.5-2.0': {'image': <xga.products.phot.Image at 0x7f309d8e18b0>,\n",
       "    'expmap': <xga.products.phot.ExpMap at 0x7f309d8e1880>,\n",
       "    'ratemap': <xga.products.phot.RateMap at 0x7f309d8e1610>},\n",
       "   'bound_2.0-10.0': {'image': <xga.products.phot.Image at 0x7f309d8e19a0>,\n",
       "    'expmap': <xga.products.phot.ExpMap at 0x7f309d8e1970>,\n",
       "    'ratemap': <xga.products.phot.RateMap at 0x7f309d8e1850>}},\n",
       "  'mos1': {'events': <xga.products.misc.EventList at 0x7f309d936fa0>,\n",
       "   'bound_0.5-2.0': {'image': <xga.products.phot.Image at 0x7f309d8e1ac0>,\n",
       "    'expmap': <xga.products.phot.ExpMap at 0x7f309d8e1a90>,\n",
       "    'ratemap': <xga.products.phot.RateMap at 0x7f309d8e1820>},\n",
       "   'bound_2.0-10.0': {'image': <xga.products.phot.Image at 0x7f309d8e1bb0>,\n",
       "    'expmap': <xga.products.phot.ExpMap at 0x7f309d8e1b80>,\n",
       "    'ratemap': <xga.products.phot.RateMap at 0x7f309d8e1a60>}},\n",
       "  'mos2': {'events': <xga.products.misc.EventList at 0x7f309d8e1700>,\n",
       "   'bound_0.5-2.0': {'image': <xga.products.phot.Image at 0x7f309d8e1cd0>,\n",
       "    'expmap': <xga.products.phot.ExpMap at 0x7f309d8e1ca0>,\n",
       "    'ratemap': <xga.products.phot.RateMap at 0x7f309d8e1250>},\n",
       "   'bound_2.0-10.0': {'image': <xga.products.phot.Image at 0x7f309d8e1dc0>,\n",
       "    'expmap': <xga.products.phot.ExpMap at 0x7f309d8e1d90>,\n",
       "    'ratemap': <xga.products.phot.RateMap at 0x7f309d8e1c70>}}},\n",
       " 'combined': {'bound_0.5-2.0': {'combined_image': <xga.products.phot.Image at 0x7f309d7bbd90>,\n",
       "   'combined_expmap': <xga.products.phot.ExpMap at 0x7f309d7c6a00>,\n",
       "   'combined_ratemap': <xga.products.phot.RateMap at 0x7f309d7b6670>}}}"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "src._products"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once we get below the top level of the storage structure, things are slightly different - any entry stored under an ObsID will be another dictionary, with keys corresponding to the available instruments. As you might expect I store data products from the different XMM instruments under these different keys.\n",
    "\n",
    "Below that level (in a specific ObsID-Instrument dictionary) we start to find keys for specific types of product. Products that are generated within a specific energy range are stored under 'bound_lower-upper' keys, where lower is the lower energy limit in keV and upper is the upper energy limit in keV. These products will include images, expmaps, and ratemaps, which are all stored under keys that match the name of their product classes.\n",
    "\n",
    "The top level combined key is structured nearly identically, but skips out the dictionary layer that uses instruments as key names. Once you get into an energy bound sub-dictionary however, you will notice that combined products have a prefix on their class keys, 'combined_' is added to the type of the object, e.g. 'combined_image'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dict_keys(['pn', 'mos1', 'mos2']) \n",
      "\n",
      "dict_keys(['events', 'bound_0.5-2.0', 'bound_2.0-10.0']) \n",
      "\n",
      "dict_keys(['image', 'expmap', 'ratemap']) \n",
      "\n",
      "{'combined_image': <xga.products.phot.Image object at 0x7f309d7bbd90>, 'combined_expmap': <xga.products.phot.ExpMap object at 0x7f309d7c6a00>, 'combined_ratemap': <xga.products.phot.RateMap object at 0x7f309d7b6670>}\n"
     ]
    }
   ],
   "source": [
    "# Showing the instrument dictionary layer for observation 0404910601\n",
    "print(src._products['0404910601'].keys(), '\\n')\n",
    "\n",
    "# And the layer where energy bound and non-energy bound products are stored separately.\n",
    "print(src._products['0404910601']['pn'].keys(), '\\n')\n",
    "\n",
    "# Here we can see how images, expmaps, and ratemaps are stored in an energy bound sub-dictionary\n",
    "print(src._products['0404910601']['pn']['bound_0.5-2.0'].keys(), '\\n')\n",
    "\n",
    "print(src._products['combined']['bound_0.5-2.0'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A general method for retrieving products"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To actually retrieve the product objects I have implemented a [get_products()](../../xga.sources.rst#xga.sources.base.BaseSource.get_products) method. To use this method you need to specify the type of object to retrieve, (e.g. 'image', o), the specific ObsID and instrument (if these options are not specified then the method will retrieve any product that matches the other criteria you provide).\n",
    "\n",
    "You can also pass an 'extra key', which would be something like a specific bound energy key; this, and knowing the key that corresponds to the type of product you wish to retrieve, are the hardest parts of using this method, and the reason I will be introducing separate methods for specific product types.\n",
    "\n",
    "The final keyword argument taken by get_products is 'just_obj', which tells the method if you would just like a list of product objects, or if you'd like a list of lists which contained the key path a particular product was stored under."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you just wanted to retrieve all images then you could call the method like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<xga.products.phot.Image at 0x7f309d948ca0>,\n",
       " <xga.products.phot.Image at 0x7f309d948d60>,\n",
       " <xga.products.phot.Image at 0x7f309d948e80>,\n",
       " <xga.products.phot.Image at 0x7f309d948f70>,\n",
       " <xga.products.phot.Image at 0x7f309d948c10>,\n",
       " <xga.products.phot.Image at 0x7f309d8e11c0>,\n",
       " <xga.products.phot.Image at 0x7f309d8e12e0>,\n",
       " <xga.products.phot.Image at 0x7f309d8e13d0>,\n",
       " <xga.products.phot.Image at 0x7f309d8e14f0>,\n",
       " <xga.products.phot.Image at 0x7f309d8e1580>,\n",
       " <xga.products.phot.Image at 0x7f309d8e16a0>,\n",
       " <xga.products.phot.Image at 0x7f309d8e1790>,\n",
       " <xga.products.phot.Image at 0x7f309d8e18b0>,\n",
       " <xga.products.phot.Image at 0x7f309d8e19a0>,\n",
       " <xga.products.phot.Image at 0x7f309d8e1ac0>,\n",
       " <xga.products.phot.Image at 0x7f309d8e1bb0>,\n",
       " <xga.products.phot.Image at 0x7f309d8e1cd0>,\n",
       " <xga.products.phot.Image at 0x7f309d8e1dc0>]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "src.get_products('image')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Though by itself its not very useful, as you can't see which image is which. The image class has properties that will tell you the energy band, ObsID, and instrument ([energy_bounds](../../xga.products.rst#xga.products.base.BaseProduct.energy_bounds), [obs_id](../../xga.products.rst#xga.products.base.BaseProduct.obs_id), [instrument](../../xga.products.rst#xga.products.base.BaseProduct.instrument)), but perhaps you want that information in the returned list? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[['0404910601',\n",
       "  'pn',\n",
       "  'bound_0.5-2.0',\n",
       "  <xga.products.phot.Image at 0x7f309d948ca0>],\n",
       " ['0404910601',\n",
       "  'pn',\n",
       "  'bound_2.0-10.0',\n",
       "  <xga.products.phot.Image at 0x7f309d948d60>],\n",
       " ['0404910601',\n",
       "  'mos1',\n",
       "  'bound_0.5-2.0',\n",
       "  <xga.products.phot.Image at 0x7f309d948e80>],\n",
       " ['0404910601',\n",
       "  'mos1',\n",
       "  'bound_2.0-10.0',\n",
       "  <xga.products.phot.Image at 0x7f309d948f70>],\n",
       " ['0404910601',\n",
       "  'mos2',\n",
       "  'bound_0.5-2.0',\n",
       "  <xga.products.phot.Image at 0x7f309d948c10>],\n",
       " ['0404910601',\n",
       "  'mos2',\n",
       "  'bound_2.0-10.0',\n",
       "  <xga.products.phot.Image at 0x7f309d8e11c0>],\n",
       " ['0201901401',\n",
       "  'pn',\n",
       "  'bound_0.5-2.0',\n",
       "  <xga.products.phot.Image at 0x7f309d8e12e0>],\n",
       " ['0201901401',\n",
       "  'pn',\n",
       "  'bound_2.0-10.0',\n",
       "  <xga.products.phot.Image at 0x7f309d8e13d0>],\n",
       " ['0201901401',\n",
       "  'mos1',\n",
       "  'bound_0.5-2.0',\n",
       "  <xga.products.phot.Image at 0x7f309d8e14f0>],\n",
       " ['0201901401',\n",
       "  'mos1',\n",
       "  'bound_2.0-10.0',\n",
       "  <xga.products.phot.Image at 0x7f309d8e1580>],\n",
       " ['0201901401',\n",
       "  'mos2',\n",
       "  'bound_0.5-2.0',\n",
       "  <xga.products.phot.Image at 0x7f309d8e16a0>],\n",
       " ['0201901401',\n",
       "  'mos2',\n",
       "  'bound_2.0-10.0',\n",
       "  <xga.products.phot.Image at 0x7f309d8e1790>],\n",
       " ['0201903501',\n",
       "  'pn',\n",
       "  'bound_0.5-2.0',\n",
       "  <xga.products.phot.Image at 0x7f309d8e18b0>],\n",
       " ['0201903501',\n",
       "  'pn',\n",
       "  'bound_2.0-10.0',\n",
       "  <xga.products.phot.Image at 0x7f309d8e19a0>],\n",
       " ['0201903501',\n",
       "  'mos1',\n",
       "  'bound_0.5-2.0',\n",
       "  <xga.products.phot.Image at 0x7f309d8e1ac0>],\n",
       " ['0201903501',\n",
       "  'mos1',\n",
       "  'bound_2.0-10.0',\n",
       "  <xga.products.phot.Image at 0x7f309d8e1bb0>],\n",
       " ['0201903501',\n",
       "  'mos2',\n",
       "  'bound_0.5-2.0',\n",
       "  <xga.products.phot.Image at 0x7f309d8e1cd0>],\n",
       " ['0201903501',\n",
       "  'mos2',\n",
       "  'bound_2.0-10.0',\n",
       "  <xga.products.phot.Image at 0x7f309d8e1dc0>]]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "src.get_products('image', just_obj=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you wanted a very specific image, perhaps a 0.5-2.0keV EPIC-PN image from observation 0201903501, you could call the get_products method like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<xga.products.phot.Image at 0x7f309d8e18b0>]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "src.get_products('image', '0201903501', 'pn', extra_key='bound_0.5-2.0')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And finally, if you were actually interested in the combined image (all observations, all instruments - XGA does not generate combined data products for individual observations), then you would call get_products with a different product name:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<xga.products.phot.Image at 0x7f309d7bbd90>]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "src.get_products('combined_image', extra_key='bound_0.5-2.0')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Methods for retrieving specific types of product"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Above I have detailed the general method for retrieving any type of product, but I have also implemented methods that will return specific types on request, though they are essentially all wrappers of that original, general function. Many have been implemented, and I advise you to look at the API documentation for the specific source class you're interested in, but I shall list and demonstrate a few here.\n",
    "\n",
    "Firstly, we can retrieve single camera images and exposure maps with their own get methods, calling them like this will retrieve the individual images and exposure maps (in all energy bands), though please note that to retrieve PSF corrected images you must pass `psf_corr=True`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<xga.products.phot.Image at 0x7f309d948ca0>,\n",
       " <xga.products.phot.Image at 0x7f309d948d60>,\n",
       " <xga.products.phot.Image at 0x7f309d948e80>,\n",
       " <xga.products.phot.Image at 0x7f309d948f70>,\n",
       " <xga.products.phot.Image at 0x7f309d948c10>,\n",
       " <xga.products.phot.Image at 0x7f309d8e11c0>,\n",
       " <xga.products.phot.Image at 0x7f309d8e12e0>,\n",
       " <xga.products.phot.Image at 0x7f309d8e13d0>,\n",
       " <xga.products.phot.Image at 0x7f309d8e14f0>,\n",
       " <xga.products.phot.Image at 0x7f309d8e1580>,\n",
       " <xga.products.phot.Image at 0x7f309d8e16a0>,\n",
       " <xga.products.phot.Image at 0x7f309d8e1790>,\n",
       " <xga.products.phot.Image at 0x7f309d8e18b0>,\n",
       " <xga.products.phot.Image at 0x7f309d8e19a0>,\n",
       " <xga.products.phot.Image at 0x7f309d8e1ac0>,\n",
       " <xga.products.phot.Image at 0x7f309d8e1bb0>,\n",
       " <xga.products.phot.Image at 0x7f309d8e1cd0>,\n",
       " <xga.products.phot.Image at 0x7f309d8e1dc0>]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "specific_ims = src.get_images()\n",
    "specific_ims"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If I cycle through the list of image products and look at their energy bounds, ObsIDs, and instruments we can see that we are indeed retrieving the images for all ObsID-Instrument combinations, in all energy bands:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(<Quantity 0.5 keV>, <Quantity 2. keV>) 0404910601 pn\n",
      "(<Quantity 2. keV>, <Quantity 10. keV>) 0404910601 pn\n",
      "(<Quantity 0.5 keV>, <Quantity 2. keV>) 0404910601 mos1\n",
      "(<Quantity 2. keV>, <Quantity 10. keV>) 0404910601 mos1\n",
      "(<Quantity 0.5 keV>, <Quantity 2. keV>) 0404910601 mos2\n",
      "(<Quantity 2. keV>, <Quantity 10. keV>) 0404910601 mos2\n",
      "(<Quantity 0.5 keV>, <Quantity 2. keV>) 0201901401 pn\n",
      "(<Quantity 2. keV>, <Quantity 10. keV>) 0201901401 pn\n",
      "(<Quantity 0.5 keV>, <Quantity 2. keV>) 0201901401 mos1\n",
      "(<Quantity 2. keV>, <Quantity 10. keV>) 0201901401 mos1\n",
      "(<Quantity 0.5 keV>, <Quantity 2. keV>) 0201901401 mos2\n",
      "(<Quantity 2. keV>, <Quantity 10. keV>) 0201901401 mos2\n",
      "(<Quantity 0.5 keV>, <Quantity 2. keV>) 0201903501 pn\n",
      "(<Quantity 2. keV>, <Quantity 10. keV>) 0201903501 pn\n",
      "(<Quantity 0.5 keV>, <Quantity 2. keV>) 0201903501 mos1\n",
      "(<Quantity 2. keV>, <Quantity 10. keV>) 0201903501 mos1\n",
      "(<Quantity 0.5 keV>, <Quantity 2. keV>) 0201903501 mos2\n",
      "(<Quantity 2. keV>, <Quantity 10. keV>) 0201903501 mos2\n"
     ]
    }
   ],
   "source": [
    "for im in specific_ims:\n",
    "    print(im.energy_bounds, im.obs_id, im.instrument)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then, if we wanted to be more even more specific, we could decide that we only wanted to retrieve the 0.5-2.0keV exposure map for the PN camera of observation 0201903501. When we specify exactly what exposure map we want, the method returns a single ExpMap instance, rather than a list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<xga.products.phot.ExpMap at 0x7f309d8e1880>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "specific_exs = src.get_expmaps('0201903501', 'pn', Quantity(0.5, 'keV'), Quantity(2.0, 'keV'))\n",
    "specific_exs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then I might perhaps want to retrieve the 0.5-2.0keV **merged** image and ratemap generated for this source, I would use different methods again, specifically for combined images and combined ratemaps (again the `psf_corr=True` argument must be passed to either of these if you want to retrieve a PSF corrected image/ratemap):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "comb_rt = src.get_combined_ratemaps(Quantity(0.5, 'keV'), Quantity(2.0, 'keV'))\n",
    "comb_im = src.get_combined_images(Quantity(0.5, 'keV'), Quantity(2.0, 'keV'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are many other specific get methods for products, including for individual and annular spectra, combined_exposure maps, and many different types of profiles (dependant on the particular source class you're using)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## NoProductAvailableError"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This exception is triggered if you try and use one of the specific get methods to retrieve a product that does not exist. Here, for instance, I am trying to retrieve a merged RateMap in the 0.5-4.2keV energy range, an energy range for which I have not even generated individual Images/ExpMaps, let alone a merged RateMap:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "ename": "NoProductAvailableError",
     "evalue": "Cannot find any combined ratemaps matching your input.",
     "output_type": "error",
     "traceback": [
      "\u001B[0;31m---------------------------------------------------------------------------\u001B[0m",
      "\u001B[0;31mNoProductAvailableError\u001B[0m                   Traceback (most recent call last)",
      "\u001B[0;32m<ipython-input-13-b512f325b934>\u001B[0m in \u001B[0;36m<module>\u001B[0;34m\u001B[0m\n\u001B[0;32m----> 1\u001B[0;31m \u001B[0msrc\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mget_combined_ratemaps\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0mQuantity\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0;36m0.5\u001B[0m\u001B[0;34m,\u001B[0m \u001B[0;34m'keV'\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m,\u001B[0m \u001B[0mQuantity\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0;36m4.2\u001B[0m\u001B[0;34m,\u001B[0m \u001B[0;34m'keV'\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[0m",
      "\u001B[0;32m~/code/PycharmProjects/XGA/xga/sources/base.py\u001B[0m in \u001B[0;36mget_combined_ratemaps\u001B[0;34m(self, lo_en, hi_en, psf_corr, psf_model, psf_bins, psf_algo, psf_iter)\u001B[0m\n\u001B[1;32m   2902\u001B[0m             \u001B[0mmatched_prods\u001B[0m \u001B[0;34m=\u001B[0m \u001B[0mmatched_prods\u001B[0m\u001B[0;34m[\u001B[0m\u001B[0;36m0\u001B[0m\u001B[0;34m]\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[1;32m   2903\u001B[0m         \u001B[0;32melif\u001B[0m \u001B[0mlen\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0mmatched_prods\u001B[0m\u001B[0;34m)\u001B[0m \u001B[0;34m==\u001B[0m \u001B[0;36m0\u001B[0m\u001B[0;34m:\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[0;32m-> 2904\u001B[0;31m             \u001B[0;32mraise\u001B[0m \u001B[0mNoProductAvailableError\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0;34m\"Cannot find any combined ratemaps matching your input.\"\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[0m\u001B[1;32m   2905\u001B[0m \u001B[0;34m\u001B[0m\u001B[0m\n\u001B[1;32m   2906\u001B[0m         \u001B[0;32mreturn\u001B[0m \u001B[0mmatched_prods\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n",
      "\u001B[0;31mNoProductAvailableError\u001B[0m: Cannot find any combined ratemaps matching your input."
     ]
    }
   ],
   "source": [
    "src.get_combined_ratemaps(Quantity(0.5, 'keV'), Quantity(4.2, 'keV'))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}