docs/notebooks/single_dish_calibration.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "single_dish_calibration.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yYNnG67l0EDu"
      },
      "source": [
        "# Single Dish Calibration \n",
        "\n",
        "This section describes the motivation, background, and process for calibrating ALMA single-dish observations.\n",
        "\n",
        "Below are brief descriptions of the calibration process, followed by a description of the various observing modes, and then a step-by-step description of the calibration process in CASA.\n",
        "\n",
        "It is recommended that newcomers review the description of calibration in [Single-dish calibration: background](single_dish_calibration.ipynb#Background-SD-Calibration) and descriptions of various observing modes in [Single-dish observing modes](single_dish_calibration.ipynb#observing-modes) before examining the reduction processes decribed in [Single-dish data calibration and reduction](single_dish_calibration.ipynb#calibration-&-reduction). Future directions are broadly outlined in [Future development goals for CASA single-dish](single_dish_calibration.ipynb#future-development-goals).\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DRicQDqM0EDv"
      },
      "source": [
        "## Background SD Calibration \n",
        "\n",
        "Brief description of calibrating ALMA single-dish observations\n",
        "\n",
        "Like any single-dish telescope, ALMA\\'s single dish antennas (nominally, four 12m antennas) detect and quantify brightness temperature ($T_B$, in Kelvin). In the Rayleigh-Jeans approximation, the Planck blackbody law reduces to $T_B=\\frac{B\\lambda^2}{2k}$.  \n",
        "\n",
        "An ALMA single-dish observation includes contributions from sky targets in the beam, the telescope surface and receiver equipment, the ground (through reflections), the atmosphere and cosmic background, and any other electronics (necessarily noisy) following the receiver front end. Observations made with a single dish towards a target ($T_{ON}$) are calibrated using an additional observation towards blank sky (i.e. sky at a similar elevation, absent of any target emission at the frequencies of interest to the observer ($T_{OFF}$)).\n",
        "\n",
        "To determine the signal from the target, we can compute:\n",
        "\n",
        "$\\frac{T_{targ}}{T_{sys}}=\\frac{T_{ON}-T_{OFF}}{T_{OFF}}$.   (In CASA, this is accomplished during the \\\"sky calibration\\\" step).\n",
        "\n",
        "The position of the OFF is made as close as possible (in az/el) to the ON position.  As a practical matter, there may be some differences in the two measurements aside from the target being in the ON position. In most cases, the differences arise chiefly from the atmospheric contribution, though any target coincidentally within the beam of the OFF measurement will contaminate and affect the accuracy of the OFF measurement, and consequently, the measurement of the calibrated brightness temperature of the target.\n",
        "\n",
        "To calibrate single dish data, we require a measurement of $T_{sys}$, which is done though $T_{atm}$ (i.e. \\\"atmosphere\\\") measurements at the start of each scheduling block.  (In CASA, this is applied through the $T_{sys}$ calibration step.)  $T_{sys}$ determination includes separate observations of the sky, and two \\\"loads\\\" of different, known temperatures.\n",
        "\n",
        "Note that $T_{sys}(\\nu)$ measurements are spectral; that is, they determine $T_{sys}$ as function of frequency. Since they incorporate an observation of the sky, they may include atmospheric features such as the water absorption line in Band 5 at $\\sim$183 GHz. So the calibration of the entire band must be done in the frequency domain.\n",
        "\n",
        "It is policy that ALMA single-dish data must only be observed to supplement and be combined with interferometer observations. Therefore, the single-dish data needs to be converted from its native units of brightness temperature ($T_A^*$) to flux density units (Jy/beam) before combination with the interferometric data. The conversion from $T_A^*$ to Jy/beam is done empirically, and incorporates a factor for forward beam efficiency. The empirical conversion (Jy to K) is computed through mapping observations (done recently in time) of a standard target - either a planet or a quasar - and the scaling from $T_A^*$ to Jy/beam is then made simply and directly from the calibrator map and applied to the science target map. \n",
        "\n",
        "\n",
        "**Bibliography**\n",
        "\n",
        "1. O'Neil, 2002 in \"The NAIC/NRAO School on Single Dish Radio Astronomy\" C. Salter, et.al eds.\n",
        "2. PdBI mm astro summer school notes (Dutrey, Dutrey & Neri; Guélin) \n",
        "3. Unpublished ALMA memo: Robert Lucas, 2005\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6AyiZeWN0EDw"
      },
      "source": [
        "## Observing Modes \n",
        "\n",
        "At present, ALMA single dish observing has three modes: PS (position switched), OTF (on the fly), and OTF-raster (Rasterized on the fly). These have slightly different observation/scanning patterns, and also have differences in the cadence and position of the OFF measurements.\n",
        "\n",
        "In the case of the PS mode, the OFF position is specified by the PI before the observations.  The OFF positions have a specific and periodic cadence and should, where possible, be at approximately the same azimuth as the observations of the science target. Note that PS mode observations can include mapping observations, and this is actually the general mode for ALMA single-dish observations.\n",
        "\n",
        "For OTF-raster, the telescope scans the observing target, including some additional length on either side of the target.  The reference data are interpolated from the OFF positions at the edge of the scans. It is assumed that the bandpass profile varies slowly as a function of scan position. This mode yields a slightly better calibration than position switching, since the variation in the air mass is typically more consistent between the OFF data and the target.  At present, ALMA uses OTF-raster mode to calibrate observations of the amplitude (i.e. Jy-to-K) calibrators.\n",
        "\n",
        "For OTF mode, the observations are not rasterized.  The scanning pattern for OTF observations can take several forms.  Solar observations, for example, use a double-circle scan. In this case the entire periphery of the observed region is identified as an OFF measurement.  The OTF-raster mode, on the other hand, uses the first and last points of the raster rows as the OFF. A mean bandpass is formed from the OFF measurements, interpolated in time, and applied to any measurements not identified as OFF.  For OTF and OTF-raster, specific OFF posiotions are not explicitly nominated prior to the observations. \n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bVLx0h6E0EDw"
      },
      "source": [
        "## Calibration & Reduction \n",
        "\n",
        "**Single-dish data calibration and reduction**\n",
        "\n",
        "Generally, the calibration of ALMA single-dish data requires essentially only two steps - the application of the $T_{sys}$ calibration, and the application of the \\\"sky\\\" calibration (i.e. OFF). With just these steps, the product is $T_A^*$ in units of Kelvin. While the steps are described in detail here, an example of the full single-dish calibration process can be found in the [M100 Band 3 Single Dish CASAguide](https://casaguides.nrao.edu/index.php/M100_Band3_SingleDish_4.3).\n",
        "\n",
        "In the following description, we refer to data in the ALMA-native format of ASDM (ALMA Science Data Model) with a variable name *asdm*, while we refer to the CASA-native format of MeasurementSet with a variable name *sd_ms*. In the case of single-dish data, the MeasurementSet data are not formally *visibilities*, since they are simply auto-correlations.\n",
        "\n",
        "1.  **importasdm**: Import the ASDM\n",
        "2.  **listobs** - List observation parameters\n",
        "3.  **flagdata** and **flagcmd** - A priori flagging\n",
        "4.  **gencal**, **applycal**, and **sdcal**- Generate and apply the $T_{sys}$ and sky calibration tables\n",
        "5.  **flagdata** - Do initial flagging\n",
        "6.  **applycal** -  Calibrate the data into Kelvins\n",
        "7.  **sdbaseline** - Subtract the baseline\n",
        "\n",
        "**1. Import of the ASDM: *ImportASDM***\n",
        "\n",
        "Import of ASDM data and conversion into the MeasurementSet format is achieved via the **importasdm** task. The task settings are a little different to those used when importing interferometer ASDMs. For instance, *bdfflags=False* will de-select application of the BDF format flags, which are of no consequence in the normal reduction process, and *with_pointing_correction=True* will directly apply the *measured* pointing direction (in Azimuth and Elevation) to the data, rather than the *commanded* pointing positions. For example:\n",
        "\n",
        "```\n",
        "importasdm(asdm, asis='Antenna Station Receiver Source CalAtmosphere CalWVR CorrelatorMode SBSummary', bdfflags=False, process_caldevice=False, with_pointing_correction=True)\n",
        "```\n",
        "\n",
        "\n",
        "**2. List observations parameters: listobs**\n",
        "\n",
        "**listobs** works for single dish observations in the same way as for interferometry, and will detail the calibration scans.  It identifies pointing, $T_{sys}$, and target scans, the science target and OFF scans and their cadence, the correlator frequencies and configuration, and the antennas used in the observations. Unique to the single-dish data is the \\#OFF_SOURCE intent. This task is used to identify which spectral windows are used for $T_{sys}$ observations.  This information is critical for those who wish to build their own calibrations.\n",
        "\n",
        "```\n",
        "listobs(sd_ms)\n",
        "```\n",
        "\n",
        "produces feedback in the logger (or optionally, to a file) from which we can determine which spectral windows are $T_{sys}$ observations and which are science observations.\n",
        "\n",
        "```python\n",
        "Spectral Windows: (25 unique spectral windows and 2 unique polarization setups)\n",
        "SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs\n",
        "\n",
        "  0  BB_1#SQLD                                       1   TOPO  195994.575   2000000.000   2000000.0 195994.5750        1  XX  YY\n",
        "  1  BB_2#SQLD                                       1   TOPO  197932.075   2000000.000   2000000.0 197932.0750        2  XX  YY\n",
        "  2  BB_3#SQLD                                       1   TOPO  207994.575   2000000.000   2000000.0 207994.5750        3  XX  YY\n",
        "  3  BB_4#SQLD                                       1   TOPO  209994.575   2000000.000   2000000.0 209994.5750        4  XX  YY\n",
        "  4  WVR#NOMINAL                                     4   TOPO  184550.000   1500000.000   7500000.0 187550.0000        0  XX\n",
        "  5  X436890472#ALMA_RB_05#BB_1#SW-01#FULL_RES     128   TOPO  196986.763    -15625.000   2000000.0 195994.5750        1  XX  YY\n",
        "  6  X436890472#ALMA_RB_05#BB_1#SW-01#CH_AVG         1   TOPO  195978.950   1796875.000   1796875.0 195978.9500        1  XX  YY\n",
        "  7  X436890472#ALMA_RB_05#BB_2#SW-01#FULL_RES     128   TOPO  198924.263    -15625.000   2000000.0 197932.0750        2  XX  YY\n",
        "  8  X436890472#ALMA_RB_05#BB_2#SW-01#CH_AVG         1   TOPO  197916.450   1796875.000   1796875.0 197916.4500        2  XX  YY\n",
        "  9  X436890472#ALMA_RB_05#BB_3#SW-01#FULL_RES     128   TOPO  207002.388     15625.000   2000000.0 207994.5750        3  XX  YY\n",
        "  10 X436890472#ALMA_RB_05#BB_3#SW-01#CH_AVG         1   TOPO  207978.950   1796875.000   1796875.0 207978.9500        3  XX  YY\n",
        "  11 X436890472#ALMA_RB_05#BB_4#SW-01#FULL_RES     128   TOPO  209002.388     15625.000   2000000.0 209994.5750        4  XX  YY\n",
        "  12 X436890472#ALMA_RB_05#BB_4#SW-01#CH_AVG         1   TOPO  209978.950   1796875.000   1796875.0 209978.9500        4  XX  YY\n",
        "  13 BB_1#SQLD                                       1   TOPO  183375.638   2000000.000   2000000.0 183375.6378        1  XX  YY\n",
        "  14 BB_2#SQLD                                       1   TOPO  181427.463   2000000.000   2000000.0 181427.4627        2  XX  YY\n",
        "  15 BB_3#SQLD                                       1   TOPO  169374.840   2000000.000   2000000.0 169374.8404        3  XX  YY\n",
        "  16 BB_4#SQLD                                       1   TOPO  170917.638   2000000.000   2000000.0 170917.6378        4  XX  YY\n",
        "  17 X1857092512#ALMA_RB_05#BB_1#SW-01#FULL_RES   4096   TOPO  183162.808       122.070    500000.0 183412.7471        1  XX  YY\n",
        "  18 X1857092512#ALMA_RB_05#BB_1#SW-01#CH_AVG        1   TOPO  183412.686    500000.000    500000.0 183412.6861        1  XX  YY\n",
        "  19 X1857092512#ALMA_RB_05#BB_2#SW-01#FULL_RES   4096   TOPO  181177.524       122.070    500000.0 181427.4627        2  XX  YY\n",
        "  20 X1857092512#ALMA_RB_05#BB_2#SW-01#CH_AVG        1   TOPO  181427.402    500000.000    500000.0 181427.4017        2  XX  YY\n",
        "  21 X1857092512#ALMA_RB_05#BB_3#SW-01#FULL_RES   4096   TOPO  169587.670   -122.070    500000.0 169337.7310        3  XX  YY\n",
        "  22 X1857092512#ALMA_RB_05#BB_3#SW-01#CH_AVG        1   TOPO  169337.670    500000.000    500000.0 169337.6700        3  XX  YY\n",
        "  23 X1857092512#ALMA_RB_05#BB_4#SW-01#FULL_RES   4096   TOPO  171158.788   -122.070    500000.0 170908.8487        4  XX  YY\n",
        "  24 X1857092512#ALMA_RB_05#BB_4#SW-01#CH_AVG        1   TOPO  170908.788    500000.000    500000.0 170908.7877        4  XX  YY\n",
        "```\n",
        "\n",
        "From this output, we see the science spectral windows are 17, 19, 21 and 23, and have 4096 channels, while the $T_{sys}$ spectral windows at 5, 7, 9 and 11 have 128 channels.\n",
        "\n",
        "\n",
        "**3. A priori flagging: *flagcmd/flagdata***\n",
        "\n",
        "**flagcmd** works the same way on single-dish data as for interferometry. In this case, invoking it here applies flagging, by default, from the FLAG_CMD file within the MeasurementSet.\n",
        "\n",
        "```\n",
        "flagcmd(vis = 'uid___A002_Xb978c3_X5c4b.ms', inpmode = 'table', useapplied = True, action = 'apply')\n",
        "```\n",
        "\n",
        "**flagdata** is used at this point to remove problematic data. Conventionally, 5% of the edges of the bands are removed, as these parts of the band are significantly and detrimentally affected by the low-sensitivity edges of the filter passband. In principle, they can be retained in the cases where spectral lines of interest fall in that area, though the sensitivity losses are significant.\n",
        "\n",
        "Users should examine their spectra using **plotms**, and ensure any atmospheric lines are properly accounted for. This is particularly true for Band 5 which has a strong atmospheric absorption line at $\\sim$ 183 GHz. There is no real way to remove the signature of the atmospheric lines in position-switched data, since the elevations of the ON (science target) and OFF (sky-calibration position) are almost always different, and therefore have different air masses. The most effective approach in this case is to complete the normal calibrations as described here, then apply a judiciously-selected bandpass correction polynomial and spectral window channel range, as described by the **sdbaseline** step below.\n",
        "\n",
        "```\n",
        "flagdata(vis=vis, mode='manual', spw='17:0~119;3960~4079,19:0~119;3960~4079,21:0~119;3960~4079,23:0~119;3960~4079', action='apply', flagbackup=True)\n",
        "```\n",
        "\n",
        "Both steps **flagcmd** and **flagdata** are generally useful, but care should be taken in case the emission lines of interest are being inadvertantly flagged out.\n",
        "\n",
        "\n",
        "**4. Generation of the **$T_{sys}$ and $T_{sky}$ calibration tables: *gencal, sdcal and applycal*\n",
        "\n",
        "There are two ways to proceed in CASA when computing and applying calibration tables for single dish observations.\n",
        "\n",
        "1.  Build the $T_{sys}$ calibration tables with **gencal**, build the sky calibration tables with **sdcal,** and apply them with **applycal**\n",
        "2.  Build and apply both the $T_{sys}$ and sky calibration tables with **sdcal**\n",
        "\n",
        "The second option is faster, but users familar with the **gencal** and **applycal** tasks may prefer the first option.\n",
        "\n",
        "In either case, the mapping between the $T_{sys}$ scans and science scans must be determined either by examination of the output of **listobs**, or by running the **sdcal** and specifying the method to be used to obtain the OFF position. Usually ALMA will take position-switched observations via *mode=*\\'*ps*\\', though other alternatives exist which do not need any OFF positions to be explicitly observed. The OFF can be obtained from the source data itself via *mode=*\\'*otfraster*\\' or *mode=*\\'*off*\\'.\n",
        "\n",
        " \n",
        "\n",
        "In the first of the two cases mentioned above (having identified the target spectral windows as 17,19,21 and 23, and using a target identified by the variable name, \\\"source\\\") :\n",
        "\n",
        "```\n",
        "gencal(vis = sd_ms, caltable = sd_ms+'.tsys', caltype = 'tsys')\n",
        "\n",
        "sdcal(infile = sd_ms, outfile = sd_ms+'.sky', calmode = 'ps')\n",
        "\n",
        "from recipes.almahelpers import tsysspwmap\n",
        "tsysmap = tsysspwmap(vis = sd_ms, tsystable = sd_ms+'.tsys', trim = False)\n",
        "\n",
        "applycal(vis = sd_ms, applymode = 'calflagstrict', spw = '17,19,21,23', field = source, gaintable = [sd_ms+'.tsys', sd_ms+'.sky'], gainfield = ['nearest', source], spwmap = tsysmap)\n",
        "```\n",
        "\n",
        "\n",
        "In the second case:\n",
        "\n",
        "```\n",
        "sdcal(infile=sd_ms, calmode='ps,tsys,apply')\n",
        "```\n",
        "\n",
        "Note that we didn\\'t specify the $T_{sys}$ spectral windows in the call to **sdcal. ** For ALMA single-dish data from Cycle 3 onward, this is okay since the $T_{sys}$ and science data share the same spectral window.   Alternatively, the mapping between the $T_{sys}$ and science spectral windows can be explicitly set with *spwmap* and *spw.* In this case, we would use:\n",
        "\n",
        "```\n",
        "sdcal(infile=vis, calmode='ps,tsys,apply', spwmap={17:[17], 19:[19], 21:[21],23:[23]}, spw='17,19,21,23')\n",
        "```\n",
        "\n",
        "The general structure of *spwmap* is {Tsys spw 0: \\[science spw 0\\],\\....,Tsys spw n: \\[science spw n\\]} for 0 to n spectral windows.\n",
        "\n",
        "**gencal** applied at this stage builds (and optionally applies) the $T_{sys}$ calibration tables. These calibrations are an intrinsic part of the ASDM. There are no re-computations applied to the $T_{sys}$ data by CASA. Ultimately, the $T_{sys}$ calibration tables will be applied in the **applycal** step, consistent with the descriptions of calibrations given in the sections above. We point out that the $T_{sys}$ calibrations are a multiplicative factor, so the order of the application of the $T_{sys}$ cal tables relative to the application of the $T_{sky}$ calibrations is immaterial.\n",
        "\n",
        "**8. Subtracting the baseline: *sdbaseline***\n",
        "\n",
        "It\\'s important at this point to define exactly what is meant by *baseline* in the context of single-dish data. In interferometry, *baseline* refers to the spatial separation of antenna pairs.  For a single dish observation, *baseline* refers to the spectral pattern produced by the atmosphere and instrument. Since single-dish antennas measure total power, not an interference pattern, they are responsive to emission wherever it exists within the single-dish beam or signal path. This signal is dominated by the receiver/correlator/backend sampling function, but has a significant time-varying component usually dominated by atmospheric fluctuations. The power yielded by atmospheric fluctuations are invisible to interferometer observations, as they are in the *near field*, and are therefore generally resolved out from the data.  Note, though, that the atmospheric variability can contaminate interferometric measurements by introducing a decoherence in phase, and such losses in phase are not relevant for single-dish observations.\n",
        "\n",
        "**sdbaseline** removes a spectral baseline from the data on a per-integration basis. The options here are extensive, and baseline subtraction can be complex when emission is strongly variable throughout the map, or when there are nearby atmospheric absorption features.  But CASA is effective at choosing intelligent defaults with *mode=\\'auto\\'*. With *mode=\\'auto\\'*, CASA will examine the brightness variability per integration and determine the most appropriate channel ranges for computing the spectral baseline, based on the mean absolute deviation of the channels. This approach is successful even when applied to spectra crowded heavily with emission lines.  As long as the emission-free parts of the spectrum have statistically significant representation in the data, then the *mode=\\'auto\\'* will be successful. Baseline corrections employed by CASA are subtracted, and therefore can be applied iteratively, as needed.\n",
        "\n",
        "**sdbaseline** supports Polynomial, Chebychev and Sinusoid baseline removal. Sinusoidal baselines are determined with a Fourier transform of the spectral data - again, an automatic mode is available, where CASA will determine the most significant Fourier components and remove them, though specific wavenumbers can be explicitly added or removed on top of the automatic operation. Sinusoidal components occur in many single-dish telescopes, and are a typical manifestation of a standing-wave resonation of the main-reflector/subreflector cavity. ALMA has employed scattering cones in the single-dish subreflectors to effectively mitigate the strength of this standing wave. It\\'s worth noting that removal of Fourier components should be applied with utmost caution; the result is effectively a convolution of the spectra with a spectral filter, and MUST affect the resulting emission spectra. Users who use this baseline mode should explore and characterize the consequences and subsquent error propagation, in the context of their own data.In this example, we remove a 1st order polynomial from spectral windows 17, 19, 21 & 23, automatically finding and masking out any lines brighter than 5 $\\sigma$, and referencing the \\\"corrected\\\" (i.e. calibrated) data column.\n",
        "\n",
        "```\n",
        "sdbaseline(infile = sd_ms, datacolumn = 'corrected', spw = '17,19,21,23', maskmode = 'auto', thresh = 5.0, avg_limit = 4, blfunc = 'poly', order = 1, outfile = sd_ms+'.cal')\n",
        "```\n",
        "\n",
        "Note that at this point, the product dataset (sd_ms+\\'.cal) has only four spectral windows. These are (if all is going well) the science observations which are $T_{sys}$ and sky calibrated, and are now bandpass-corrected.\n",
        "\n",
        "\n",
        "**9. Convert the Science Target Units from Kelvin to Jansky: *scaleAutocorr***\n",
        "\n",
        "To convert the units of the single-dish observations from $T_A^*$ (K) into Janskys and to prepare for combination with interferometer data, we need to obtain the empirically-determined Jy-to-K conversion data. These data already take into account any correlator non-linearities and also factor in the various subsystem efficiencies. \n",
        "\n",
        "The easiest way to obtain this is simply with a call to a specialized CASA task that obtains the Jy-to-K factors that accesses polynomal fits from ongoing calibration campaign data.\n",
        "\n",
        "```\n",
        "jyperk = es.getJyPerK(sd_ms+'.cal')\n",
        "```\n",
        "\n",
        "The contents of this variable jyperk is a python dictionary:\n",
        "\n",
        "```\n",
        "for ant in jyperk.keys():\n",
        "for spw in jyperk[ant].keys():\n",
        "scaleAutocorr(vis=sd_ms+'.cal', scale=jyperk[ant][spw]['mean'], antenna=ant, spw=spw)\n",
        "```\n",
        "\n",
        "\n",
        "**scaleAutocorr** simply applies the scaling from $T_A^*$ to Jy/beam. The scaling factors are determined empirically, as part of the QA2-level calibrations provided by ALMA. The scaling factors are to be provided to **scaleAutocorr** as a float, but are most conveniently applied in calls that iterate through antenna and spectral window, where the Jy-per-K factors are retained as a list with the format:\n",
        "\n",
        "\n",
        "```python\n",
        " jyperk = \n",
        "\n",
        "  { antenna01_name { spw0: { 'mean': 44.345, 'n': '', 'std': ''},\n",
        "\n",
        "              spw1: { 'mean': 44.374, 'n': '', 'std': ''},\n",
        "\n",
        "              spw2: { 'mean': 44.227, 'n': '', 'std': ''},\n",
        "\n",
        "              spw3: { 'mean': 44.203, 'n': '', 'std': ''}},\n",
        "\n",
        "    antenna02_name: { spw0: { 'mean': 44.345, 'n': '', 'std': ''},\n",
        "\n",
        "              spw1: { 'mean': 44.374, 'n': '', 'std': ''},\n",
        "\n",
        "              spw2: { 'mean': 44.227, 'n': '', 'std': ''},\n",
        "\n",
        "              spw3: { 'mean': 44.203, 'n': '', 'std': ''}}}\n",
        "```\n",
        "\n",
        " \n",
        "\n",
        "which can be iterated and applied to the actual data with the following loop:\n",
        "\n",
        "```\n",
        "\n",
        "to_amp_factor = lambda x: 1. / sqrt(x)\n",
        "\n",
        "for ant in jyperk.keys():\n",
        "\n",
        "   factors=[]\n",
        "\n",
        "   for spw in jyperk[ant].keys():\n",
        "\n",
        "      factors.append(jyperk[ant][spw]['mean'])\n",
        "\n",
        "   gencal(vis=sd_ms, caltable=sd_ms+'.jy2ktbl', caltype='amp', spw=\",\".join(str(x) for x in jyperk[ant].keys()), parameter=map(to_amp_factor, factors))\n",
        "\n",
        "   applycal(vis=sd_ms+'.cal', gaintable=sd_ms+'.jy2ktbl')\n",
        "```\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kQtn8sfa0EDx"
      },
      "source": [
        "## Future Development Goals \n",
        "\n",
        "The top development priority for CASA single-dish data reduction is the conversion to the MeasurementSet format for the full reduction process. Dispensing with the ASAP (scan table) format will help unify data processing by providing a more homogenous basis for data processing, reduction, and analysis, as well as streamline and make more versatile the development aspects; current and future developers need only be familiar with one overall format of data. As such, ASAP will be gone in 5.1. However, please note that the **importasap** task is the only exception and will not go away even after removing ASAP. This task supports importing existing scantables in CASA for backward compatibility.There is also significant active development to add header information to plots made by **plotms**. While preserving the multi-plot capability and building a layout that will not cramp or obfuscate the plot output, header information (i.e. target name, frequency, integration time, etc.) can be optionally added to the plots output by **plotms**.There are a number of additional improvements that we do not detail here. ALMA-related development requests and bug notices can be sent to the CASA Single Dish Team via the [ALMA Helpdesk](https://help.almascience.org/).\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    }
  ]
}