Skip to content

Latest commit

 

History

History
161 lines (122 loc) · 4.6 KB

index.rst

File metadata and controls

161 lines (122 loc) · 4.6 KB
Raw

CARACal workflow

A CARACal run involves running a sequence of CARACal workers following the order in which they are listed in the user configuration file. For reference see workerlist.

Users can design their own CARACal workflow with some level of flexibility. The exact workflow (i.e., the exact sequence of CARACal workers) will depend on science goals but also on I/O constraints. Below we provide two examples.

Recommended workflow

The currently recommended CARACal workflow minimises the data volume increase during the course of a full CARACal run, and allows to treat the input .MS file(s) as read-only.

This workflow is implemented in all sample configuration files located at https://github.com/caracal-pipeline/caracal/tree/master/caracal/sample_configurations .

general
Compulsory worker to set up data/input/output directories.
getdata
Compulsory worker to specify the input .MS files.
obsconf
Compulsory worker to set up target/calibrators names.
transform
Split calibrators-only .MS files, one per input .MS file.
prep
Prepare the calibrators-only .MS files for processing.
flag
Flag the calibrators-only .MS files.
crosscal
Derive the cross-calibration tables and apply them to the calibrators.
inspect
Inspect the calibrated calibrator's visibilities to check the quality of the
cross-calibration.
transform
Split target-only .MS files, one per input .MS file and target, applying the
cross-calibration on the fly.
prep
Prepare the target-only .MS files for processing.
flag
Flag the target-only .MS files.
transform
Average the target-only .MS files in frequency for continuum imaging.
flag
Flag line channels in the averaged target-only .MS files.
selfcal
Make a continuum image of each target, self-calibrate, and transfer both gains
and continuum model to the full-frequency-resolution target-only .MS files.
line
Subtract the continuum, Doppler correct and make the line cube and moment
images from the target-only .MS files.
mosaic
Mosaic the continuum images of the targets.
mosaic
Mosaic the line cubes.

Note that this workflow includes multiple runs of several workers. These require a "__<suffix>" in the configuration file, as described at the page configfile.

The data volume budget of this workflow is:

  • input .MS:
    • DATA
  • target-only .MS:
    • DATA
    • CORRECTED_DATA
    • MODEL
  • calibrators-only .MS (negligible data volume):
    • DATA
    • CORRECTED_DATA
    • MODEL
  • target-only frequency-averaged .MS (negligible data volume):
    • DATA
    • CORRECTED_DATA
    • MODEL

Simple workflow

Simpler workflows than the recommended one are possible. For example, the workflow below employs less worker runs. However, it modifies the input .MS file(s), results into a larger data volume increase, and runs the self-calibration loop on larger .MS files.

general
Compulsory worker to set up data/input/output directories.
getdata
Compulsory worker to specify the input .MS files.
obsconf
Compulsory worker to set up target/calibrators names.
prep
Prepare the input .MS files for processing.
flag
Flag the calibrators in the input .MS files.
crosscal
Derive the cross-calibration tables and apply them to target and calibrators.
inspect
Inspect the calibrated calibrator's visibilities to check the quality of the
cross-calibration.
transform
Split cross-calibrated target-only .MS files, one per input .MS file and target.
flag
Flag the target-only .MS files.
selfcal
Make a continuum image of each target, self-calibrate, and transfer both gains
and continuum model to the full-frequency-resolution target-only .MS files.
line
Subtract the continuum, Doppler correct and make the line cube and moment
images from the target-only .MS files.
mosaic
Mosaic the continuum images of the targets.
mosaic
Mosaic the line cubes.

The data volume budget of this workflow is:

  • input .MS
    • DATA
    • CORRECTED_DATA
    • MODEL
  • target-only .MS
    • DATA
    • CORRECTED_DATA
    • MODEL

Typically, this is 50% larger than for the recommended above.