Skip to content

Commit

Permalink
fixes on RS programmer man text.
Browse files Browse the repository at this point in the history
  • Loading branch information
kiloRomeoAlpha committed May 16, 2019
2 parents 1758553 + ced1d2c commit 1efc729
Show file tree
Hide file tree
Showing 10 changed files with 234 additions and 60 deletions.
27 changes: 14 additions & 13 deletions geminidr/doc/tutorials/F2Img-DRTutorial/01_introduction.rst
Original file line number Diff line number Diff line change
Expand Up @@ -51,12 +51,11 @@ be installed. If you have an error message, make sure:
Download Sample Files
=====================

This tutorial will use observations from program GS-2013B-Q-15 (PI:Leggett),
This tutorial will use observations from program GS-2013B-Q-15 (PI: Leggett),
NIR photometry of the faint T-dwarf star WISE J041358.14-475039.3, obtained on
2013-Nov-21. Images of this sparse field were obtained in the Y, J, H, Ks bands
using a dither sequence; dayCal DARKS and GCAL flats were obtained as well.
Leggett, et al. (2015; `[L15]
<https://ui.adsabs.harvard.edu/#abs/2015ApJ...799...37L/abstract>`_)
`Leggett, et al. (2015) <https://ui.adsabs.harvard.edu/#abs/2015ApJ...799...37L/abstract>`_
briefly describes the data reduction procedures they followed, which are
similar to those described below.

Expand All @@ -68,44 +67,46 @@ chosen program.

::

https://archive.gemini.edu/searchform/GS-2013B-Q-15-39
https://archive.gemini.edu/searchform/GS-2013B-Q-15-39/RAW/cols=CTOWEQ/notengineering/NotFail

The bottom of the page contains a button to download the data. You can click on
that, or you can download the images by `clicking directly
here <https://archive.gemini.edu/download/GS-2013B-Q-15-39/present/canonical>`_.
here <https://archive.gemini.edu/download/GS-2013B-Q-15-39/RAW/present/NotFail/notengineering/canonical>`_.
Alternatively, you can download the data by copy-and-pasting the address below
in your browser:

::

https://archive.gemini.edu/download/GS-2013B-Q-15-39/present/canonical
https://archive.gemini.edu/download/GS-2013B-Q-15-39/RAW/present/NotFail/notengineering/canonical

After retrieving the science data, click the Load Associated Calibrations tab on
the search results page and download the associated dark and flat-field
exposures. Again, the calibration files can be downloaded by `clicking here
<https://archive.gemini.edu/download/associated_calibrations/GS-2013B-Q-15-39/canonical>`_
<https://archive.gemini.edu/download/associated_calibrations/GS-2013B-Q-15-39/RAW/NotFail/notengineering/canonical>`_
or by copying the following URL to your browser:

::

https://archive.gemini.edu/download/associated_calibrations/GS-2013B-Q-15-39/canonical
https://archive.gemini.edu/download/associated_calibrations/GS-2013B-Q-15-39/RAW/NotFail/notengineering/canonical

Unpack all of them in a subdirectory of your working directory (assumed to be
named /raw in this tutorial). Be sure to uncompress the files.
named ``/raw`` in this tutorial). Be sure to uncompress the files.

.. code-block:: bash
$ cd <my_main_working_directory>
$ tar -xvf *calib*.tar # extract calibration files from .TAR file
$ tar -xvf *calib*.tar # extract calibration files from .TAR file
$ tar -xvf *data*.tar # extract science files from .TAR file
$ tar -xvf *data*.tar # extract science files from .TAR file
$ bunzip2 *.fits.bz2 # command that will decompress FITS files
$ bunzip2 *.fits.bz2 # command that will decompress FITS files
$ rm *_dark.fits *_flat.fits # remove pre-reduced data (recommended)
$ mkdir raw/ # create directory named "raw" (optional)
$ mv *.fits raw/ # move all the raw FITS files to raw (optional)
$ mv *.fits raw/ # move all the raw FITS files to raw (optional)
The full de-compressed data set will have 310 files and use 4.9 Gb of disk
space.
40 changes: 18 additions & 22 deletions geminidr/doc/tutorials/F2Img-DRTutorial/02_data_reduction.rst
Original file line number Diff line number Diff line change
Expand Up @@ -54,22 +54,19 @@ time available. If you don't know what are the existing exposure times, you can

.. code-block:: bash
$ dataselect --tags DARK --xtags PROCESSED raw/*.fits | showd -d exposure_time
$ dataselect --tags DARK raw/*.fits | showd -d exposure_time
The ``|`` is what we call "pipe" and it is used to pass output from dataselect_
to showd_. The following line creates a list of DARK files that were not
processed and that have exposure time of 20 seconds:

.. code-block:: bash
$ dataselect --tags DARK --xtags PROCESSED \
--expr "exposure_time==20" raw/*.fits > darks_020s.list
$ dataselect --tags DARK --expr "exposure_time==20" raw/*.fits > darks_020s.list
The ``\`` is simply a special character to break the line. The ``--tags`` is a
comma-separated argument that is used to select the files that matches the
tag(s) listed there. ``--xtags`` is used to exclude the files which tags
matches the one(s) listed here. ``--expr`` is used to filter the files based
on their attributes. Here we are selecting files with exposure time of
The ``--tags`` is a comma-separated argument that is used to select the files
that matches the tag(s) listed there. ``--expr`` is used to filter the
files based on their attributes. Here we are selecting files with exposure time of
20 seconds. You can repeat the same command for the other existing exposure
times (3 s, 8 s, 15 s, 60 s, 120 s). Use ``dataselect --help`` for more
information.
Expand All @@ -92,8 +89,7 @@ same exposure times:

.. code-block:: bash
$ dataselect --tags DARK --xtags PROCESSED \
--expr "exposure_time==120" raw/*.fits > darks_120s.list
$ dataselect --tags DARK --expr "exposure_time==120" raw/*.fits > darks_120s.list
And then pass this list to the `reduce`_ command.

Expand Down Expand Up @@ -151,8 +147,7 @@ not interfere in the results.

.. code-block:: bash
$ dataselect --tags FLAT --xtags PREPARED \
--expr "filter_name=='Y'" raw/*.fits > flats_Y.list
$ dataselect --tags FLAT --expr "filter_name=='Y'" raw/*.fits > flats_Y.list
$ reduce @flats_Y.list @darks_003s.list -r makeProcessedBPM
Expand All @@ -170,8 +165,7 @@ corresponding files for each filter:

.. code-block:: bash
$ dataselect --tags FLAT --xtags PREPARED \
--expr "filter_name=='Y'" raw/*.fits > flats_Y.list
$ dataselect --tags FLAT --expr "filter_name=='Y'" raw/*.fits > flats_Y.list
.. note::
Expand Down Expand Up @@ -212,13 +206,13 @@ Now that we have the Master Dark and Master Flat images, we can tell `reduce`_
to process our data. `reduce`_ will look at the remote or at the local database
for calibration files. Make sure that you have `configured your database <caldb>`_
before running it. We want to run `reduce`_ on any file that is not calibration
nor a bad-pixel-mask (``--xtags CAL,BPM``). We also want to run this pipeline
only on Y band images (``--expr 'filter_name=="Y"'``)
nor a bad-pixel-mask. To exclude files from our selection we can use the
``--xtags``, e.g., ``--xtags CAL,BPM``. We also want to run this pipeline
only on Y band images (``--expr 'filter_name=="Y"'``).

.. code-block:: bash
$ dataselect --xtags CAL,BPM --expr 'filter_name=="Y"' \
raw/*.fits > sci_images_Y.list
$ dataselect --xtags CAL,BPM --expr 'filter_name=="Y"' raw/*.fits > sci_images_Y.list
$ reduce @sci_images_Y.list
Expand All @@ -234,12 +228,14 @@ files will be stacked together in a single file.
The science exposures in all bands suffer from vignetting of the field in
the NW quadrant (upper left in the image above). This may have been caused
by the PWFS2 guide probe, which was used because of a hardware problem with
the OIWFS (see the F2 instrument status note for 2013 Sep. 5). Therefore the
photometry of this portion of the image will be seriously compromised.
the OIWFS (see the `F2 instrument status note <https://www.gemini.edu/sciops/instruments/flamingos2/status-and-availability>`_
for 2013 Sep. 5). Therefore the photometry of this portion of the image will
be seriously compromised.

The final product file will have a ``_stack.fits`` sufix and it is shown below:

.. the figure below can be created using the script inside the ``savefig``
folder.
.. figure:: _static/S20131121S0075_stack.fits.png
:align: center

S20131121S0075_stack.fits.png
3 changes: 2 additions & 1 deletion geminidr/doc/tutorials/F2Img-DRTutorial/03_api_reduction.rst
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,8 @@ DARK frame for the files that have 20s exposure time:
Note the empty list ``[]`` in the fourth line of each command. This
position argument receives a list of tags that will be used to exclude
any files with the matching tag from our selection.
any files with the matching tag from our selection (i.e., equivalent to the
``--xtags`` option).

Now you must create a list of FLAT images for each filter. You can do that by
using the following commands:
Expand Down
3 changes: 2 additions & 1 deletion geminidr/doc/tutorials/F2Img-DRTutorial/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ DRAGONS - Flamingos 2 Data Reduction Tutorial
This is a brief tutorial on how to reduce Flamingos-2 images using DRAGONS (Data
Reduction for Astronomy from Gemini Observatory North and South). It is based on
information found in the `GEMINI Flamingos-2
WebPage <https://www.gemini.edu/sciops/instruments/flamingos2/>`_.
WebPage <https://www.gemini.edu/sciops/instruments/flamingos2/>`_ and in the
`DRAGONS Documentation on Read The Docs <https://dragons.readthedocs.io/en/latest/>`_.

.. In order to carry on, you will need `Python <https://www.python.org/>`_ (which
is probably installed on your machine) and a few libraries. We recommend Python 3
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
#!/usr/bin/env python
"""
Script created to generate a PNG file of the final stack produced during the
Flamingos-2 Tutorial
"""

import astrodata
import gemini_instruments
import numpy as np
import os

from astropy import visualization, wcs
from copy import copy
from matplotlib import pyplot as plt


def main():

filename = get_stack_filename()

ad = astrodata.open(filename)

data = ad[0].data
mask = ad[0].mask
header = ad[0].hdr

masked_data = np.ma.masked_where(mask, data, copy=True)

palette = copy(plt.cm.viridis)
palette.set_bad('gray')

norm_factor = visualization.ImageNormalize(
masked_data,
stretch=visualization.LinearStretch(),
interval=visualization.ZScaleInterval(),
)

fig, ax = plt.subplots(subplot_kw={'projection': wcs.WCS(header)})

ax.imshow(masked_data,
cmap=palette,
vmin=norm_factor.vmin,
vmax=norm_factor.vmax)

ax.set_title(os.path.basename(filename))
ax.set_xticklabels([])
ax.set_yticklabels([])

fig.savefig(filename.replace('.fits', '.png'))
plt.show()


def get_stack_filename():

import argparse

parser = argparse.ArgumentParser()

parser.add_argument('filename', type=str,
help='Path to the stack fits file')

args = parser.parse_args()

return args.filename


if __name__ == main():
main()
11 changes: 5 additions & 6 deletions geminidr/doc/tutorials/GSAOIImg-DRTutorial/01_introduction.rst
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ machine. You can test that by typing the following commands:
Where ``geminiconda`` is the name of the conda environment where DRAGONS should
be installed. If you have an error message, make sure:

- Conda is properly installed;
- AnaConda or MiniConda is properly installed;

- A Conda Virtual Environment is properly created and is active;

Expand All @@ -51,9 +51,8 @@ be installed. If you have an error message, make sure:
Download Sample Files
=====================

For this tutorial we selected a very sparse field. This makes the sky
subtraction less prone to errors. The selected data set was observed for the
GS-2017A-Q-29 program on the night starting on May 04, 2017.
For this tutorial we selected data observed for for the GS-2017A-Q-29 program on
the night starting on May 04, 2017.

You can search and download the files on the
`Gemini Archive <https://archive.gemini.edu/searchform>`_ using the
Expand All @@ -78,14 +77,14 @@ can be downloaded using `this link
<https://archive.gemini.edu/download/exposure_time=150/notengineering/GSAOI/Pass/DARK/present/canonical>`_.
or copying and pasting the following address to search form into your browser: ::

.. https://archive.gemini.edu/searchform/exposure_time=150/cols=CTOWEQ/notengineering/GSAOI/Pass/DARK
https://archive.gemini.edu/searchform/exposure_time=150/cols=CTOWEQ/notengineering/GSAOI/Pass/DARK


The FLAT files to build the BPM can be downloaded directly by `clicking here
<https://archive.gemini.edu/download/20171201-20171231/object=Domeflat/filter=H/notengineering/GSAOI/Pass/present/canonical>`_
or using the following address: ::

.. https://archive.gemini.edu/searchform/object=Domeflat/cols=CTOWEQ/filter=H/notengineering/GSAOI/20171201-20171231/Pass
https://archive.gemini.edu/searchform/object=Domeflat/cols=CTOWEQ/filter=H/notengineering/GSAOI/20171201-20171231/Pass


Copy all the files to the same place in your computer. Then use ``tar`` and
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ For this tutorial, we will be also using other `Supplemental tools
<https://dragons-recipe-system-users-manual.readthedocs.io/en/latest/supptools.html>`_,
like dataselect_, showd_, typewalk_, and caldb_.

.. warning:: Some primitives use a lot of RAM memory and they can make `reduce`
.. warning:: Some primitives use a lot of RAM memory and they can make reduce_
crash. Our team is aware of this problem and we are working on that. For
now, if that happens to you, you might need to run the pipeline on a
smaller data set.
Expand Down Expand Up @@ -83,8 +83,8 @@ reduction pipeline does not organize the data.

That means that we first need to identify these files and create lists that will
be used in the data-reduction process. For that, we will use the dataselect_
command line. Please, refer to the `dataselect page <dataselect>`_ for details
regarding its usage. Let us start with the DARK files: ::
command line. Please, refer to the dataselect_ page for details regarding its
usage. Let us start with the DARK files: ::

$ dataselect --tags DARK raw/*.fits > list_of_darks.txt

Expand Down

0 comments on commit 1efc729

Please sign in to comment.