Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Observation table subset selection #295

Merged
merged 19 commits into from
Aug 4, 2015
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
ff80935
Fixed problem with header in make_test_observation_table. Defining a …
mapazarr Jun 29, 2015
c8dcb4e
Renaming info method to summary in astropy Table subclasses, to avoid…
mapazarr Jun 29, 2015
8e5f910
Added generic function to filter observations on any variable specifi…
mapazarr Jun 29, 2015
33e2395
Testing filters on Quantity and Angle objects.
mapazarr Jun 29, 2015
5a002ab
Added time utility function to get a human-readable date and time fro…
mapazarr Jun 30, 2015
7343cb7
Finished implementing 1D selection criteria for filtering observation…
mapazarr Jul 1, 2015
3e7b0d4
Updating in the tests the name change of info -> summary method of cl…
mapazarr Jul 1, 2015
1106913
Fixed sphinx docstring warning.
mapazarr Jul 1, 2015
559ce72
Updated test values for filter observations.
mapazarr Jul 30, 2015
9837217
Fixed indentation.
mapazarr Jul 30, 2015
6f7ae9c
Major revision of filter observation table methods.
mapazarr Aug 1, 2015
d6e3125
Modifications needed after 4 times rebase because of PR #310.
mapazarr Aug 1, 2015
697b4f9
Added entry to CHANGE log.
mapazarr Aug 1, 2015
b037fda
Producing a PRELIMINARY working version of find-obs. Renaming filter …
mapazarr Aug 3, 2015
1611117
Updated PR name in CHANGES log.
mapazarr Aug 3, 2015
5755f3e
Updated PR name in CHANGES log once more.
mapazarr Aug 3, 2015
fa9cf95
Small changes and fixes for code related to observation selection.
mapazarr Aug 3, 2015
9e1072e
Using logging in find_obs.
mapazarr Aug 4, 2015
1827d16
Working on select observations. Improved inline command: make it func…
mapazarr Aug 4, 2015
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGES.rst
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,7 @@ Pull requests
- Add Feldman Cousins algorithm [#311] (Dirk Lennarz)
- Simplified attribute docstrings [#301] (Manuel Paz Arribas)
- Add a simple Makefile for Gammapy. [#289] (Manuel Paz Arribas)
- Observation table subset selection [#295] (Manuel Paz Arribas)

.. _gammapy_0p2_release:

Expand Down
15 changes: 11 additions & 4 deletions docs/dataformats/observation_lists.rst
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,8 @@ DEC pointing position declination in equatorial (ICRS) coordinates
AZ average azimuth angle during the observation no
ALT average altitude angle during the observation no
MUON_EFFICIENCY average muon efficiency of the telescopes yes?
TIME_START start time of the observation stored as number of seconds after [MET]_ yes?
TIME_START start time of the observation stored as number of seconds after [MET]_ or yes?
as absolute times in UTC (see description of header keyword `TIME FORMAT`)
TIME_STOP end time of the observation in the same format as TIME_START no
TIME_OBSERVATION duration of the observation no
TIME_LIVE duration of the observation without dead time no
Expand All @@ -51,11 +52,13 @@ keyword description
================ =========================================================================== =========
OBSERVATORY_NAME name of the observatory where the observations were taken. This is no
important for instance for coordinate transformations between celestial
(i.e. RA/dec) and terrestrial (i.e. az/alt) coordinate systems.
(i.e. RA/dec) and terrestrial (i.e. az/alt) coordinate systems
MJDREFI reference time: integer value in mean julian days; details in yes?
:ref:`time_handling`.
:ref:`time_handling`
MJDREFF reference time: fraction of integer value defined in MJDREFI; details in yes?
:ref:`time_handling`.
:ref:`time_handling`
TIME_FORMAT format in which times are stored: `absolute` (UTC) or `relative` ([MET]_); yes?
see details for both formats in :ref:`time_handling`
================ =========================================================================== =========

Extra user-defined header entries are allowed; Gammapy will ignore them.
Expand Down Expand Up @@ -97,3 +100,7 @@ Table:
+------+----------------+---------+-------------+-------------+-------------+-------------+-------------+--------------+------+---------------+
| 10| 1800.0| 1500.0|144207430.361|144209230.361|323.562657045|45.4005803262|324.596045439| 13.6761217326| 3| 0.694201696626|
+------+----------------+---------+-------------+-------------+-------------+-------------+-------------+--------------+------+---------------+

A test observation list file in fits format is located in the
`~gammapy-extra` repository
(`test_observation_table.fits <https://github.com/gammapy/gammapy-extra/blob/master/test_datasets/obs/test_observation_table.fits>`_).
73 changes: 73 additions & 0 deletions docs/obs/find_observations.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
.. _obs_find_observations:

Find observations
=================

The ``gammapy-find-obs`` command line tool can be used to select a
subset of observations from a given observation list. The format for
observation lists is described in :ref:`dataformats_observation_lists`.

This tool works with `fits` files as input/output for now.

For more details, please refer to `~gammapy.scripts.find_obs`.

.. note:: Noel Dawe's ``goodruns`` tool for `ATLAS <http://atlas.ch>`__ run selection
(`docs <http://ndawe.github.io/goodruns/>`__, `code <https://github.com/ndawe/goodruns>`__)
is a nice example for a run selection tool.

Examples
--------

The ``gammapy-find-obs`` tool has many options. Only a few examples
are shown here. For a full list of options, please use:

.. code-block:: bash

$ gammapy-find-obs --help

at the command line.

In order to test the examples below, the test observation list
file located in the `~gammapy-extra` repository
(`test_observation_table.fits <https://github.com/gammapy/gammapy-extra/blob/master/test_datasets/obs/test_observation_table.fits>`_)
can be used as input observation list.

* Find all observations within 5 deg of the Galactic center:

.. code-block:: bash

$ gammapy-find-obs all_obs.fits galactic_center_obs.fits \
--x 0 --y 0 --r 50 --system 'galactic'

* Select all observations in a box along the Galactic plane
(GLON = -20 .. +20 deg, GLAT = -3 .. +3 deg) (can of course be
combined with other selections shown above):

.. code-block:: bash

$ gammapy-find-obs all_obs.fits galactic_plane_obs.fits \
--x 0 --y 0 --dx 20 --dy 3 --system 'galactic'

* Select all observations in a given date range (can of course be
combined with other selections shown above):

.. code-block:: bash

$ gammapy-find-obs all_obs.fits obs_2010-04-26_to_2010-04-29-12h42.fits \
--t_start '2012-04-20' --t_stop '2012-04-30T12:42'

* Select all observations in a given observation ID range (can of
course be combined with other selections shown above):

.. code-block:: bash

$ gammapy-find-obs all_obs.fits obs_042_to_100.fits \
--par_name 'OBS_ID' --par_min 42 --par_max 101

* Select all observations in a given altitude range (can of course
be combined with other selections shown above):

.. code-block:: bash

$ gammapy-find-obs all_obs.fits alt_70_to_90_deg_obs.fits \
--par_name 'ALT' --par_min 70 --par_max 90
50 changes: 0 additions & 50 deletions docs/obs/findruns.rst

This file was deleted.

2 changes: 1 addition & 1 deletion docs/obs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ Using `gammapy.obs`
.. toctree::
:maxdepth: 1

findruns
find_observations

Reference/API
=============
Expand Down
4 changes: 2 additions & 2 deletions docs/references.rst
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,6 @@ Glossary
--------

.. [CSV] comma-separated values; see also :ref:`CSV_files`.
.. [MET] mission elapsed time; see also :ref:`time_handling`.
.. [MET] mission elapsed time; see also :ref:`MET` in :ref:`time_handling`.
.. [RST] restructured text; the markup format used for documentation and docstrings.
See `here <http://en.wikipedia.org/wiki/ReStructuredText>`__ and `here <http://sphinx-doc.org/rest.html>`__.
See `here <http://en.wikipedia.org/wiki/ReStructuredText>`__ and `here <http://sphinx-doc.org/rest.html>`__.
23 changes: 15 additions & 8 deletions docs/time/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -100,23 +100,30 @@ no need for explicit time scale transformations in Gammapy
The "Time Systems in a nutshell" section `here <http://fermi.gsfc.nasa.gov/ssc/data/analysis/documentation/Cicerone/Cicerone_Data/Time_in_ScienceTools.html>`__
gives a good, brief explanation of the differences between the relevant time scales `UT1`, `UTC` and `TT`.

.. _MET:

Mission elapsed times (MET)
---------------------------

It's not clear yet how to best implement METs, it's one of the tasks here:
https://github.com/gammapy/gammapy/issues/284

For now, we use the `gammapy.time.time_ref_from_dict` and `gammapy.time.time_relative_to_ref` functions
to convert MET floats to `~astropy.time.Time` objects via the reference times stored in FITS headers.

See `Cicerone: Data - Time in Fermi Data Analysis
[MET]_ time references are times representing UTC seconds after a
specific origin. Each experiment may have a different MET origin
that should be included in the header of the corresponding data
files. For more details see `Cicerone: Data - Time in Fermi Data Analysis
<http://fermi.gsfc.nasa.gov/ssc/data/analysis/documentation/Cicerone/Cicerone_Data/
Time_in_ScienceTools.html>`_.

It's not clear yet how to best implement METs in Gammapy, it's one
of the tasks here:
https://github.com/gammapy/gammapy/issues/284

For now, we use the `gammapy.time.time_ref_from_dict`, `gammapy.time.time_relative_to_ref`
and `gammapy.time.absolute_time` functions to convert MET floats to `~astropy.time.Time`
objects via the reference times stored in FITS headers.

Time differences
----------------

TODO: discuss when to use `astropy.time.TimeDelta` or `astropy.units.Quantity` or MET floats and
TODO: discuss when to use `astropy.time.TimeDelta` or `astropy.units.Quantity` or [MET]_ floats and
where one needs to convert between those and what to watch out for.

Reference/API
Expand Down
Loading