Flesh out datamodels for files written by desisurvey

doc/DESISURVEY_OUTPUT/ephem.rst

@@ -0,0 +1,122 @@
+=====
+ephem
+=====
+
+:Summary: Cached ephemeris calculations for desisurvey.
+:Naming Convention: ``ephem_YEAR-MM-DD_YEAR-MM-DD.fits``, where ``YEAR-MM-DD``
+    are the first and last dates covered by this ephemeris cache.
+:Regex: ``ephem_[0-9]{4}-[0-9]{2}-[0-9]{2}_[0-9]{4}-[0-9]{2}-[0-9]{2}\.fits``
+:File Type: FITS, 100 KB
+
+Contents
+========
+
+====== ======= ======== ===============
+Number EXTNAME Type     Contents
+====== ======= ======== ===============
+HDU0_          IMAGE    Blank
+HDU1_  EPHEM   BINTABLE Ephemeris table
+====== ======= ======== ===============
+
+FITS Header Units
+=================
+
+HDU0
+----
+
+EXTNAME = (None)
+
+Empty HDU.
+
+HDU1
+----
+
+EXTNAME = EPHEM
+
+Ephemeris table
+
+Required Header Keywords
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+====== ================== ==== ===================================
+KEY    Example Value      Type Comment
+====== ================== ==== ===================================
+NAXIS1 2920               int  length of dimension 1
+NAXIS2 31                 int  length of dimension 2
+NAME   Survey Ephemerides str
+START  2020-03-15         str  Calculated ephemerides start on the evening of this date.
+STOP   2020-04-15         str  Calculated ephemerides stop on the morning of this date.
+====== ================== ==== ===================================
+
+Required Data Table Columns
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+================= =========== ===== ===========
+Name              Type        Units Description
+================= =========== ===== ===========
+noon              float64           MJD of local noon before night
+dusk              float64           MJD of dark/gray sunset
+dawn              float64           MJD of dark/gray sunrise
+brightdusk        float64           MJD of bright sunset
+brightdawn        float64           MJD of bright sunrise
+brightdusk_LST    float64     deg   Apparent LST at brightdawn
+brightdawn_LST    float64     deg   Apparent LST at brightdusk
+moonrise          float64           MJD of moonrise before/during night
+moonset           float64           MJD of moonset after/during night
+moon_illum_frac   float64           Illuminated fraction of moon surface
+nearest_full_moon float64           Nearest full moon - local midnight in days
+programs          int16[4]          Program sequence between dusk and dawn (see notes)
+changes           float64[3]        MJD of program changes between dusk and dawn
+moon_ra           float64[25] deg   RA of moon tabulated during the night (see notes)
+moon_dec          float64[25] deg   DEC of moon tabulated during the night (see notes)
+venus_ra          float64[25] deg   RA of venus
+venus_dec         float64[25] deg   DEC of venus
+mars_ra           float64[25] deg   RA of mars
+mars_dec          float64[25] deg   DEC of mars
+jupiter_ra        float64[25] deg   RA of jupiter
+jupiter_dec       float64[25] deg   DEC of jupiter
+saturn_ra         float64[25] deg   RA of saturn
+saturn_dec        float64[25] deg   DEC of saturn
+neptune_ra        float64[25] deg   RA of neptune
+neptune_dec       float64[25] deg   DEC of neptune
+uranus_ra         float64[25] deg   RA of uranus
+uranus_dec        float64[25] deg   DEC of uranus
+================= =========== ===== ===========
+
+Notes and Examples
+==================
+
+The default date range is chosen large enough to cover commissioning,
+survey validation and the 5-year main survey, so should not normally need
+to be changed.
+
+The ``desisurvey`` package includes a `convenience wrapper <https://desisurvey.readthedocs.io/en/latest/api.html#desisurvey.ephem.get_ephem>`__
+that should normally be used to access this file::
+
+    import desisurvey.ephem
+    ephem = desisurvey.ephem.get_ephem()
+
+This wrapper implements memory and disk caching, for efficiency, and returns
+an `Ephemerides <https://desisurvey.readthedocs.io/en/latest/api.html#desisurvey.ephem.Ephemerides>`__
+with useful methods for working with the table data. If you need direct access to the
+table use, for example::
+
+    print(ephem.table[:10])
+
+The integer codes used for the nightly program sequence are DARK=0, GRAY=1, BRIGHT=2.
+Use program names where possible, instead of integer codes: the mapping between them
+is defined in the ``desisurvey.tiles`` module::
+
+    import desisurvey.tiles
+    assert deisurvey.tiles.Tiles.PROGRAM_INDEX['GRAY'] == 1
+
+RA, DEC coordinates for the moon and planets are tabulated on an hourly
+grid spanning local noon - noon (inclusive, so 25 grid points) spanning each night.
+You will normally want to interpolate these values using the `get_object_interpolator <https://desisurvey.readthedocs.io/en/latest/api.html#desisurvey.ephem.get_object_interpolator>`__
+convenience method::
+
+    night_ephem = ephem.get_night('2022-12-25')
+    moon_altaz = desisurvey.ephem.get_object_interpolator(night_ephem, 'moon', altaz=True)
+    alt, az = moon_altaz(mjd)
+
+Note that you can either interpolate in (RA,DEC) or (Alt,Az).

doc/DESISURVEY_OUTPUT/exposures.rst

@@ -0,0 +1,128 @@
+===================
+exposures_surveysim
+===================
+
+:Summary: List of exposures from surveysim.
+:Naming Convention: ``exposures_surveysim.fits``
+:Regex: ``exposures_surveysim\.fits``
+:File Type: FITS, 15 KB  *This section gives the type of the file
+    and its approximate size.*
+
+*Note*: currently this is only an output from surveysim, but it may become an
+output of survey operations, caching in a file the information that is also
+contained in the operations database.
+
+Contents
+========
+
+====== ========= ======== ===================
+Number EXTNAME   Type     Contents
+====== ========= ======== ===================
+HDU0_            IMAGE    Blank
+HDU1_  EXPOSURES BINTABLE Exposure metadata
+HDU2_  TILEDATA  BINTABLE Tile metadata
+====== ========= ======== ===================
+
+
+FITS Header Units
+=================
+
+HDU0
+----
+
+EXTNAME = (None)
+
+*Summarize the contents of this HDU.*
+
+Required Header Keywords
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+======= ================= ==== =======
+KEY     Example Value     Type Comment
+======= ================= ==== =======
+TILES   ./test-tiles.fits str
+NEXP    21                int
+INITIAL 2020-03-15        str
+======= ================= ==== =======
+
+Empty HDU.
+
+HDU1
+----
+
+EXTNAME = EXPOSURES
+
+Exposure metadata.
+
+Required Header Keywords
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+======= ============= ==== =====================
+KEY     Example Value Type Comment
+======= ============= ==== =====================
+NAXIS1  36            int  length of dimension 1
+NAXIS2  21            int  length of dimension 2
+EXTNAME EXPOSURES     str  extension name
+======= ============= ==== =====================
+
+Required Data Table Columns
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+======== ======= ===== ===========
+Name     Type    Units Description
+======== ======= ===== ===========
+MJD      float64
+EXPTIME  float32
+TILEID   int32
+SNR2FRAC float32
+AIRMASS  float32
+SEEING   float32
+TRANSP   float32
+SKY      float32
+======== ======= ===== ===========
+
+HDU2
+----
+
+EXTNAME = TILEDATA
+
+Tile metadata.
+
+Required Header Keywords
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+======= ============= ==== =====================
+KEY     Example Value Type Comment
+======= ============= ==== =====================
+NAXIS1  20            int  length of dimension 1
+NAXIS2  10            int  length of dimension 2
+EXTNAME TILEDATA      str  extension name
+======= ============= ==== =====================
+
+Required Data Table Columns
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+======== ======= ===== ===========
+Name     Type    Units Description
+======== ======= ===== ===========
+AVAIL    int32
+PLANNED  int32
+EXPTIME  float32
+SNR2FRAC float32
+NEXP     int32
+======== ======= ===== ===========
+
+*TODO*: please make it very clear whether AVAIL=available for fiber assignment
+vs. available for observations or something else;
+ditto for PLANNED=date on which fiber assignment was run after which it becomes
+available for observing?  Or something else?  The two concepts I'm looking for:
+
+  * the date on which all required overlaps are complete and the tile becomes
+    eligible for fiber assignment (AVAIL? or is that tracked in the planner?)
+  * the date on which fiber assignment was actually run and the tile becomes
+    eligible for observations (PLANNED?)
+
+Notes and Examples
+==================
+
+*Add notes and examples here.  You can also create links to example files.*

doc/DESISURVEY_OUTPUT/index.rst

@@ -0,0 +1,19 @@
+===========
+DESI_TARGET
+===========
+
+:envvar:`DESISURVEY_OUTPUT` contains the outputs of desisurvey and surveysim,
+with the canonical location of ``$DESI_ROOT/survey``.  We expect this
+directory structure will be updated as part of integrating desisurvey
+(afternoon planning and next tile selector) into online operations.
+
+.. toctree::
+   :maxdepth: 1
+
+   ephem_{start}-{stop}.fits : cached ephemeris for survey planning <ephem>
+   surveyinit.fits : Design hour angles? <surveyinit>
+   planner_YEAR-MM-DD.fits : Afternoon planner state <planner>
+   scheduler_YEAR-MM-DD.fits : Next tile selector state <scheduler>
+   stats_surveysim.fits : per-tile and per-night statistics from surveysim run <stats>
+   exposures_surveysim.fits : exposure metadata from surveysim run <exposures>
+

doc/DESISURVEY_OUTPUT/planner.rst

@@ -0,0 +1,89 @@
+=======
+planner
+=======
+
+:Summary: Cached state of the survey afternoon planner.
+:Naming Convention: ``planner_YEAR-MM-DD.fits``, where ``YEAR-MM-DD`` is the
+    date of the afternoon when the planner was run.
+:Regex: ``planner_[0-9]{4}-[0-9]{2}-[0-9]{2}\.fits``
+:File Type: FITS, 400 KB
+
+Contents
+========
+
+====== ======= ======== =======================
+Number EXTNAME Type     Contents
+====== ======= ======== =======================
+HDU0_          IMAGE    Blank.
+HDU1_  PLAN    BINTABLE Survey planner state.
+====== ======= ======== =======================
+
+FITS Header Units
+=================
+
+HDU0
+----
+
+EXTNAME = (None)
+
+Empty HDU.
+
+HDU1
+----
+
+EXTNAME = PLAN
+
+Snapshot of the internal state of a `desisurvey.plan.Planner
+<https://desisurvey.readthedocs.io/en/latest/api.html?highlight=Planner#desisurvey.plan.Planner>`__
+object.
+
+Required Header Keywords
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+======= ============= ==== =====================
+KEY     Example Value Type Comment
+======= ============= ==== =====================
+NAXIS1  29            int  length of dimension 1.
+NAXIS2  16071         int  length of dimension 2.
+CADENCE monthly       str  Fiber assignment cadence (monthly / daily).
+FIRST   2019-12-01    str  First night that was planned.
+LAST    2024-11-29    str  Last night that has been planned so far.
+======= ============= ==== =====================
+
+Required Data Table Columns
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+========= ======= ===== ===========
+Name      Type    Units Description
+========= ======= ===== ===========
+TILEID    int64         Tile ID from the tiles file.
+COVERED   int64         Day number that tile was first covered relative to ``FIRST`` or -1.
+COUNTDOWN int64         Countdown of remaining fiber assignment cycles until this tile will be assigned.
+AVAILABLE logical       Does this tile have fibers already assigned?
+PRIORITY  float64       Relative priority for scheduling this tile. Must be >= 0.
+========= ======= ===== ===========
+
+The meaning of "covered" is specified by the ``fiber_assignment_order`` configuration parameter,
+which also specifies the starting ``COUNTDOWN`` value for each pass.
+
+Each row of the table corresponds to one tile with indexing that matches ``desisurvey.tiles.Tiles``.
+
+Notes and Examples
+==================
+
+A `Planner object
+<https://desisurvey.readthedocs.io/en/latest/api.html?highlight=Planner#desisurvey.plan.Planner>`__
+manages updates during afternoon planning::
+
+    import desisurvey.rules
+    import desisurvey.plan
+    rules = desisurvey.rules.Rules()
+    planner = desisurvey.plan.Planner(rules)
+
+Its internal state after each afternoon can be saved using, for example::
+
+    planner.save('planner_snapshot.fits')
+
+This state can then be later restored using::
+
+    planner = desisurvey.plan.Planner(rules, restore='planner_snapshot.fits')