Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update raw data model based on spectrograph tests #73

Merged
merged 23 commits into from
Apr 4, 2019
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
d7368d1
preparing for updates related to spectrograph tests
weaverba137 Feb 18, 2019
bb6543d
add link to document INHERIT
weaverba137 Feb 18, 2019
811ed41
attempting to implement EXTNAME policy
weaverba137 Feb 18, 2019
6ac7a4e
fix most errors; still need to update fvc file from PROTODESI
weaverba137 Feb 18, 2019
59802aa
fix formatting errors that prevented header format identification
weaverba137 Feb 19, 2019
ab7610f
fix explicit path name in test
weaverba137 Feb 19, 2019
c771da0
add support for range descriptions
weaverba137 Feb 19, 2019
6b285f2
clean up notes section
weaverba137 Feb 20, 2019
19a42ad
update raw data model based on spectrograph tests
weaverba137 Feb 21, 2019
5adc6b3
fix extname specification indexing
weaverba137 Feb 21, 2019
c2abc32
reorder HDUs based on anticipated final data format
weaverba137 Feb 26, 2019
e446942
update raw data model
weaverba137 Apr 3, 2019
27720df
fix table columns
weaverba137 Apr 3, 2019
2cd2336
pin sphinx version
weaverba137 Apr 3, 2019
2a2e2cc
pin sphinx version
weaverba137 Apr 3, 2019
80002b7
handle additional pointer types
weaverba137 Apr 3, 2019
71f55bc
change number of HDUs
weaverba137 Apr 3, 2019
4d1102e
fix extension name
weaverba137 Apr 3, 2019
aab404b
fix test values
weaverba137 Apr 3, 2019
b5e82eb
support 'Q' pointers
weaverba137 Apr 3, 2019
19525a7
suggested updates for documenting all cameras
sbailey Apr 4, 2019
7cbaf55
whitespace matters
sbailey Apr 4, 2019
e13b63a
Merge pull request #75 from desihub/spectrograph-tests-allcams
weaverba137 Apr 4, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
370 changes: 267 additions & 103 deletions doc/DESI_SPECTRO_DATA/NIGHT/EXPID/desi-EXPID.rst

Large diffs are not rendered by default.

36 changes: 12 additions & 24 deletions doc/DESI_SPECTRO_SIM/PIXPROD/NIGHT/EXPID/simpix-CAMERA-EXPID.rst
Original file line number Diff line number Diff line change
@@ -1,30 +1,16 @@
=================
======
simpix
=================

General Description
===================

Summary
-------

simpix files contain the noiseless CCD pixel image.
The corresponding pix-\*.fits files contain the noisy realization of this
image like the real data would see. It optionally contains the x and y
trace locations vs. wavelength.
======

Naming Convention
-----------------
:Summary: simpix files contain the noiseless CCD pixel image.
The corresponding pix-\*.fits files contain the noisy realization of this
image like the real data would see. It optionally contains the x and y
trace locations vs. wavelength.
:Naming Convention: ``simpix-{CAMERA}-{EXPID}.fits``, where ``{CAMERA}`` is the spectrograph
camera, *e.g.* b0, r1, z9; and ``{EXPID}`` is the 8-digit exposure ID.
:Regex: ``simpix-[brz][0-9]-[0-9]{8}\.fits``
:File Type: FITS, 128 MB

``simpix-{CAMERA}-{EXPID}.fits``, where ``{CAMERA}`` is the spectrograph
camera, e.g. b0, r1, z9; and ``{EXPID}`` is the 8-digit exposure ID.

regex: ``simpix-[brz][0-9]{1}-[0-9]{8}.fits``

File Type
---------

FITS, 128 MB
Contents
========

Expand Down Expand Up @@ -58,6 +44,8 @@ NAXIS2 4096 int Number of pixels in x (fiber direction)
VSPECTER 0.0.0 str Specter version used to simulate this image
======== ============= ==== =====================

Data: FITS image [float64]

HDU1
----

Expand Down
29 changes: 9 additions & 20 deletions doc/DESI_TARGET/targets-dr3.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,25 +2,14 @@
targets-\*.fits
===============

General Description
===================

Summary
-------

DESI target selection files contain a single binary table covering the
entire footprint. They contain the variables used by target selection
(*e.g.* fluxes), variables needed by fiber assignment (*e.g.* RA, DEC),
and variables needed for traceability (*e.g.* TARGETFLAG, TARGETID).

Naming Convention
-----------------

TBD, let's try ``targets-{source}-{version}.fits`` where ``source`` is where the
input data came from (*e.g.* 'dr1', 'dr2') and ``version`` is the code version
that wrote this, preferably a git tag of desitargets.

regex: ``targets-dr[0-9]+-v?[0-9]+\.[0-9]+(\.[0-9]+|)\.fits``
:Summary: DESI target selection files contain a single binary table covering the
entire footprint. They contain the variables used by target selection
(*e.g.* fluxes), variables needed by fiber assignment (*e.g.* RA, DEC),
and variables needed for traceability (*e.g.* TARGETFLAG, TARGETID).
:Naming Convention: TBD, let's try ``targets-{source}-{version}.fits`` where ``source`` is where the
input data came from (*e.g.* 'dr1', 'dr2') and ``version`` is the code version
that wrote this, preferably a git tag of desitargets.
:Regex: ``targets-dr[0-9]+-v?[0-9]+\.[0-9]+(\.[0-9]+|)\.fits``

Contents
========
Expand All @@ -41,7 +30,7 @@ HDU0

EXTNAME = PRIMARY

Empty header.
Empty HDU.

HDU1
----
Expand Down
101 changes: 50 additions & 51 deletions doc/PROTODESI/images/fvc/fvc.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,16 @@ fvc
:Summary: Fiber View Camera images for ProtoDESI.
:Naming Convention: ``fvc\.[0-9]{8}(_[0-9]{4}|[0-9]{6})\.fits``
:Regex: ``fvc\.[0-9]{8}(_[0-9]{4}|[0-9]{6})\.fits``

:File Type: FITS, 72 MB

Contents
========

====== ======== ============================== ================================================================
Number EXTNAME Type Contents
====== ======== ============================== ================================================================
HDU0_ PRIMARY NPIXxNPIX float image raw image
====== ======== ============================== ================================================================
====== ======== ===== =========
Number EXTNAME Type Contents
====== ======== ===== =========
HDU0_ PRIMARY IMAGE Raw image
====== ======== ===== =========

FITS Header Units
=================
Expand All @@ -24,53 +24,52 @@ HDU0

EXTNAME = PRIMARY

FITS Image: Raw

Raw image data.

Required Header Keywords
~~~~~~~~~~~~~~~~~~~~~~~~

======== ========= ==== ========================================
Header Value Type Comment
======== ========= ==== ========================================
XTENSION BINTABLE str Binary table written by MWRFITS v1.8
BITPIX 8 int Required value
NAXIS 2 int Required value
NAXIS1 6132 int Number of bytes per row
NAXIS2 8176 int Number of rows
PCOUNT 0 int Normally 0 (no varying arrays)
GCOUNT 1 int Required value
TFIELDS int Number of columns in table
======== ========= ==== ========================================

Required Columns
~~~~~~~~~~~~~~~~
======== ===================== ===== ===========================================
Header Value Type Comment
======== ===================== ===== ===========================================
BITPIX 16 int number of bits per data pixel
NAXIS 2 int number of data axes
NAXIS1 6000 int length of data axis 1
NAXIS2 6000 int length of data axis 2
BZERO 32768 int offset data range to that of unsigned short
BSCALE 1 int default scaling factor
EXPID 200728.000032 float Exposure ID
EXPTIME 1. float Exposure time in seconds
RDTIME 9.25288391113281 float readout time in seconds
DRKEXP 0 int 0/1 for open-shutter/dark exposure
DRKFLAG 0 int 0/1 for unsubtracted/dark-corrected
SIMFLAG 0 int 0/1 for real/fake image
SIMFIB 0 int 0/1 for real/fake fiber spots
CCDTEMP -10. float CCD temperature (C)
BASETMP -10. float Camera base temperature (C)
TEMPSET -10. float CCD temperature set point (C)
COOLPOW 50. float CCD cooler power (%)
PIXSZX 6.00000021222513E-06 float Camera Pixel Size in x (m)
PIXSZY 6.00000021222513E-06 float Camera Pixel Size in y (m)
CCDX1 0 int Physical array start x
CCDX2 8304 int Physical array end x
CCDY1 0 int Physical array start y
CCDY2 6220 int Physical array end y
VISX1 64 int Visible array start x
VISX2 8240 int Visible array end x
VISY1 45 int Visible array start y
VISY2 6177 int Visible array end y
SUBX1 1152 int Sub-array start x
SUBX2 7151 int Sub-array end x
SUBY1 111 int Sub-array start y
SUBY2 6110 int Sub-array end y
HBIN 0 int Horizontal binning
VBIN 0 int Vertical binning
OBSNUM 200728 int Observation Number
OBSFRM 32 int Observation Frame
HDREV 256 int Camera hardware revision
FWREV 513 int Camera firmware revision
DATE '2016-09-28T11:11:12' str file creation date (YYYY-MM-DDThh:mm:ss UT)
======== ===================== ===== ===========================================

================= ======== =======
Column Type Comment
================= ======== =======
TAI-BEG float Timestamp
EXPTIME float Exposure Time
IMG-X int64 Image size -x
IMG-Y int64 Image size -y
TEMP float Temp of camera
TEL-RA float Telescope RA
TEL-DEC float Tel.DEC
AMASS float Airmass
F0-X float Fiber 1 X pixel value
F0-Y float Fiber 1 Y pixel value
MAG0 float Mag of Fiber 0
PSF0 float Size of PSF for Fiber 0
F1-X float Fiber 1 X pixel value
F1-Y float Fiber 1 Y pixel value
MAG1 float Mag of Fiber 1
PSF1 float Size of PSF for Fiber 1
F2-X float Fiber 2 X pixel value
F2-Y float Fiber 2 Y pixel value
MAG2 float Mag of Fiber 2
PSF2 float Size of PSF for Fiber 2
F3-X float Fiber 3 X pixel value
F3-Y float Fiber 3 Y pixel value
MAG3 float Mag of Fiber 3
PSF3 float Size of PSF for Fiber 3
================= ======== =======
Data: int16 FITS image [6000, 6000]
97 changes: 53 additions & 44 deletions doc/PROTODESI/images/gfa/PROTODESI_GFA.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,12 @@ PROTODESI_GFA
Contents
========

====== ======== ============================== ================================================================
Number EXTNAME Type Contents
====== ======== ============================== ================================================================
HDU0_ PRIMARY NPIXxNPIX float image raw image
====== ======== ============================== ================================================================
====== ======== ===== =========
Number EXTNAME Type Contents
====== ======== ===== =========
HDU0_ PRIMARY IMAGE Raw image
HDU1_ GFA IMAGE Raw image
====== ======== ===== =========

FITS Header Units
=================
Expand All @@ -24,48 +25,56 @@ HDU0

EXTNAME = PRIMARY

FITS Image: Postage Stamp of guide star. Raw.

Empty HDU.

Required Header Keywords
~~~~~~~~~~~~~~~~~~~~~~~~

======== ========= ==== ========================================
Header Value Type Comment
======== ========= ==== ========================================
XTENSION BINTABLE str Binary table written by MWRFITS v1.8
BITPIX 8 int Required value
NAXIS 2 int Required value
NAXIS1 2048 int Number of bytes per row
NAXIS2 2064 int Number of rows
PCOUNT 0 int Normally 0 (no varying arrays)
GCOUNT 1 int Required value
TFIELDS int Number of columns in table
======== ========= ==== ========================================
======== ========================================================= ===== ==============================================
Header Value Type Comment
======== ========================================================= ===== ==============================================
DEVICE 'GFA1G ' str
OBSNUM 1622 int
OBSFRAME 7 int
OBSTYPE 'ROI ' str
DATE-OBS '2016-08-31T03:10:47.945228' str
TIME-OBS '03:10:47.945228' str
MJD-OBS 57631.13249937 float
OPENSHUT '2016-08-31T03:10:47.945228' str
LST '18:23:19.990' str
EXPREQ 1. float
EXPTIME 1. float
FILENAME '/data/images/gfa/PROTODESI_GFA1G_roi_00001622_0007.fits' str
ROI_LIST '(190.51099891749917, 124.5090160850238)' str
CHECKSUM 'Y72la70kS70kY70k' str HDU checksum updated 2016-08-31T03:10:47
DATASUM ' 0' str data unit checksum updated 2016-08-31T03:10:47
======== ========================================================= ===== ==============================================

HDU1
----

EXTNAME = GFA

This is the actual GFA image, which got stored in an extension instead
of the primary.

Required Columns
~~~~~~~~~~~~~~~~
======== ================== ===== ==============================================
Header Value Type Comment
======== ================== ===== ==============================================
BITPIX 16 int number of bits per data pixel
NAXIS 2 int number of data axes
NAXIS1 64 int length of data axis 1
NAXIS2 64 int length of data axis 2
PCOUNT 0 int required keyword; must = 0
GCOUNT 1 int required keyword; must = 1
BZERO 32768 int offset data range to that of unsigned short
BSCALE 1 int default scaling factor
ROI_X 190.510998917499 float
ROI_Y 124.509016085024 float
SHAPE '64,64 ' str
DTYPE 'uint16 ' str
CHECKSUM 'EeRhFZQZEbQfEZQZ' str HDU checksum updated 2016-08-31T03:10:47
DATASUM '1260997802' str data unit checksum updated 2016-08-31T03:10:47
======== ================== ===== ==============================================

================= ======== =======
Column Type Comment
================= ======== =======
TAI-BEG float Timestamp
EXPTIME float Exposure Time
BIN int32 binning
IMG-X int64 Image size -x
IMG-Y int64 Image size -y
TEL-RA float Telescope RA
TEL-DEC float Tel.DEC
AMASS float Airmass
TELFOC float Tel.Focus
ZEN float Zenith Distance
HA float Hour Angle
FIELD int64 Field ID
TEMP float Camera Temperature
MODE str Mode of Guider
GS-RA float Guide ref pixel RA
GS-DEC float Guide ref pixel DEC
GS-X float Guide ref pixel X
GS-Y float Guide ref pixel Y
GS-PSF float FWHM of PSF size for guide star
================= ======== =======
Data: int16 FITS image [64, 64]
4 changes: 2 additions & 2 deletions doc/examples/sdR.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,12 @@ Contents
====== ======== =========================== ===========
Number EXTNAME Type Contents
====== ======== =========================== ===========
HDU0_ FLUX NAXIS1 x NAXIS2 float image Flux in ADU
HDU0_ FLUX IMAGE Flux in ADU
====== ======== =========================== ===========

HDU0
====

EXTNAME = FLUX

Data.
Data: FITS image [float64]