Skip to content

Commit

Permalink
Merge pull request #4 from ivoa-std/rfe_order
Browse files Browse the repository at this point in the history 
RFE: order and limits
  • Loading branch information
@lmichel
lmichel committed Jul 21, 2022
2 parents d3453e7 + 71af8d4 commit d0a65d4
Show file tree
Hide file tree
Showing 7 changed files with 203 additions and 129 deletions.
4 changes: 2 additions & 2 deletions doc/Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ DOCNAME = Spectrum
DOCVERSION = 1.2

# Publication date, ISO format; update manually for "releases"
DOCDATE = 2022-07-10
SDATE = 20220710
DOCDATE = 2022-07-20
SDATE = 20220720

# What is it you're writing: NOTE, WD, PR, REC, PEN, or EN
DOCTYPE = WD
Expand Down
54 changes: 40 additions & 14 deletions doc/Spectrum.tex
View file
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@
\par\noindent \htmtitle{IVOA Spectrum Data Model}{Version 1.2}
\Large
\vskip 0.1in
\par\noindent{\bf IVOA Working Draft 2022-06-22}
\par\noindent{\bf IVOA Working Draft 2022-07-20}
\color{black}
\vskip 0.2in

Expand All @@ -74,11 +74,11 @@

Data Model Working Group

\par\noindent{\bf This version (Mod 0)}
\par\noindent{\bf This version (Mod 2)}

{
\color{dblue}
WD-SpectrumDM-1.2-20220622
WD-SpectrumDM-1.2-20220719
%\link{http://www.ivoa.net/Documents/latest/SpectrumDM.html}
}

Expand Down Expand Up @@ -173,6 +173,8 @@ \section{Introduction and Motivation}
\subsection{Change Log}

\begin{verbatim}
2022 Jul 20 V1.2 Mod 2 - Add upper/lower limit elements
2022 Jul 19 V1.2 Mod 1 - Add spectral order elements
2022 Jun 22 V1.2 Mod 0 - Document reset to WD for enhancement
\end{verbatim}

Expand Down Expand Up @@ -211,6 +213,8 @@ \section{Requirements}
detail to understand the differences between two spectra of the same
object and between two spectra of different objects.

We need to be able to identify and distinguish data points by spectral order.

We need to represent time series photometry, with many photometry points
of the same object at different times.

Expand All @@ -221,8 +225,6 @@ \section{Requirements}
object. The 'SED' model will be described in
a separate document which builds on the structures described here.

TODO: Add new requirements leading to order and limits attributes.


\section{Spectral Data Model summary}

Expand Down Expand Up @@ -340,7 +342,7 @@ \subsection{Use Cases and Required Fields}
% Data diagram:
\begin{figure}[h]
\begin{center}
\includegraphics[width=4.0in]{diagrams/data.png}
\includegraphics[width=5.0in]{diagrams/data.png}
\caption{Diagram for Data object}
\end{center}
\end{figure}
Expand Down Expand Up @@ -565,6 +567,23 @@ \subsection{Spectral coordinate}
Note 3: `em.wl;obs.atmos' (AWAV) is provided for air wavelengths. The basic spectral
choices em.wl, em.freq, em.energy are understood to be vacuum values.

\subsubsection{Spectral order}

We define two elements for expressing the spectral order for each point.
These elements only have meaning for Spectrum.Data (the per-data-point values).

\begin{description}
\item[order:] The order number as defined by the instrument configuration.
It represents the ‘order of diffraction’, given by the number of wavelengths
of monochromatic light where the diffraction occurs with given geometric
(eg: angles of rays incidence, reflection) and physical (eg: spacing of
the grating grooves) conditions.

\item[relorder:] Relative order is a processing dependent order, typically
counting from 1 and increasing as you go either towards red or blue.

\end{description}


\begin{flushleft}
\colorbox{iblue}{\small
Expand Down Expand Up @@ -822,17 +841,11 @@ \subsubsection{Uncertainties}
We also use a very simple error model for the fluxes: we include plus and minus flux
errors, and a quality flag. The errors are understood as
1 sigma gaussian errors which are uncorrelated for different points
in the spectrum. If the data provider has only upper limit
information, it should be represented by setting the flux value and
the lower error value equal to the limit, and the upper error value
equal to zero (e.g. 5 (+0,-5)). In general applications
may choose to render measurements as upper limits if the flux value
is less than some multiple (e.g. 3) of the lower error.
in the spectrum.
We also allow a systematic error value, assumed constant across
a given spectrum and fully correlated (so that, e.g. it does not enter
into estimating spectral slopes).


{\bf CLARIFICATION: } the two-sided errors StatErrLow and StatErrHigh are the plus/minus ERRORS,
not the (value+error, value-error). In other words, if Value = 10 and there
is a symmetric uncertainty of 3, the ErrorLow and ErrorHigh are both +3.0,
Expand All @@ -848,7 +861,20 @@ \subsubsection{Uncertainties}
indicates that the errors are unknown. Data providers are STRONGLY
encouraged to provide explicit error measures whenever possible.


\subsubsection{Statistical Limits}

The UpperLimit and LowerLimit fields allow data providers to explicitly
represent statistical limits on the flux values. In many cases, it can
be useful to provide both the measured flux, with associated errors, AND
statistical limits. For this reason, these limits may be supplied
independently of the flux values themselves.

{\bf NOTE: } The previous version of this model suggests representing upper
limits by setting the flux value and the lower error (StatErrLow) value
equal to the limit, and the upper error (StatErrHigh) value equal to
zero (e.g. 5 (+0,-5)). While this remains a valid representation, we
recommend data providers use the more explicit model elements whenever possible.


\subsubsection{Resolution}

Expand Down
Binary file modified doc/diagrams/data.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
58 changes: 31 additions & 27 deletions doc/fields_table.tex
View file
Original file line number Diff line number Diff line change
Expand Up @@ -209,19 +209,21 @@
\hline
\multicolumn{4}{c}{Data Fields}\\
\hline
{\bf Data.FluxAxis.Value } &TTYPEn = 'FLUX' & (see sec 4.1, table 3) & Flux values for points& MAN\\
Data.FluxAxis.ucd &TUCDn & meta.ucd & ucd for flux & OPT & Char.FluxAxis.ucd\\
Data.FluxAxis.unit &TUNITn & meta.unit & Unit for flux& OPT & Char.FluxAxis.unit\\
{\bf Data.SpectralAxis.Value } &TTYPEn ='WAVE' & (see sec 4.2, table 2) & Spectral coordinates for points& MAN & (Char.SpectralAxis.Location)\\
& or 'ENER' or 'FREQ' &&\\
{Data.SpectralAxis.ucd } &TUCDn & meta.ucd & ucd for spectral coord& OPT & Char.SpectralAxis.ucd\\
{Data.SpectralAxis.unit } &TUNITn & meta.unit & Unit for spectral coord& OPT & Char.SpectralAxis.unit\\
{Data.TimeAxis.Value } & TTYPEn ='TIME' & &Time coordinates for points & OPT & Char.TimeAxis.Location\\
Data.TimeAxis.ucd &TUCDn & meta.ucd & ucd for time& OPT & Char.TimeAxis.ucd\\
Data.TimeAxis.unit &TUNITn & meta.unit & Unit for time& OPT & Char.TimeAxis.Unit\\
{Data.BackgroundModel.Value } &TTYPEn='BGFLUX'& &Flux values for points& OPT & No background model\\
{Data.BackgroundModel.ucd } &TUCDn & meta.ucd & ucd for background flux & OPT & Points.FluxAxis.ucd\\
{Data.BackgroundModel.unit } &TUNITn & meta.unit & Unit for background flux& OPT & Points.FluxAxis.unit\\
{\bf Data.FluxAxis.Value } &TTYPEn = 'FLUX' & (see sec 4.1, table 3) & Flux values for points & MAN\\
Data.FluxAxis.ucd &TUCDn & meta.ucd & ucd for flux & OPT & Char.FluxAxis.ucd\\
Data.FluxAxis.unit &TUNITn & meta.unit & Unit for flux & OPT & Char.FluxAxis.unit\\
{\bf Data.SpectralAxis.Value }&TTYPEn ='WAVE' & (see sec 4.2, table 2) & Spectral coordinates for points& MAN & (Char.SpectralAxis.Location)\\
& or 'ENER' or 'FREQ'& &\\
Data.SpectralAxis.ucd &TUCDn & meta.ucd & ucd for spectral coord & OPT & Char.SpectralAxis.ucd\\
Data.SpectralAxis.unit &TUNITn & meta.unit & Unit for spectral coord & OPT & Char.SpectralAxis.unit\\
Data.SpectralAxis.order &TTYPEn = 'ORDER' & instr.order & Absolute spectral order & OPT & (None)\\
Data.SpectralAxis.relorder &TTYPEn = 'RELORDER' & instr.order & Relative spectral order & OPT & (None)\\
Data.TimeAxis.Value & TTYPEn ='TIME' & & Time coordinates for points & OPT & Char.TimeAxis.Location\\
Data.TimeAxis.ucd &TUCDn & meta.ucd & ucd for time & OPT & Char.TimeAxis.ucd\\
Data.TimeAxis.unit &TUNITn & meta.unit & Unit for time & OPT & Char.TimeAxis.Unit\\
Data.BackgroundModel.Value &TTYPEn='BGFLUX' & & Flux values for points & OPT & No background model\\
Data.BackgroundModel.ucd &TUCDn & meta.ucd & ucd for background flux & OPT & Points.FluxAxis.ucd\\
Data.BackgroundModel.unit &TUNITn & meta.unit & Unit for background flux & OPT & Points.FluxAxis.unit\\
\end{tabular}

\end{minipage}
Expand All @@ -238,20 +240,22 @@
\hline
\multicolumn{4}{c}{Accuracy Fields - per data point (default to corresponding Spectrum.Char values)} \\
\hline
Data.FluxAxis.Accuracy.StatError &TTYPEn = 'ERR' & stat.error;phot.flux.density; em.* & symmetric error & OPT & (Char)\\
Data.FluxAxis.Accuracy.StatErrLow &TTYPEn='ERR\_LO' & stat.error;phot.flux.density; em.*;stat.min & Lower error & OPT & (Char)\\
Data.FluxAxis.Accuracy.StatErrHigh&TTYPEn='ERR\_HI' & stat.error;phot.flux.density; em.*;stat.max & Upper error & OPT & (Char)\\
Data.FluxAxis.Accuracy.SysError&TTYPEn='SYS\_ERR' & stat.error.sys;phot.flux.density; em.{\it ..} & Systematic error & OPT & (Char)\\
Data.FluxAxis.Quality &TTYPEn='QUALITY' & meta.code.qual; phot.flux.density, em.* & Quality mask& OPT & 0\\
Data.FluxAxis.Quality.n &TTYPEn='QUALn' & & String value, for n = 0,1,2..; meaning of quality value &OPT & None \\
Data.SpectralAxis.Accuracy.BinSize &TTYPEn='WAVE\_BIN',etc & em.*;spect.binSize & Wavelength bin size & OPT & (Char)\\
Data.SpectralAxis.Accuracy.BinLow & TTYPEn ='WAVE\_LO',etc & em.*;stat.min & Spectral coord bin lower end & OPT & \\
Data.SpectralAxis.Accuracy.BinHigh & TTYPEn ='WAVE\_HI',etc & em.*;stat.max &Spectral coord bin upper end & OPT & \\
Data.SpectralAxis.Accuracy.StatError&TTYPEn ='WAVE\_ERR',etc & stat.error;em.* & Spectral coord measurement error & OPT & (Char)\\
Data.SpectralAxis.Accuracy.StatErrLow&TTYPEn='WAVE\_ELO',etc & stat.error; em.*;stat.min & Spectral coord measurement lower error & OPT & (Char)\\
Data.SpectralAxis.Accuracy.StatErrHigh&TTYPEn='WAVE\_EHI',etc & stat.error; em.*;stat.max & Spectral coord measurement upper error & OPT & (Char)\\
Data.SpectralAxis.Accuracy.SysError &TTYPEn='WAVE\_SYE',etc & stat.error.sys;em.* & Spectral coord systematic error & OPT & (Char)\\
Data.SpectralAxis.Resolution & TTYPEn ='WAVE\_RES',etc & spect.resolution; em.* & Spectral resolution FWHM & OPT & (Char)\\
Data.FluxAxis.Accuracy.LowerLimit &TTYPEn='LOWER\_LIMIT' & stat.error;phot.flux.density; em.*;stat.min & Lower Limit & OPT & (None)\\
Data.FluxAxis.Accuracy.UpperLimit &TTYPEn='UPPER\_LIMIT' & stat.error;phot.flux.density; em.*;stat.max & Upper Limit & OPT & (None)\\
Data.FluxAxis.Accuracy.StatError &TTYPEn='ERR' & stat.error;phot.flux.density; em.* & symmetric error & OPT & (Char)\\
Data.FluxAxis.Accuracy.StatErrLow &TTYPEn='ERR\_LO' & stat.error;phot.flux.density; em.*;stat.min & Lower error & OPT & (Char)\\
Data.FluxAxis.Accuracy.StatErrHigh &TTYPEn='ERR\_HI' & stat.error;phot.flux.density; em.*;stat.max & Upper error & OPT & (Char)\\
Data.FluxAxis.Accuracy.SysError &TTYPEn='SYS\_ERR' & stat.error.sys;phot.flux.density; em.{\it ..} & Systematic error & OPT & (Char)\\
Data.FluxAxis.Quality &TTYPEn='QUALITY' & meta.code.qual; phot.flux.density, em.* & Quality mask & OPT & 0\\
Data.FluxAxis.Quality.n &TTYPEn='QUALn' & & String value, for n = 0,1,2..; meaning of quality value &OPT & (None)\\
Data.SpectralAxis.Accuracy.BinSize &TTYPEn='WAVE\_BIN',etc & em.*;spect.binSize & Wavelength bin size & OPT & (Char)\\
Data.SpectralAxis.Accuracy.BinLow &TTYPEn='WAVE\_LO',etc & em.*;stat.min & Spectral coord bin lower end & OPT & \\
Data.SpectralAxis.Accuracy.BinHigh &TTYPEn='WAVE\_HI',etc & em.*;stat.max & Spectral coord bin upper end & OPT & \\
Data.SpectralAxis.Accuracy.StatError &TTYPEn='WAVE\_ERR',etc & stat.error;em.* & Spectral coord measurement error & OPT & (Char)\\
Data.SpectralAxis.Accuracy.StatErrLow &TTYPEn='WAVE\_ELO',etc & stat.error; em.*;stat.min & Spectral coord measurement lower error & OPT & (Char)\\
Data.SpectralAxis.Accuracy.StatErrHigh &TTYPEn='WAVE\_EHI',etc & stat.error; em.*;stat.max & Spectral coord measurement upper error & OPT & (Char)\\
Data.SpectralAxis.Accuracy.SysError &TTYPEn='WAVE\_SYE',etc & stat.error.sys;em.* & Spectral coord systematic error & OPT & (Char)\\
Data.SpectralAxis.Resolution &TTYPEn='WAVE\_RES',etc & spect.resolution; em.* & Spectral resolution FWHM & OPT & (Char)\\
\end{tabular}

\end{minipage}
Expand Down
60 changes: 39 additions & 21 deletions doc/specfits.tex
View file
Original file line number Diff line number Diff line change
Expand Up @@ -350,28 +350,44 @@ \subsection{Mapping Spectrum to FITS}
\multicolumn{4}{c}{ Per-data-point values }\\
\hline
\\
Data.FluxAxis.Value & TTYPEn & S & FLUX\\
UTYPE of above ... & TUTYPn & N & 'Spectrum.Data.FluxAxis.Value'\\\
Data.FluxAxis.Unit & TUNITn& S\\
Data.FluxAxis.ucd & TUCDn & N& (same as Char)\\
Data.FluxAxis.Accuracy.StatError & TTYPEn & N & ERR\\
Data.FluxAxis.Value & TTYPEn & S & FLUX\\
UTYPE of above ... & TUTYPn & N & 'Spectrum.Data.FluxAxis.Value'\\
Data.FluxAxis.Unit & TUNITn & S\\
Data.FluxAxis.ucd & TUCDn & N & (same as Char)\\
Data.FluxAxis.Accuracy.LowerLimit & TTYPEn & N & LOWER\_LIMIT\\
Data.FluxAxis.Accuracy.UpperLimit & TTYPEn & N & UPPER\_LIMIT\\
Data.FluxAxis.Accuracy.StatError & TTYPEn & N & ERR\\
Data.FluxAxis.Accuracy.StatErrLow & TTYPEn & C & ERR\_LO\\
Data.FluxAxis.Accuracy.StatErrHigh& TTYPEn& C & ERR\_HI\\
Data.FluxAxis.Accuracy.SysError & TTYPEn & C & SYS\_ERR \\
Data.FluxAxis.Quality & TTYPEn & C & QUALITY\\
Data.FluxAxis.QualityN & TTYPEn & C & QUALn\\
Data.SpectralAxis.Value & TTYPEn & S & WAVE,ENER,FREQ\\
UTYPE of above ... & TUTYPn & N & 'Spectrum.Data.SpectralAxis.Value'\\
Data.SpectralAxis.Unit & TUNITn & S & (same as Char)\\
Data.SpectralAxis.ucd & TUCDn &N & (same as Char) \\
Data.SpectralAxis.Accuracy.BinSize& TTYPEn & N & WAVE\_BIN,ENER\_BIN, FREQ\_BIN\\
Data.SpectralAxis.Accuracy.BinLow & TTYPEn & N & WAVE\_LO,ENER\_LO, FREQ\_LO \\
Data.SpectralAxis.Accuracy.BinHigh& TTYPEn & N & WAVE\_HI,ENER\_HI, FREQ\_HI \\
Data.SpectralAxis.Accuracy.StatError & TTYPEn& N & WAVE\_ERR,ENER\_ERR, FREQ\_ERR \\
Data.SpectralAxis.Accuracy.StatErrLow& TTYPEn& N & WAVE\_ELO,ENER\_ELO, FREQ\_ELO \\
Data.SpectralAxis.Accuracy.StatErrHigh & TTYPEn& N& WAVE\_EHI,ENER\_EHI, FREQ\_EHI \\
Data.SpectralAxis.Accuracy.SysError & TTYPEn & N & WAVE\_SYE,ENER\_SYE, FREQ\_SYE \\
Data.SpectralAxis.Resolution & TTYPEn & N & WAVE\_RES,ENER\_RES, FREQ\_RES \\
Data.FluxAxis.Accuracy.StatErrHigh& TTYPEn & C & ERR\_HI\\
Data.FluxAxis.Accuracy.SysError & TTYPEn & C & SYS\_ERR \\
Data.FluxAxis.Quality & TTYPEn & C & QUALITY\\
Data.FluxAxis.QualityN & TTYPEn & C & QUALn\\
& & & \\
Data.SpectralAxis.Value & TTYPEn & S & WAVE,ENER,FREQ\\
UTYPE of above ... & TUTYPn & N & 'Spectrum.Data.SpectralAxis.Value'\\
Data.SpectralAxis.Unit & TUNITn & S & (same as Char)\\
Data.SpectralAxis.ucd & TUCDn & N & (same as Char) \\
Data.SpectralAxis.order & TTYPEn & N & ORDER\\
Data.SpectralAxis.relorder & TTYPEn & N & RELORDER\\
Data.SpectralAxis.Accuracy.BinSize & TTYPEn & N & WAVE\_BIN,ENER\_BIN, FREQ\_BIN\\
Data.SpectralAxis.Accuracy.BinLow & TTYPEn & N & WAVE\_LO,ENER\_LO, FREQ\_LO \\
Data.SpectralAxis.Accuracy.BinHigh & TTYPEn & N & WAVE\_HI,ENER\_HI, FREQ\_HI \\
Data.SpectralAxis.Accuracy.StatError & TTYPEn & N & WAVE\_ERR,ENER\_ERR, FREQ\_ERR \\
Data.SpectralAxis.Accuracy.StatErrLow & TTYPEn & N & WAVE\_ELO,ENER\_ELO, FREQ\_ELO \\
Data.SpectralAxis.Accuracy.StatErrHigh & TTYPEn & N & WAVE\_EHI,ENER\_EHI, FREQ\_EHI \\
Data.SpectralAxis.Accuracy.SysError & TTYPEn & N & WAVE\_SYE,ENER\_SYE, FREQ\_SYE \\
Data.SpectralAxis.Resolution & TTYPEn & N & WAVE\_RES,ENER\_RES, FREQ\_RES \\
\\
\end{tabular}
\end{minipage}
}

\colorbox{iblue}{
\begin{minipage}[l]{7.0in}
\begin{tabular}{lllp{1.5in}}
Data model field & FITS keyword& Source & Value if fixed \\
\hline
\\
Data.TimeAxis.Value & TTYPEn & C & TIME\\
UTYPE of above ... & TUTYPn & N & 'Spectrum.Data.TimeAxis.Value' \\
Data.TimeAxis.Unit & TUNITn& S & (same as Char)\\
Expand All @@ -384,6 +400,7 @@ \subsection{Mapping Spectrum to FITS}
Data.TimeAxis.Accuracy.StatErrLow & TTYPEn & N & TIME\_ELO \\
Data.TimeAxis.Accuracy.StatErrHigh& TTYPEn & N & TIME\_EHI \\
Data.TimeAxis.Accuracy.SysError & TTYPEn & N & TIME\_SYE \\
& & & \\
Data.BackgroundModel.Value & TTYPEn & N & BGFLUX \\
UTYPE of above ... & TUTYPn & N & 'Spectrum.Data.BackgroundModel.Value'\\
Data.BackgroundModel.Unit & TUNITn &S& (same as FluxAxis) \\
Expand All @@ -397,6 +414,7 @@ \subsection{Mapping Spectrum to FITS}
\end{tabular}
\end{minipage}
}

}
\vskip 0.2in
\clearpage
Expand Down
Loading

0 comments on commit d0a65d4

Please sign in to comment.