test sections

lsst · Jun 26, 2017 · 036a385 · 036a385
1 parent 8db85bd
commit 036a385
Show file tree

Hide file tree

Showing 4 changed files with 74 additions and 18 deletions.
diff --git a/body.tex b/body.tex
@@ -16,7 +16,8 @@
 \input{constraints}
 
 \input{schedule}
-
+\section{Verification Tests \label{sect:vts}}
+\input{testsections}
 % ------------------------------- VALIDATION TOOLS -------------------
 
 \input{tools}

diff --git a/dmtestmilstones.tex b/dmtestmilstones.tex
@@ -1,69 +1,65 @@
 LDM-503-2 & 2017-11-30 &
- NCSA & \textbf{Test report: HSC reprocessing} 
+ NCSA & \textbf{ HSC reprocessing} 
 • Validate the data products with the LSST stack match or improve upon HSC products.
-• Validate the ops platform in NCSA, including installing the stack, starting and stopping production.
-• Generate a validation data set for weekly integration and other tests.
  \\ \hline
 LDM-503-3 & 2017-11-30 &
- NCSA & \textbf{Test report: Alert generation validation} 
+ NCSA & \textbf{ Alert generation validation} 
 • Validate the alert generation stack performance on several DECam and HSC datasets.
  \\ \hline
 LDM-503-4 & 2018-02-28 &
- NCSA & \textbf{Test report: Camera DAQ integration funtionalty test} 
+ NCSA & \textbf{ Camera DAQ integration funtionalty test} 
 • The data acquisition hardware should be available to DM on 2018-02-30. We should prepare by testing that adjacent archive systems.
  \\ \hline
 LDM-503-4b & 2018-03-31 &
  NCSA & \textbf{Test Report: Camera DAQ interface Integration Verification } 
 • The data acquisition hardware should be available to DM on 2018-02-30. We should test integration with the adjacent archive systems.
  \\ \hline
 LDM-503-5 & 2018-05-31 &
- NCSA & \textbf{Test report: Alert distribution validation} 
+ NCSA & \textbf{ Alert distribution validation} 
 • Validate alert distribution system and mini-broker fed by live or simulated live data.
  \\ \hline
 LDM-503-6 & 2018-06-30 &
- NCSA & \textbf{Test report: DM ComCam interface verification readiness} 
+ NCSA & \textbf{ DM ComCam interface verification readiness} 
 • ComCam will be in Tucson on 2018-07-24. The DM system must be ready to deal with it.
  \\ \hline
 LDM-503-7 & 2018-08-31 &
- NCSA & \textbf{Test report: Camera data processing} 
+ NCSA & \textbf{ Camera data processing} 
 • Partial camera data should be available to DM July 31st. We plan to test DM stack with it.
  \\ \hline
 LDM-503-8 & 2018-11-30 &
- NCSA & \textbf{Test report: Spectrograph data acquisition} 
+ NCSA & \textbf{ Spectrograph data acquisition} 
 • Demonstrate that we can acquire (and process?) data from the spectrograph.
  \\ \hline
 LDM-503-9 & 2018-11-30 &
  NCSA & \textbf{Verification tests in advance of pre-ops rehearsal for commissioning \#1} 
 • Test how the system will run during commissioning.
-• Chuck requests that this initial test focus on ISR.
  \\ \hline
 LDM-503-10 & 2019-02-28 &
- NCSA & \textbf{Test report: DAQ validation} 
+ NCSA & \textbf{ DAQ validation} 
 • There is a project Milestone that DAQ/DM/Networks are available March 15th. We need to run tests in Feb to show this is ready.
  \\ \hline
 LDM-503-11a & 2019-10-20 &
- NCSA & \textbf{Test report: DM ComCam operations readiness } 
+ NCSA & \textbf{ DM ComCam operations readiness } 
 • ComCam will be in use in Nov . The DM system must be ready to deal with it.
  \\ \hline
 LDM-503-11 & 2019-10-31 &
  NCSA & \textbf{Verification tests in advance of pre-ops rehearsal for commissioning \#2} 
 • More complete commissioning rehearsal: how do the scientists look at data? How do they provide feedback to the telescope? How do we create/update calibrations?
-• Exercise control loops.
  \\ \hline
 LDM-503-12 & 2020-01-31 &
  NCSA & \textbf{Verification tests in advance of pre-ops rehearsal for commissioning \#3} 
 • Dress rehearsal: commissioning starts in April, so by this stage we should be ready to do everything needed.
  \\ \hline
 LDM-503-13 & 2020-11-30 &
- NCSA & \textbf{Test report: Ops rehearsal DRP (ComCam data)} 
+ NCSA & \textbf{ Ops rehearsal DRP (ComCam data)} 
 • ComCam data will now be available. Demonstrate its use in producing a data release.
  \\ \hline
 LDM-503-14 & 2021-03-31 &
- NCSA & \textbf{Test report: DM Software for Science Verification} 
+ NCSA & \textbf{ DM Software for Science Verification} 
 • Science Verification starts in April. Demonstrate that all required DM software is available.
  \\ \hline
 LDM-503-15 & 2021-11-30 &
- NCSA & \textbf{Test report: Ops rehearsal DRP (SV data)} 
+ NCSA & \textbf{ Ops rehearsal DRP (SV data)} 
 • Science Verification data will now be available. Demonstate its use in producting a data release.
  \\ \hline
 LDM-503-16 & 2022-02-28 &

diff --git a/makeDMtestmilestones.py b/makeDMtestmilestones.py
@@ -6,6 +6,7 @@
 
 fn = sys.argv[1]
 of = fn.replace(".csv", ".tex")
+tf = open("testsections.tex","w")
 out = open(of, "w")
 
 
@@ -19,12 +20,16 @@
             c= c+ 1
             desc = row[c]
             desc=re.sub(r'\#', '\\#', desc)
+            desc=re.sub(r'Test report:', '', desc)
             c= c+ 1
             date = row[c]
             c= c+ 1
             txt = row[c]
             txt=re.sub(r'\&', 'and', txt)
             txt=re.sub(r'\#', '\\#', txt)
-
             c= c+ 1
+            note = row[c]
             out.write(tid+" & "+date +" &\n NCSA & \\textbf{"+ desc +"} \n" + txt + "\n \\\\ \hline\n")
+            tf.write("\subsection{"+desc+"\\textbf{("+tid+")}\label{"+tid+"}}\n")
+            tf.write(txt+"\n \\newline")
+            tf.write(note+"\n")
diff --git a/testsections.tex b/testsections.tex
@@ -0,0 +1,54 @@
+\subsection{ HSC reprocessing\textbf{(LDM-503-2)}\label{LDM-503-2}}
+• Validate the data products with the LSST stack match or improve upon HSC products.
+ \newline• From the pipelines perspective, I think there's no new work involved here beyond the v13.0 release (at which point the HSC merge is complete and QA has been performed). Suggest we'd run this with the latest release as of the date of the test (so this is 14.N, where 14.0 is the end-of-S17 release).
+\subsection{ Alert generation validation\textbf{(LDM-503-3)}\label{LDM-503-3}}
+• Validate the alert generation stack performance on several DECam and HSC datasets.
+ \newline• "Stack" is probably ill-defined here — is this simply testing science logic, or are we going after a wider integration exercise?
+\subsection{ Camera DAQ integration funtionalty test\textbf{(LDM-503-4)}\label{LDM-503-4}}
+• The data acquisition hardware should be available to DM on 2018-02-30. We should prepare by testing that adjacent archive systems.
+ \newline• A minimal DM-only system that can archive mocked-up images and demonstrate that they can be retrieved, with provenance and metadata.
+\subsection{Test Report: Camera DAQ interface Integration Verification \textbf{(LDM-503-4b)}\label{LDM-503-4b}}
+• The data acquisition hardware should be available to DM on 2018-02-30. We should test integration with the adjacent archive systems.
+ \newline• A minimal system that can archive simulated images from the DAQ Test Stand and demonstrate that they can be retrieved.
+\subsection{ Alert distribution validation\textbf{(LDM-503-5)}\label{LDM-503-5}}
+• Validate alert distribution system and mini-broker fed by live or simulated live data.
+ \newline• Can we test a SUIT interface to the broker at this point? I believe it's not scheduled until later in construction.
+\subsection{ DM ComCam interface verification readiness\textbf{(LDM-503-6)}\label{LDM-503-6}}
+• ComCam will be in Tucson on 2018-07-24. The DM system must be ready to deal with it.
+ \newline• "The DM system" should use some further definition: what do we want to test here? Data flow from ComCam to the Data Backbone, or science processing of ComCam data?
+\subsection{ Camera data processing\textbf{(LDM-503-7)}\label{LDM-503-7}}
+• Partial camera data should be available to DM July 31st. We plan to test DM stack with it.
+ \newline
+\subsection{ Spectrograph data acquisition\textbf{(LDM-503-8)}\label{LDM-503-8}}
+• Demonstrate that we can acquire (and process?) data from the spectrograph.
+ \newline• Per LSE-79, AuxTel delivery in Nov 2017 (ie, a year before this milestone) includes: → EFD ETL service → Aux Telescope Archiving Service → Data backbone in support of Aux Telescope archiving Do we need to schedule another test to cover that?
+\subsection{Verification tests in advance of pre-ops rehearsal for commissioning \#1\textbf{(LDM-503-9)}\label{LDM-503-9}}
+• Test how the system will run during commissioning.
+ \newline• "Focus on ISR" — we should test whatever we have available. See LSE-79 for a list of requirements.
+\subsection{ DAQ validation\textbf{(LDM-503-10)}\label{LDM-503-10}}
+• There is a project Milestone that DAQ/DM/Networks are available March 15th. We need to run tests in Feb to show this is ready.
+ \newline
+\subsection{ DM ComCam operations readiness \textbf{(LDM-503-11a)}\label{LDM-503-11a}}
+• ComCam will be in use in Nov . The DM system must be ready to deal with it.
+ \newline• "The DM system" should use some further definition: what do we want to test here? Data flow from ComCam to the Data Backbone, or science processing of ComCam data?
+\subsection{Verification tests in advance of pre-ops rehearsal for commissioning \#2\textbf{(LDM-503-11)}\label{LDM-503-11}}
+• More complete commissioning rehearsal: how do the scientists look at data? How do they provide feedback to the telescope? How do we create/update calibrations?
+ \newline
+\subsection{Verification tests in advance of pre-ops rehearsal for commissioning \#3\textbf{(LDM-503-12)}\label{LDM-503-12}}
+• Dress rehearsal: commissioning starts in April, so by this stage we should be ready to do everything needed.
+ \newline
+\subsection{ Ops rehearsal DRP (ComCam data)\textbf{(LDM-503-13)}\label{LDM-503-13}}
+• ComCam data will now be available. Demonstrate its use in producing a data release.
+ \newline• Note that LSE-79 requires a suite of DM services in support of the full camera in May 2020. It seems inappropriate to test them as part of a commissioning ops rehearsal, but they are well before this data. Do we need another test milestone?
+\subsection{ DM Software for Science Verification\textbf{(LDM-503-14)}\label{LDM-503-14}}
+• Science Verification starts in April. Demonstrate that all required DM software is available.
+ \newline• SV will include calculating all KPMs to demonstrate that we are reaching the science requirements. That obviously means we'll need code which is both capable of reaching those requirements, and calculating the KPMs. 
+\subsection{ Ops rehearsal DRP (SV data)\textbf{(LDM-503-15)}\label{LDM-503-15}}
+• Science Verification data will now be available. Demonstate its use in producting a data release.
+ \newline
+\subsection{Verification tests in advance of full scale ops rehearsal \#1\textbf{(LDM-503-16)}\label{LDM-503-16}}
+• Test readiness for operations.
+ \newline
+\subsection{Verification tests in advance of full scale ops rehearsal \#2\textbf{(LDM-503-17)}\label{LDM-503-17}}
+• Confirm readiness for operations.
+ \newline