Implement failure analysis report as use collections for non-function…

…al requirements

(refs idaholab#16410)

framework/doc/acronyms.yml

@@ -35,3 +35,4 @@ SDD: Software Design Description
 SRS: Software Requirement Specification
 STP: Software Test Plan
 VVR: Verification and Validation Report
+FAR: Failure Analysis Report

framework/doc/content/css/sqa_moose.css

@@ -20,7 +20,7 @@
     color:red;
 }
 
-.moose-sqa-items a:not(:last-child):after{
+.moose-sqa-items *:not(:last-child):after{
     content:"; "
 }
 

framework/doc/content/source/index.md

@@ -4,4 +4,4 @@
 
 The following is a complete list of documented C++ source.
 
-!content location=source
+!content list location=source

framework/doc/content/sqa/framework_far.md

@@ -0,0 +1,130 @@
+!template load file=far.md.template project=Framework
+
+!template! item key=introduction
+MOOSE and MOOSE-based applications are designed to operate as a library of functionality. While
+each library may be tailored for solving a certain set of equations, the ability to create arbitrary
+simulations exists. This flexibility exists by design within the framework, modules, and applications.
+With respect to performing failure analysis, the flexibility is detrimental since there lacks
+a will defined problem to asses. To minimize the possibility of failure for a simulation various
+automated methods exist for developers. This document discusses these features and includes a
+list of requirements associated with software failure analysis.
+!template-end!
+
+!template! item key=failure-analysis
+MOOSE has three primary methods for handling simulation failures that range from input errors to
+simulation convergence problems. These potential failures and the associated handling of the failure
+are ubiquitous across MOOSE and MOOSE-based applications. The next three sections detail the handling
+of these common sources of failures.
+
+1. Input file syntax failure,
+2. Input parameter errors, and
+3. Convergence failure.
+
+To complement the automatic handling of these three failure mechanisms the MOOSE testing system
+includes a mechanism for creating tests to verify that errors are captured and reported correctly.
+This testing method is detailed in the [#failure-testing] section.
+
+### Input File Failure
+
+The input file parsing [(see Parser)](Parser.md) automatically handles syntax mistakes are reports
+errors. For example, consider the following input file that contains a missing closing
+bracket.
+
+!listing parser/hit_error/hit_error.i
+
+If this input file is executed with the application, it will automatically report the error
+and associated line number where it occurred as follows.
+
+```text
+hit_error.i:5: missing closing '[]' for section
+```
+
+### Input Parameter Errors id=input-parameter-errors
+
+The input parameter system (see [utils/InputParameters.md]) is the second step in input file
+parsing. The system details the available inputs for an object. The system allows for
+parameters to be marked as required, provide a default, or check for correct range to name a few.
+For example, consider the `validParams` function below that defines a required parameter "D" that
+must be supplied in an input file.
+
+!listing test/src/kernels/CoeffParamDiffusion.C start=MooseDocs::start end=MooseDocs::end include-start=False
+
+If an input file does not include this parameter, as shown below then it will provide an error
+with details regarding the missing parameter.
+
+!listing param_error/param_error.i block=Kernels
+
+```text
+param_error.i:10: missing required parameter 'Kernels/diffusion/D'
+	Doc String: "The diffusivity coefficient."
+```
+
+### Convergence Failure
+
+MOOSE includes automatic methods to handle convergence failures during the numeric solve. If those
+attempts fail, it will exit with an error indicating of the failed solve and the reason. By default
+if a transient simulation fails to solve a time step, the timestep will automatically be cut and the
+solve re-attempted. This cutback will continue until the solve converges or if the minimum allowed
+timestep is reached.
+
+For example, the following input when executed will demonstrate the behavior. This input file
+includes a custom `TimeStepper` block, but by default a similar behavior exists.
+
+!listing cutback_factor_at_failure/constant_dt_cutback.i block=Executioner
+
+When executed this input file at time step 3 fails to converge, the timestep ("dt") is
+cut by the supplied factor and the solve is re-attempted. In both the converged and non-converged
+iterations the reason for the resulting solve is displayed.
+
+
+```text
+Time Step 3, time = 0.3, dt = 0.1
+ 0 Nonlinear |R| = 7.103698e-02
+      0 Linear |R| = 7.103698e-02
+      1 Linear |R| = 1.154171e-03
+      2 Linear |R| = 4.325671e-06
+      3 Linear |R| = 2.434939e-08
+  Linear solve converged due to CONVERGED_RTOL iterations 3
+ 1 Nonlinear |R| = 2.429061e-08
+      0 Linear |R| = 2.429061e-08
+      1 Linear |R| = 2.035627e-10
+      2 Linear |R| = 9.270880e-13
+      3 Linear |R| = 6.368586e-15
+  Linear solve converged due to CONVERGED_RTOL iterations 3
+ 2 Nonlinear |R| = 6.368769e-15
+Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 2
+ Solve Did NOT Converge!
+Aborting as solve did not converge
+
+Time Step 3, time = 0.28, dt = 0.08
+ 0 Nonlinear |R| = 7.103698e-02
+      0 Linear |R| = 7.103698e-02
+      1 Linear |R| = 8.875771e-04
+      2 Linear |R| = 3.163939e-06
+      3 Linear |R| = 1.554863e-08
+  Linear solve converged due to CONVERGED_RTOL iterations 3
+ 1 Nonlinear |R| = 1.565086e-08
+      0 Linear |R| = 1.565086e-08
+      1 Linear |R| = 1.120313e-10
+      2 Linear |R| = 4.275206e-13
+      3 Linear |R| = 2.854434e-15
+  Linear solve converged due to CONVERGED_RTOL iterations 3
+ 2 Nonlinear |R| = 2.874867e-15
+Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 2
+ Solve Converged!
+```
+
+### Failure Testing id=failure-testing
+
+In general, failures are tested using a test type of `RunException` (see [framework_stp.md]). An
+example of such as test is provided below, which is a test that exists for the previous
+input parser example in [#input-parameter-errors]. By default all `RunException` tests
+are listed below in the list in of requirements ([#failure-analysis-requirements]) that comprise
+failure analysis.
+
+!listing param_error/tests
+
+!template-end!
+
+!template item key=failure-analysis-requirements
+!sqa requirements link=True collections=FAILURE_ANALYSIS category=framework

framework/doc/content/sqa/framework_rtm.md

@@ -38,27 +38,13 @@ can be found in the [Software Test Plan](sqa/framework_stp.md).
 !template-end!
 
 !template item key=functional-requirements
-!sqa requirements prefix=F category=framework
+!sqa requirements collections=FUNCTIONAL category=framework
 
 !template item key=usability-requirements
-!sqa requirements prefix=U category=framework
+!sqa requirements collections=USABILITY category=framework
 
 !template item key=performance-requirements
-!sqa requirements prefix=P category=framework
+!sqa requirements collections=PERFORMANCE category=framework
 
 !template item key=system-interfaces-requirements
-!sqa requirements prefix=S category=framework
-
-!template! item key=requirement-collections-intro
-A "collection" is a grouping of requirements that are serving a similar purpose. For example, the
-"FAILURE_ANALYSIS" collection is comprised of requirements that perform checks for simulation
-failures. The following table lists the names and descriptions of the available collections.
-
-!sqa collections-list caption=List of requirement "collections" names and descriptions.
-
-The following is a complete list of each requirement that has been assigned to a collection.
-!template-end!
-
-!template! item key=requirement-collections
-!sqa collections category=framework
-!template-end!
+!sqa requirements collections=SYSTEM category=framework

framework/doc/content/sqa/framework_srs.md

@@ -81,16 +81,16 @@ each addition to the MOOSE framework or its modules.
 !template-end!
 
 !template item key=functional-requirements
-!sqa requirements link=False prefix=F category=framework
+!sqa requirements link=False collections=FUNCTIONAL category=framework
 
 !template item key=usability-requirements
-!sqa requirements link=False prefix=U category=framework
+!sqa requirements link=False collections=USABILITY category=framework
 
 !template item key=performance-requirements
-!sqa requirements link=False prefix=P category=framework
+!sqa requirements link=False collections=PERFORMANCE category=framework
 
 !template item key=system-interfaces-requirements
-!sqa requirements link=False prefix=S category=framework
+!sqa requirements link=False collections=SYSTEM category=framework
 
 !template! item key=human-system-integration
 [!ac](MOOSE) is a command line driven application which conforms to all standard terminal
@@ -182,17 +182,3 @@ No special requirements are needed for packaging or shipping any media containin
 some [!ac](MOOSE)-based applications maybe be export controlled in which case all export control restrictions must
 be adhered to when packaging and shipping media.
 !template-end!
-
-!template! item key=requirement-collections-intro
-A "collection" is a grouping of requirements that are serving a similar purpose. For example, the
-"FAILURE_ANALYSIS" collection is comprised of requirements that perform checks for simulation
-failures. The following table lists the names and descriptions of the available collections.
-
-!sqa collections-list caption=List of requirement "collections" names and descriptions.
-
-The following is a complete list of each requirement that has been assigned to a collection.
-!template-end!
-
-!template! item key=requirement-collections
-!sqa collections category=framework link=False
-!template-end!

framework/doc/content/templates/sqa/app_far.md.template

@@ -0,0 +1,26 @@
+# {{app}} Failure Analysis Report
+
+## Introduction id=introduction
+
+!template field key=introduction required=False
+The [!ac](SRS) for {{app}} describes the system functional and non-functional requirements that
+describe the expected interactions that the software shall provide.
+
+## Dependencies id=dependencies
+
+!template field key=dependencies-intro required=False
+The {{app}} application is developed using MOOSE and is based on various modules, as such
+the [!ac](SRS) for {{app}} is dependent upon the following documents.
+
+!template! field key=dependencies required=False
+!sqa dependencies suffix=far category={{category}}
+!template-end!
+
+## Failure Analysis Requirements id=failure-analysis-requirements
+
+!template field key=failure-analysis-requirements-intro required=False
+The following is a complete list for all the  requirements related to failure analysis for {{app}}.
+
+!template! field key=failure-analysis-requirements required=False
+!sqa requirements link=True collections=FAILURE_ANALYSIS category={{category}}
+!template-end!

framework/doc/content/templates/sqa/app_index.md.template

@@ -5,5 +5,6 @@
 - [!sqa!document category={{category}} suffix=rtm](Requirement Traceability Matrix (RTM))
 - [!sqa!document category={{category}} suffix=vvr](Verification and Validation Report (VVR))
 - [!sqa!document category={{category}} suffix=stp](Software Test Plan (STP))
+- [!sqa!document category={{category}} suffix=far](Failure Analysis Report (FAR))
 
 !sqa report category={{category}}

framework/doc/content/templates/sqa/app_rtm.md.template

@@ -22,14 +22,26 @@ the [!ac](RTM) for {{app}} is dependent upon the following documents.
 The following is a complete list for all the functional requirements including links to the
 design documents and test cases for {{app}}.
 
-!template! field key=requirements required=False
-!sqa requirements link=True category={{category}}
+### Functional Requirements
+
+!template! field key=functional-requirements required=False
+!sqa requirements link=True collections=FUNCTIONAL category={{category}}
+!template-end!
+
+### Usability Requirements
+
+!template! field key=usability-requirements required=False
+!sqa requirements link=True collections=USABILITY category={{category}}
 !template-end!
 
-## Requirement Collections id=requirement-collections
+### Performance Requirements
+
+!template! field key=performance-requirements required=False
+!sqa requirements link=True collections=PERFORMANCE category={{category}}
+!template-end!
 
-!template field key=requirement-collections-intro required=False
+### System Requirements
 
-!template! field key=requirement-collections required=False
-!sqa collections category={{category}}
+!template! field key=system-requirements required=False
+!sqa requirements link=True collections=SYSTEM category={{category}}
 !template-end!

framework/doc/content/templates/sqa/app_srs.md.template

@@ -21,14 +21,26 @@ the [!ac](SRS) for {{app}} is dependent upon the following documents.
 !template field key=requirements-intro required=False
 The following is a complete list for all the functional requirements for {{app}}.
 
-!template! field key=requirements required=False
-!sqa requirements link=False category={{category}}
+### Functional Requirements
+
+!template! field key=functional-requirements required=False
+!sqa requirements link=True collections=FUNCTIONAL category={{category}}
+!template-end!
+
+### Usability Requirements
+
+!template! field key=usability-requirements required=False
+!sqa requirements link=True collections=USABILITY category={{category}}
 !template-end!
 
-## Requirement Collections id=requirement-collections
+### Performance Requirements
+
+!template! field key=performance-requirements required=False
+!sqa requirements link=True collections=PERFORMANCE category={{category}}
+!template-end!
 
-!template field key=requirement-collections-intro required=False
+### System Requirements
 
-!template! field key=requirement-collections required=False
-!sqa collections category={{category}} link=False
+!template! field key=system-requirements required=False
+!sqa requirements link=True collections=SYSTEM category={{category}}
 !template-end!

framework/doc/content/templates/sqa/far.md.template

@@ -0,0 +1,22 @@
+# {{project}} Failure Analysis Report
+
+## Introduction id=introduction
+
+!template field key=introduction
+Introduce the failure analysis performed.
+
+## References id=references
+
+!bibtex bibliography title=None
+
+## Failure Analysis
+
+!template field key=failure-analysis
+Detail the failure analysis performed, with respect to the requirements/tests marked as
+failure analysis.
+
+## Failure Analysis Requirements id=failure-analysis-requirements
+
+!template! field key=failure-analysis-requirements
+This section of the specification should contain all of the requirements related to failure analysis.
+!template-end!

framework/doc/content/templates/sqa/rtm.md.template

@@ -107,13 +107,3 @@ Include the tracebility matrix for performance requirements.
 
 !template field key=system-interfaces-requirements
 Include the tracebility matrix for system interface requirements.
-
-## Requirement Collections id=requirement-collections
-
-!template field key=requirement-collections-intro required=False
-
-!template! field key=requirement-collections
-This section should contain lists of requirement collections, where a "collection" is simply a
-grouping of requirements that are related in some fashion. For example, all tests for error handling
-could be grouped into a "failure analysis" collection.
-!template-end!

framework/doc/content/templates/sqa/srs.md.template

@@ -1,4 +1,4 @@
-# {{project}} System Requirements Specification
+# {{project}} Failure Analysis Report
 
 ## Introduction id=introduction
 
@@ -358,13 +358,3 @@ transported within its intended operational context.
 Provide the verification approaches and methods planned to qualify the system or system element. The
 information items for verification are recommended to be given in a parallel manner with the
 information items in Subsections 3.1 through 3.13.
-
-## Requirement Collections id=requirement-collections
-
-!template field key=requirement-collections-intro required=False
-
-!template! field key=requirement-collections
-This section should contain lists of requirement collections, where a "collection" is simply a
-grouping of requirements that are related in some fashion. For example, all tests for error handling
-could be grouped into a "failure analysis" collection.
-!template-end!

modules/contact/doc/content/modules/contact/sqa/contact_far.md

@@ -0,0 +1 @@
+!template load file=sqa/app_far.md.template category=contact app=Contact

modules/doc/config.yml

@@ -117,6 +117,8 @@ Extensions:
                 repo: idaholab/moose
     MooseDocs.extensions.template:
         active: True
+        args:
+            company: Ford # used for example in python/MooseDocs/extensions/template.md
     MooseDocs.extensions.sqa:
         active: True
         reports: !include modules/doc/sqa_reports.yml