[New feature] Implement special running states

[jsturdy]

Brief summary of issue

Need to implement RCMS states RunningDegraded and RunningSoftErrorDetected states and actions associated (FixingSoftErrorDetected)

Types of issue

• Feature request (request for change which adds functionality)

Expected Behavior

Depending on the situation, we will need to raise awareness of a degraded state (fixable or not)

• Fixable (RunningSoftErrorDetected)
  • VFATs power is unexpectedly dropped/RunMode is disabled, caught by monitoring
  • One or more links have been AUTOKILLED and an action can recover this
  • Return VFATs to RunMode, etc
• Not fixable (RunningDegraded)
  • V2 link is lost, requires LV cycle, human intervention

Other situations can also arise which should trigger one of these running states, and this will be linked partially to the FM implementation cms-gem-daq-project/GEM-CMS-FM#20

[mexanick]

Irrelevant for legacy release.