Skip to content

How to restart the DAQ status monitor in DCS summary page

Jump to bottom
Emanuele Leonardi edited this page Nov 19, 2020 · 11 revisions

The DAQ status info is shown in the "PADME DAQ Status" window on the DCS/GENERAL/DCS_Summary/Summary monitor page.

These info are of primary importance for the shifters and are updated every ~30 secs.

Troubleshooting the DAQ Status Monitor server

If the "PADME DAQ Status" info in the DCS summary page stops being updated (you can look at the UTC timestamp) it means that the script collecting these information has crashed and must be restarted.

To check if the DAQ status script is running, login to l0padme3 as monitor and then execute the command:

[daq@l0padme3 ~]$ ps -fu monitor | grep DAQStatusServer | grep -v grep
monitor   42686      1  0 08:18 ?        00:00:02 /usr/bin/python -u ./DAQStatusServer

If no process is shown, this means that the server is not running and you should restart it: always logged on l0padme3 as monitor, execute the commands:

[monitor@l0padme3 ~]$ cd /home/monitor/DAQStatusServer
[monitor@l0padme3 RecoMonitor]$ ./DAQStatusServer
Starting DAQStatusServer in background

If the DAQStatusServer is running but the information are not being updated, you should kill the server ("kill -9 42686" from previous example) and restart it with the previously described procedure.

Then check again if the server is running and verify that the DAQ information in the DCS summary page is updated (can take up to 1 minute before this happens).

If after the restart information are still not updated, please write a note in the eLog and contact the Run Coordinator.

Information shown by the DAQ Status Monitor server

The general summary table "PADME DAQ Status" updates every ~30 secs and shows:

  • UTC timestamp in the header (check if it is up to date!)
  • current shifter
  • current run
  • merger node
  • setup used
  • run type
  • disk space on l1padme1 (orange if occupied space > 60% - red if occupied space > 80%)
  • disk space on l1padme2 (orange if occupied space > 60% - red if occupied space > 80%)
  • disk space on DAQ disk (orange if occupied space > 60% - red if occupied space > 80%)
  • last rawdata file written
  • last reco file written
  • if the monitor reconstruction processes (PadmeReco, RecoMonitor, RunRecoMonitor, Trend) are running
  • last trigger event and current trigger rate (or trigger summary when run is over)
  • trigger mask counters and rates (or trigger massk summary when run is over)
  • last merger event (or merger summary when run is over)
  • last level1 event for each level1 stream
  • the total size of data written during the run in GB and the total number of files written by ALL streams

The "PADME DAQ Trigger Rate Timeline" plot shows the trigger rates of the last ~30min.

The "PADME DAQ Configuration" shows the configuration of the on-going run:

  • Name and type of the current run
  • Name of the setup in use
  • Name of the user who started the run
  • Start of run comment
  • List of ADC boards used in the run
  • List of detectors used in the run
  • List of ADC boards with zero suppression enabled
  • Trigger mask used in the run with corresponding scale and autopass factors

The previously shown info on beam collimators-slits and trigger timing are now available under the dcs/beam/BTF_beam page

© 2015 PADME Collaboration

Clone this wiki locally