Beamspot DIP server (normal DQM client) for 12_0_X [retry]

[sikler]

[reload of #35193 since that was messed up while rebasing to latest IB]

PR description:

• the aim was to get rid of the java-based DIP server (DQM/BeamMonitor/test/dip/BeamSpotDipServer.java) used in the past years to transmit beamspot information to LHC
• detailed information on the Data Interchange Protocol is at https://readthedocs.web.cern.ch/display/ICKB/DIP+and+DIM
• our beamspot server is now rewritten in C++, in the form of a normal DQM client
• the central piece of code is the plugin DQM/BeamMonitor/plugins/BeamSpotDipServer.{cc,h}, the config at DQM/BeamMonitor/python/BeamSpotDipServer_cff.py, and the live client at DQM/Integration/python/clients/beamspotdip_dqm_sourceclient-live_cfg.py
• it uses two external tools: dip (version 5.7.0) and log4cplus [in fact log4cplus is containted in the dip distribution], available at https://readthedocs.web.cern.ch/display/ICKB/DIP+API; now provided as external tools
• mode of operation: the client is alive from begin to end of run; it checks the result of the previous beamspot fit at the end of each lumisection (in dqmEndLuminosityBlock); for now it looks at files containing the last beamspot fit and the actual status of the tracker under /nfshome0/dqmpro/BeamMonitorDQM/
• configurable through readFromNFS: work either on files /nfshome0 (default) or using beamspot from database and Dcs information on tracker status; now the reading of the legacy beamspot is implemented)

PR validation:

• compiles runs on lxplus without warning or error message (together with Add new features to member vectors of BeamSpotOnlineObjects [12_0_X] #35405)
• test using a working DIP browser or java application is not yet possible (javaws is not supported anymore, jws is not available neither on the technical network, nor on lxplus; the page https://dash.web.cern.ch/DIP/ does not work)
• live version to be tested

PR issues, questions:

[cmsbuild]

A new Pull Request was created by @sikler (Ferenc Siklr) for CMSSW_12_0_X.

It involves the following packages:

• DQM/BeamMonitor (dqm)
• DQM/Integration (dqm)

@emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @pmandrik, @pbo0, @rvenditti can you please review it and eventually sign? Thanks.
@battibass, @threus, @batinkov, @francescobrivio this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

• Backported from Beamspot DIP server (normal DQM client) #35433

[tvami]

@cmsbuild , please test

[francescobrivio]

urgent

[tvami]

@cmsbuild , ping

[tvami]

@cmsbuild , code-checks

[tvami]

@cmsbuild , abort

[tvami]

@cmsbuild , code-checks

[tvami]

@smuzaffar the bot seems to be stuck on code-checks, I tried to ping, manually trigger code-checks, abort and do it again, seems like no effect

[tvami]

urgent

• my understanding from Francesco was that this is needed for the beam tests, i.e. this is should be in the 12_0_2 (if possible)

[smuzaffar]

code-checks

[smuzaffar]

@tvami , according to the link mentioned in #35428 (comment) , code checks command is just code-checks :-)

[cmsbuild]

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-35428/25573

• This PR adds an extra 24KB to repository

Code check has found code style and quality issues which could be resolved by applying following patch(s)

• code-checks:
  https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-35428/25573/code-checks.patch
  e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-35428/25573/code-checks.patch | patch -p1
  You can also run scram build code-checks to apply code checks directly

• code-format:
  https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-35428/25573/code-format.patch
  e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-35428/25573/code-format.patch | patch -p1
  You can also run scram build code-format to apply code format directly

[cmsbuild]

Pull request #35428 was updated. @emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @pmandrik, @pbo0, @rvenditti can you please check and sign again.

[francescobrivio]

@cmsbuild please test

[perrotta]

backport of #35433

[cmsbuild]

Pull request #35428 was updated. @emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @pmandrik, @pbo0, @rvenditti can you please check and sign again.

[francescobrivio]

please test

[francescobrivio]

@perrotta @qliphy as mentioned by @smuzaffar in #35433 (comment) the last commit (451b355) it technical, but here we still need the signature from @cms-sw/dqm-l2 (pending tests in P5).

[cmsbuild]

Pull request #35428 was updated. @emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @pmandrik, @pbo0, @rvenditti can you please check and sign again.

[francescobrivio]

please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-341682/19204/summary.html
COMMIT: 75079a5
CMSSW: CMSSW_12_0_X_2021-09-27-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/35428/19204/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 39
• DQMHistoTests: Total histograms compared: 2998564
• DQMHistoTests: Total failures: 0
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2998542
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
• Checked 165 log files, 37 edm output root files, 39 DQM output files
• TriggerResults: no differences found

[jfernan2]

+1
Tested in Online DQM at P5

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_12_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[qliphy]

+1