BeamSpot DQM clients modified to log to onlineDB only when beam status from TCDS is STABLE (backport for 12_3_X) #37618

dzuolo · 2022-04-19T10:26:47Z

Following discussion with @ggovi BeamSpot DQM clients have been modified in such a way that they log to onlineDB only when the beam mode from TCDS is STABLE. This prevents the generation of dummy logs containing only "fit failed" reports during, e.g., cosmics data taking.

Code in this PR compile and the following local tests were successful

-scram b code-format
-scram b code-checks
-scramv1 b runtests

This PR is a backport of #37614

FYI: @gennai @francescobrivio

…from TCDS is STABLE

cmsbuild · 2022-04-19T10:27:06Z

A new Pull Request was created by @dzuolo (Davide Zuolo) for CMSSW_12_3_X.

It involves the following packages:

DQM/BeamMonitor (dqm)

@emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @pmandrik, @micsucmed, @rvenditti can you please review it and eventually sign? Thanks.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

Backported from BeamSpot DQM clients modified to log to onlineDB only when beam status from TCDS is STABLE #37614

dzuolo · 2022-04-19T10:28:14Z

test parameters:

addpkg = DQM/Integration

dzuolo · 2022-04-19T10:29:22Z

Apparently I am not an "authorised user" that can issue the test command so if somebody could it for me I would appreciate it :)

mmusich · 2022-04-19T10:30:08Z

test parameters:

addpkg = DQM/Integration

mmusich · 2022-04-19T10:30:14Z

please test

mmusich · 2022-04-19T10:39:36Z

@dzuolo the title is clipped (it ends in ...) and also it doesn't say it's for 12.3.X

mmusich · 2022-04-19T10:41:30Z

DQM/BeamMonitor/plugins/BeamMonitor.cc

@@ -661,6 +664,12 @@ void BeamMonitor::beginLuminosityBlock(const LuminosityBlock& lumiSeg, const Eve

 // ----------------------------------------------------------
 void BeamMonitor::analyze(const Event& iEvent, const EventSetup& iSetup) {
+  Handle<TCDSRecord> tcdsData;
+  iEvent.getByToken(tcdsToken_, tcdsData);


same comment as #37614 (comment)

cmsbuild · 2022-04-19T15:32:51Z

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-90f9f0/24005/summary.html
COMMIT: ddf05d8
CMSSW: CMSSW_12_3_X_2022-04-19-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/37618/24005/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test TestDQMOnlineClient-beam_dqm_sourceclient had ERRORS

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 2 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3697135
DQMHistoTests: Total failures: 8
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3697105
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 204 log files, 45 edm output root files, 49 DQM output files
TriggerResults: no differences found

cmsbuild · 2022-04-20T08:47:51Z

Pull request #37618 was updated. @emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @pmandrik, @micsucmed, @rvenditti can you please check and sign again.

francescobrivio · 2022-04-20T09:03:16Z

backport of #37614

cmsbuild · 2022-04-20T09:12:45Z

Pull request #37618 was updated. @emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @pmandrik, @micsucmed, @rvenditti can you please check and sign again.

francescobrivio · 2022-04-20T09:20:00Z

@cmsbuild please test

francescobrivio · 2022-04-20T09:21:04Z

@jfernan2 @ahmad3213 this should also be tested in the playback system in P5 please.

cmsbuild · 2022-04-20T13:19:59Z

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-90f9f0/24045/summary.html
COMMIT: a6e679c
CMSSW: CMSSW_12_3_X_2022-04-19-2300/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/37618/24045/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

140.56140.56_RunHI2018+RunHI2018+RECOHID18+HARVESTDHI18/step2_RunHI2018+RunHI2018+RECOHID18+HARVESTDHI18.log

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3697135
DQMHistoTests: Total failures: 2
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3697111
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 204 log files, 45 edm output root files, 49 DQM output files
TriggerResults: no differences found

dzuolo · 2022-04-20T13:30:14Z

The log file from the 140.56 workflow is not really explicative https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-90f9f0/24045/runTheMatrixINPUT-results/140.56_RunHI2018+RunHI2018+RECOHID18+HARVESTDHI18/step2_RunHI2018+RunHI2018+RECOHID18+HARVESTDHI18.log. It seems that the file that was opened has not been closed, like the process was interrupted for some reason, since the printouts are identical to the ones from https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-90f9f0/24045/runTheMatrixINPUT-results/140.57_RunHI2018Reduced+RunHI2018Reduced+RECOHID18+HARVESTDHI18/step2_RunHI2018Reduced+RunHI2018Reduced+RECOHID18+HARVESTDHI18.log

francescobrivio · 2022-04-20T14:08:38Z

@cmsbuild please test

francescobrivio · 2022-04-20T17:04:55Z

@jfernan2 @ahmad3213 this should also be tested in the playback system in P5 please.

also tagging @pmandrik

jfernan2 · 2022-04-20T17:09:41Z

@francescobrivio this PR was tested sucessfully by @pmandrik at P5 this morning. I pass the baton to @emanueleusai in what approval concerns since he has kindly agreed to replace me on the PR review. Thanks!

francescobrivio · 2022-04-20T17:27:27Z

@francescobrivio this PR was tested sucessfully by @pmandrik at P5 this morning.

Ah that's great news! Then once the tests are done we are ready to approve this (provided jenkins tests are succesful of course 😄 )

I pass the baton to @emanueleusai in what approval concerns since he has kindly agreed to replace me on the PR review. Thanks!

Thank you very much for your help Javi! ❤️

cmsbuild · 2022-04-20T18:07:28Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-90f9f0/24063/summary.html
COMMIT: a6e679c
CMSSW: CMSSW_12_3_X_2022-04-20-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/37618/24063/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3697135
DQMHistoTests: Total failures: 13
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3697099
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.004 KiB( 48 files compared)
DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
Checked 204 log files, 45 edm output root files, 49 DQM output files
TriggerResults: no differences found

emanueleusai · 2022-04-21T05:15:44Z

+1

cmsbuild · 2022-04-21T05:16:06Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_3_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_12_4_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2022-04-26T00:44:21Z

+1

BeamSpot DQM clients modified to log to onlineDB only when beam mode …

ddf05d8

…from TCDS is STABLE

cmsbuild added this to the CMSSW_12_3_X milestone Apr 19, 2022

cmsbuild added dqm-pending orp-pending pending-signatures tests-pending labels Apr 19, 2022

cmsbuild added tests-started and removed tests-pending labels Apr 19, 2022

mmusich reviewed Apr 19, 2022

View reviewed changes

dzuolo changed the title ~~BeamSpot DQM clients modified to log to onlineDB only when beam mode …~~ BeamSpot DQM clients modified to log to onlineDB only when beam status from TCDS is STABLE (backport for 12_3_X) Apr 19, 2022

cmsbuild added tests-rejected and removed tests-started labels Apr 19, 2022

Implementing Marco's suggestion and fixing BeamMonitor_Pixel client

771c523

cmsbuild added tests-pending and removed tests-rejected labels Apr 20, 2022

cmsbuild added the backport label Apr 20, 2022

Fixing BeamMonitor_HLT client

a6e679c

cmsbuild added tests-started and removed tests-pending labels Apr 20, 2022

cmsbuild added tests-rejected and removed tests-started labels Apr 20, 2022

cmsbuild added tests-started and removed tests-rejected labels Apr 20, 2022

cmsbuild added tests-approved and removed tests-started labels Apr 20, 2022

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels Apr 21, 2022

cmsbuild added backport-ok and removed backport labels Apr 23, 2022

cmsbuild added orp-approved and removed orp-pending labels Apr 26, 2022

cmsbuild merged commit 3a3f6e0 into cms-sw:CMSSW_12_3_X Apr 26, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BeamSpot DQM clients modified to log to onlineDB only when beam status from TCDS is STABLE (backport for 12_3_X) #37618

BeamSpot DQM clients modified to log to onlineDB only when beam status from TCDS is STABLE (backport for 12_3_X) #37618

dzuolo commented Apr 19, 2022

cmsbuild commented Apr 19, 2022 •

edited

dzuolo commented Apr 19, 2022

dzuolo commented Apr 19, 2022

mmusich commented Apr 19, 2022

mmusich commented Apr 19, 2022

mmusich commented Apr 19, 2022

mmusich Apr 19, 2022

cmsbuild commented Apr 19, 2022

cmsbuild commented Apr 20, 2022

francescobrivio commented Apr 20, 2022

cmsbuild commented Apr 20, 2022

francescobrivio commented Apr 20, 2022

francescobrivio commented Apr 20, 2022

cmsbuild commented Apr 20, 2022

dzuolo commented Apr 20, 2022 •

edited

francescobrivio commented Apr 20, 2022

francescobrivio commented Apr 20, 2022

jfernan2 commented Apr 20, 2022

francescobrivio commented Apr 20, 2022

cmsbuild commented Apr 20, 2022

emanueleusai commented Apr 21, 2022

cmsbuild commented Apr 21, 2022

qliphy commented Apr 26, 2022

BeamSpot DQM clients modified to log to onlineDB only when beam status from TCDS is STABLE (backport for 12_3_X) #37618

BeamSpot DQM clients modified to log to onlineDB only when beam status from TCDS is STABLE (backport for 12_3_X) #37618

Conversation

dzuolo commented Apr 19, 2022

cmsbuild commented Apr 19, 2022 • edited

dzuolo commented Apr 19, 2022

dzuolo commented Apr 19, 2022

mmusich commented Apr 19, 2022

mmusich commented Apr 19, 2022

mmusich commented Apr 19, 2022

mmusich Apr 19, 2022

Choose a reason for hiding this comment

cmsbuild commented Apr 19, 2022

Unit Tests

Comparison Summary

cmsbuild commented Apr 20, 2022

francescobrivio commented Apr 20, 2022

cmsbuild commented Apr 20, 2022

francescobrivio commented Apr 20, 2022

francescobrivio commented Apr 20, 2022

cmsbuild commented Apr 20, 2022

RelVals-INPUT

Comparison Summary

dzuolo commented Apr 20, 2022 • edited

francescobrivio commented Apr 20, 2022

francescobrivio commented Apr 20, 2022

jfernan2 commented Apr 20, 2022

francescobrivio commented Apr 20, 2022

cmsbuild commented Apr 20, 2022

Comparison Summary

emanueleusai commented Apr 21, 2022

cmsbuild commented Apr 21, 2022

qliphy commented Apr 26, 2022

cmsbuild commented Apr 19, 2022 •

edited

dzuolo commented Apr 20, 2022 •

edited