Add new features to BeamSpotOnlineObjects #35338

francescobrivio · 2021-09-19T22:08:52Z

PR description:

In #35193 it was reported that the BeamSpotOnline objects do not contain certain features that were published in DIP during Run2. This PR adds these features by extending the vector members of CondFormats/BeamSpotObjects/interface/BeamSpotOnlineObjects.h, specifically:

Events used in the BeamSpot fit [int]
Mean of the PVs distribution and error on the mean [float]
RMS of the PVs distribution and error on the RMS [float]
Max number of PVs [int]
Start (and end) time of the first (and last) LS used in the fit [string]
Start (and end) time of the first (and last) LS used in the fit [epoch in seconds]

DQM clients BeamMonitor and FakeBeamMonitor have been updated to add to the payload the new parameters for DIP. The DB logs have also been updated with the parameters.
Same as for the BeamSpot values in the FakeBeamMonitor client, all the parameters for DIP produced by the fake client are filled with random numbers coming from a TRandom3.

PR validation:

Tested running:

cmsRun DQM/Integration/python/clients/beamhltfake_dqm_sourceclient-live_cfg.py unitTest=True runNumber=3252833

on /store/express/Run2018E/ExpressPhysics/FEVT/Express-v1/000/325/283/00000/DF31860D-3EA3-2B42-A7CB-ACFE8C2810DF.root

This produces a log which contains:

...
-----------------------------------------------------
              BeamSpotOnline Data
 Beam type    = 2
       X0     = 0.118887 +/- 0 [cm]
       Y0     = -0.0745552 +/- 0 [cm]
       Z0     = 0.0711385 +/- 0 [cm]
 Sigma Z0     = 3.65177 +/- 0 [cm]
 dxdz         = 0 +/- 0 [radians]
 dydz         = 0 +/- 0 [radians]
 Beam Width X = 0.000404199 +/- 0 [cm]
 Beam Width Y = 0.000861879 +/- 0 [cm]
 Emittance X  = 0 [cm]
 Emittance Y  = 0 [cm]
 Beta star    = 0 [cm]
 Last Lumi    = 2
 Last Run     = 325283
 Last Fill    = 0
-----------------------------------------------------
[2021-09-19 21:57:18.589888] INFO: FakeBeamMonitor - Additional parameters for DIP:
[2021-09-19 21:57:18.589900] INFO: Events used in the fit: 821
[2021-09-19 21:57:18.589911] INFO: Mean PV               : 106.167
[2021-09-19 21:57:18.589926] INFO: Mean PV Error         : 9.39724
[2021-09-19 21:57:18.589938] INFO: Rms PV                : 7.43257
[2021-09-19 21:57:18.589950] INFO: Rms PV Error          : 7.42233
[2021-09-19 21:57:18.589962] INFO: Max PVs               : 108
[2021-09-19 21:57:18.589972] INFO: StartTime             : 1970.01.01 00:00:00 GMT
[2021-09-19 21:57:18.589982] INFO: StartTimeStamp        : 0
[2021-09-19 21:57:18.589993] INFO: EndTime               : 1970.01.01 00:00:01 GMT
[2021-09-19 21:57:18.590004] INFO: EndTimeStamp          : 1
...

Which shows that the new values are correectly stored and read from the BeamSpotOnline object

Backport:

A backport to 12_0_X will be provided.

FYI @sikler @mmusich @gennai @dzuolo

francescobrivio · 2021-09-19T22:11:06Z

@ggovi please provide suggestions on whether some specific checks on the new features are needed in CondFormats/BeamSpotObjects/interface/BeamSpotOnlineObjects.h in order to preserve backward compatibility.

cmsbuild · 2021-09-19T22:15:48Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-35338/25391

This PR adds an extra 40KB to repository

cmsbuild · 2021-09-19T22:16:07Z

A new Pull Request was created by @francescobrivio for master.

It involves the following packages:

CondFormats/BeamSpotObjects (db, alca)
DQM/BeamMonitor (dqm)

@malbouis, @yuanchao, @pmandrik, @emanueleusai, @ahmad3213, @rvenditti, @cmsbuild, @jfernan2, @ggovi, @francescobrivio, @pbo0, @tvami can you please review it and eventually sign? Thanks.
@mmusich, @tocheng, @seemasharmafnal this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

tvami · 2021-09-19T23:44:18Z

@cmsbuild , please test

cmsbuild · 2021-09-20T03:03:19Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-bbe8e0/18747/summary.html
COMMIT: 7c7ac74
CMSSW: CMSSW_12_1_X_2021-09-19-0000/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/35338/18747/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 3 differences found in the comparisons
DQMHistoTests: Total files compared: 40
DQMHistoTests: Total histograms compared: 3211080
DQMHistoTests: Total failures: 5
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3211052
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.004 KiB( 39 files compared)
DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
Checked 169 log files, 37 edm output root files, 40 DQM output files
TriggerResults: no differences found

jfernan2 · 2021-09-20T07:01:16Z

Thanks @francescobrivio
In order to properly test in Online DQM at P5, we definitely need the backport to 12_0_X

francescobrivio · 2021-09-20T07:20:35Z

Thanks @francescobrivio
In order to properly test in Online DQM at P5, we definitely need the backport to 12_0_X

Yes definitely, I have a branch ready for the backport already, but first I would like to hear from @ggovi if I need to add any protection in the new getter methods!

tvami · 2021-09-20T16:47:56Z

@francescobrivio is the onlineDbService_->logger().logInfo() anyhow better then just to use the MessageLogger?

DQM/BeamMonitor/plugins/FakeBeamMonitor.cc

francescobrivio · 2021-09-20T16:52:46Z

@francescobrivio is the onlineDbService_->logger().logInfo() anyhow better then just to use the MessageLogger?

onlineDbService_ is used to log the message on CondDB, e.g. https://cms-conddb.cern.ch/cmsDbBrowser/logs/show_O2O_log/Prod/BeamSpotOnlineLegacyPlayback/2021-09-20%2016:00:39.920912 (this is a log for the test tag used in the DQM playback system)

CondFormats/BeamSpotObjects/interface/BeamSpotOnlineObjects.h

tvami · 2021-09-21T13:20:06Z

urgent

we'd like the backport of this in the next 12_0_X
FYI @qliphy @perrotta

francescobrivio · 2021-09-21T13:23:55Z

urgent

* we'd like the backport of this in the next 12_0_X

* FYI @qliphy @perrotta

@qliphy @perrotta
Also: we (BeamSpot team + RunCoordination + TRK) are going to discuss this PR in two days to understand how to threat BeamSpot pubblication on DIP properly (together with #35193).

francescobrivio · 2021-09-21T17:21:20Z

hold

Will discuss on Thursday between BeamSpot + RunCoordination + TRK what is the plan for BeamSpot pubblication in DIP

ggovi · 2021-09-24T13:46:04Z

+1

francescobrivio · 2021-09-24T13:48:57Z

@tvami @malbouis

extends BeamSpotOnlineObjects for DIP pubblication and for future improvemente of the onlineBeamSpotESProducer logic
Method in CondFormats contain a protection that throws an exception when trying to access unexisting objects in old paylaods
all tests passed

tvami · 2021-09-24T13:50:41Z

+alca

see comments in Add new features to BeamSpotOnlineObjects #35338 (comment)

mmusich · 2021-09-24T14:02:25Z

as a follow-up would be nice to add the new parameters to be displayed in the Payload Inspector:

cmssw/CondCore/BeamSpotPlugins/plugins/BeamSpotOnline_PayloadInspector.cc

Lines 68 to 86 in ac2754c

    
           std::function<int(parameters)> mycutFunctor = [this](parameters my_param) { 
        
             int ret(-999.); 
        
             switch (my_param) { 
        
               case lastLumi: 
        
                 return this->m_payload->GetLastAnalyzedLumi(); 
        
               case lastRun: 
        
                 return this->m_payload->GetLastAnalyzedRun(); 
        
               case lastFill: 
        
                 return this->m_payload->GetLastAnalyzedFill(); 
        
               case nTracks: 
        
                 return this->m_payload->GetNumTracks(); 
        
               case nPVs: 
        
                 return this->m_payload->GetNumPVs(); 
        
               case END_OF_TYPES: 
        
                 return ret; 
        
               default: 
        
                 return ret; 
        
             } 
        
           };

francescobrivio · 2021-09-24T14:05:49Z

as a follow-up would be nice to add the new parameters to be displayed in the Payload Inspector:

cmssw/CondCore/BeamSpotPlugins/plugins/BeamSpotOnline_PayloadInspector.cc

Lines 68 to 86 in ac2754c

std::function<int(parameters)> mycutFunctor = [this](parameters my_param) {

int ret(-999.);

switch (my_param) {

case lastLumi:

return this->m_payload->GetLastAnalyzedLumi();

case lastRun:

return this->m_payload->GetLastAnalyzedRun();

case lastFill:

return this->m_payload->GetLastAnalyzedFill();

case nTracks:

return this->m_payload->GetNumTracks();

case nPVs:

return this->m_payload->GetNumPVs();

case END_OF_TYPES:

return ret;

default:

return ret;

}

};

will add to my to do list! 😉

jfernan2 · 2021-09-24T16:19:59Z

+1
Backport tested in Online DQM at P5

cmsbuild · 2021-09-24T16:20:20Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

francescobrivio · 2021-09-24T16:34:33Z

+1
Backport tested in Online DQM at P5

Just for reference, a log for this test can be seen in: https://cms-conddb.cern.ch/cmsDbBrowser/logs/show_O2O_log/Prod/BeamSpotOnlineHLTPlayback/2021-09-24%2016:02:14.900345

qliphy · 2021-09-25T01:44:02Z

+1

…cms-sw#35338

Add new features to member vectors of BeamSpotOnlineObjects

7c7ac74

cmsbuild added this to the CMSSW_12_1_X milestone Sep 19, 2021

cmsbuild added alca-pending code-checks-pending db-pending dqm-pending orp-pending pending-signatures tests-pending labels Sep 19, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Sep 19, 2021

cmsbuild added tests-started and removed tests-pending labels Sep 19, 2021

cmsbuild added tests-approved and removed tests-started labels Sep 20, 2021

tvami reviewed Sep 20, 2021

View reviewed changes

DQM/BeamMonitor/plugins/FakeBeamMonitor.cc Show resolved Hide resolved

mmusich reviewed Sep 20, 2021

View reviewed changes

CondFormats/BeamSpotObjects/interface/BeamSpotOnlineObjects.h Show resolved Hide resolved

cmsbuild added the urgent label Sep 21, 2021

francescobrivio changed the title ~~[RFC] Add new features to BeamSpotOnlineObjects~~ Add new features to BeamSpotOnlineObjects Sep 24, 2021

cmsbuild removed the hold label Sep 24, 2021

francescobrivio mentioned this pull request Sep 24, 2021

Beamspot DIP server (normal DQM client) for 12_0_X #35193

Closed

cmsbuild added db-approved and removed db-pending labels Sep 24, 2021

cmsbuild added alca-approved and removed alca-pending labels Sep 24, 2021

francescobrivio mentioned this pull request Sep 24, 2021

Add new features to member vectors of BeamSpotOnlineObjects [12_0_X] #35405

Merged

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels Sep 24, 2021

cmsbuild added orp-approved and removed orp-pending labels Sep 25, 2021

cmsbuild merged commit 8ec82ec into cms-sw:master Sep 25, 2021

cmsbuild mentioned this pull request Sep 25, 2021

modernize CondTools/SiStrip #35415

Merged

mmusich added a commit to mmusich/cmssw that referenced this pull request Sep 28, 2021

update BeamSpotOnline payload inspector to include parameters added in …

6f551f5

…cms-sw#35338

mmusich added a commit to mmusich/cmssw that referenced this pull request Sep 28, 2021

update BeamSpotOnline payload inspector to include parameters added in …

2ddb4d7

…cms-sw#35338

mmusich mentioned this pull request Sep 28, 2021

update BeamSpotOnline Payload Inspector to include the newly added parameters for DIP #35451

Merged

francescobrivio mentioned this pull request Dec 3, 2021

Apply Naming Rules to BeamSpotOnlineObjects methods #36348

Merged

mmusich mentioned this pull request Nov 1, 2022

runtime error from BeamOnlineMonitor on Run-2 data #39948

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add new features to BeamSpotOnlineObjects #35338

Add new features to BeamSpotOnlineObjects #35338

francescobrivio commented Sep 19, 2021

francescobrivio commented Sep 19, 2021

cmsbuild commented Sep 19, 2021

cmsbuild commented Sep 19, 2021

tvami commented Sep 19, 2021

cmsbuild commented Sep 20, 2021

jfernan2 commented Sep 20, 2021

francescobrivio commented Sep 20, 2021 •

edited

tvami commented Sep 20, 2021

francescobrivio commented Sep 20, 2021

tvami commented Sep 21, 2021

francescobrivio commented Sep 21, 2021

francescobrivio commented Sep 21, 2021 •

edited

ggovi commented Sep 24, 2021

francescobrivio commented Sep 24, 2021

tvami commented Sep 24, 2021

mmusich commented Sep 24, 2021

francescobrivio commented Sep 24, 2021

jfernan2 commented Sep 24, 2021

cmsbuild commented Sep 24, 2021

francescobrivio commented Sep 24, 2021

qliphy commented Sep 25, 2021

Add new features to BeamSpotOnlineObjects #35338

Add new features to BeamSpotOnlineObjects #35338

Conversation

francescobrivio commented Sep 19, 2021

PR description:

PR validation:

Backport:

francescobrivio commented Sep 19, 2021

cmsbuild commented Sep 19, 2021

cmsbuild commented Sep 19, 2021

tvami commented Sep 19, 2021

cmsbuild commented Sep 20, 2021

Comparison Summary

jfernan2 commented Sep 20, 2021

francescobrivio commented Sep 20, 2021 • edited

tvami commented Sep 20, 2021

francescobrivio commented Sep 20, 2021

tvami commented Sep 21, 2021

francescobrivio commented Sep 21, 2021

francescobrivio commented Sep 21, 2021 • edited

ggovi commented Sep 24, 2021

francescobrivio commented Sep 24, 2021

tvami commented Sep 24, 2021

mmusich commented Sep 24, 2021

francescobrivio commented Sep 24, 2021

jfernan2 commented Sep 24, 2021

cmsbuild commented Sep 24, 2021

francescobrivio commented Sep 24, 2021

qliphy commented Sep 25, 2021

francescobrivio commented Sep 20, 2021 •

edited

francescobrivio commented Sep 21, 2021 •

edited