Updates on GE21 online/offlineDQM plots with division on modules, a backport to 12_4_X

[quark2]

PR description:

This PR provides massive updates on GE21 onlineDQM/offlineDQM plots. This work is focusing on the division of GE21 information by modules (M1-M4). It is necessary because of the electronic configuration of GE21 chambers.

It is also included that the module information is propagated to the GEM unpacker in a better way, which is done by @yeckang.

PR validation:

Tests are done with cmsRun $CMSSW_RELEASE_BASE/src/DQM/Integration/python/clients/gem_dqm_sourceclient-live_cfg.py unitTest=True runNumber=361054 dataset=/ExpressPhysics/Run2022F-Express-v1/FEVT and runTheMatrix.py -l limited -i all --ibeos since it makes effects on P5 and reconstruction

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

This PR is a backport of #39985 to 12_4_X.

@jshlee @watson-ij @seungjin-yang @yeckang

[cmsbuild]

A new Pull Request was created by @quark2 for CMSSW_12_4_X.

It involves the following packages:

• DQM/GEM (dqm)
• DataFormats/GEMDigi (upgrade, simulation)
• EventFilter/GEMRawToDigi (reconstruction)

@civanch, @clacaputo, @mandrenguyen, @emanueleusai, @ahmad3213, @cmsbuild, @AdrianoDee, @srimanob, @jfernan2, @mdhildreth, @syuvivida, @pmandrik, @micsucmed, @rvenditti can you please review it and eventually sign? Thanks.
@giovanni-mocellin, @jshlee, @rovere, @eyigitba, @Martin-Grunewald, @missirol, @watson-ij, @trtomei, @beaucero this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

• Backported from Updates on GE21 online/offlineDQM plots with division on modules #39985

[emanueleusai]

type gem

[emanueleusai]

backport of #39985

[emanueleusai]

please test

[emanueleusai]

hold

• this backport is not to be merged since it contains changes to reco. This backport is only for the deployment to DQM online systems

[cmsbuild]

Pull request has been put on hold by @emanueleusai
They need to issue an unhold command to remove the hold state or L1 can unhold it for all

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b1e81c/29028/summary.html
COMMIT: 3d6fcf1
CMSSW: CMSSW_12_4_X_2022-11-15-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/40070/29028/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 4 differences found in the comparisons
• DQMHistoTests: Total files compared: 50
• DQMHistoTests: Total histograms compared: 3720277
• DQMHistoTests: Total failures: 56
• DQMHistoTests: Total nulls: 175
• DQMHistoTests: Total successes: 3720024
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -14085.17 KiB( 49 files compared)
• DQMHistoSizes: changed ( 10024.0,... ): -1.219 KiB GEM/Digis
• DQMHistoSizes: changed ( 11634.0,... ): -34.328 KiB GEM/Digis
• DQMHistoSizes: changed ( 11634.0,... ): 2.953 KiB GEM/DAQStatus
• DQMHistoSizes: changed ( 11634.0,... ): 0.914 KiB GEM/EventInfo
• DQMHistoSizes: changed ( 11634.0,... ): -0.094 KiB GEM/RecHits
• DQMHistoSizes: changed ( 23234.0,... ): -1086.375 KiB GEM/Digis
• DQMHistoSizes: changed ( 23234.0,... ): 4.219 KiB GEM/DAQStatus
• DQMHistoSizes: changed ( 23234.0,... ): -2.500 KiB GEM/RecHits
• DQMHistoSizes: changed ( 23234.0,... ): 1.125 KiB GEM/EventInfo
• DQMHistoSizes: changed ( 35034.0,... ): -1958.438 KiB GEM/Digis
• DQMHistoSizes: changed ( 35034.0 ): ...
• Checked 208 log files, 45 edm output root files, 50 DQM output files
• TriggerResults: no differences found

[micsucmed]

HI @quark2, we weren't able to merge this PR into our latest version of 12_4_X (12_4_10_patch3) given that it presents merge conflicts for the following files:

[0] Auto-merging DQM/GEM/plugins/GEMDAQStatusSource.cc
[0] CONFLICT (content): Merge conflict in DQM/GEM/plugins/GEMDAQStatusSource.cc
[0] Auto-merging DataFormats/GEMDigi/interface/GEMOHStatus.h
[0] CONFLICT (content): Merge conflict in DataFormats/GEMDigi/interface/GEMOHStatus.h
[0] Auto-merging DataFormats/GEMDigi/interface/GEMVFATStatus.h
[0] CONFLICT (content): Merge conflict in DataFormats/GEMDigi/interface/GEMVFATStatus.h
[0] Auto-merging DataFormats/GEMDigi/src/classes_def.xml
[0] CONFLICT (content): Merge conflict in DataFormats/GEMDigi/src/classes_def.xml
[0] Auto-merging EventFilter/GEMRawToDigi/plugins/GEMRawToDigiModule.cc
[0] CONFLICT (content): Merge conflict in EventFilter/GEMRawToDigi/plugins/GEMRawToDigiModule.cc

After verifying, we are already running PR 39489 ontop of this 12_4_X CMSSW release. This is most likely why the conflicts present while merging.

[cmsbuild]

Pull request #40070 was updated. @civanch, @clacaputo, @mandrenguyen, @emanueleusai, @ahmad3213, @cmsbuild, @AdrianoDee, @srimanob, @jfernan2, @mdhildreth, @syuvivida, @pmandrik, @micsucmed, @rvenditti can you please check and sign again.

[quark2]

Hi @micsucmed,

I've merged Yechan's PR manually. I hope it will work. But then, should I also do the same thing to the PRs to 12_5_X and 12_6_X (master)?

[emanueleusai]

@quark2 I believe the 12_5 and master version of #39489 are merged already, so it should not be necessary

[emanueleusai]

please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b1e81c/29159/summary.html
COMMIT: 048e559
CMSSW: CMSSW_12_4_X_2022-11-20-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/40070/29159/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 10 differences found in the comparisons
• DQMHistoTests: Total files compared: 50
• DQMHistoTests: Total histograms compared: 3720277
• DQMHistoTests: Total failures: 67
• DQMHistoTests: Total nulls: 176
• DQMHistoTests: Total successes: 3720012
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -14085.174 KiB( 49 files compared)
• DQMHistoSizes: changed ( 10024.0,... ): -1.219 KiB GEM/Digis
• DQMHistoSizes: changed ( 11634.0,... ): -34.328 KiB GEM/Digis
• DQMHistoSizes: changed ( 11634.0,... ): 2.953 KiB GEM/DAQStatus
• DQMHistoSizes: changed ( 11634.0,... ): 0.914 KiB GEM/EventInfo
• DQMHistoSizes: changed ( 11634.0,... ): -0.094 KiB GEM/RecHits
• DQMHistoSizes: changed ( 23234.0,... ): -1086.375 KiB GEM/Digis
• DQMHistoSizes: changed ( 23234.0,... ): 4.219 KiB GEM/DAQStatus
• DQMHistoSizes: changed ( 23234.0,... ): -2.500 KiB GEM/RecHits
• DQMHistoSizes: changed ( 23234.0,... ): 1.125 KiB GEM/EventInfo
• DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
• DQMHistoSizes: changed ( 35034.0 ): ...
• Checked 208 log files, 45 edm output root files, 50 DQM output files
• TriggerResults: no differences found

[emanueleusai]

+1

• differences in GEM DQM plots as expected

[perrotta]

As discussed and agreed with DQM and all other groups present at the ORP meeting on Dec 13, we can now close this PR (which will never be merged in the offline release anyhow) and clean up a bit our queues