Add new plots to ECAL Offline DQM with Pedestal and Laser transparency correction from the Database

[abhih1]

PR description:

At the request of the ECAL DPG, new plots are added to the Offline DQM workflow, involving pedestal RMS values and laser transparency correction of crystals, read from the database directly, as functions of eta.
These plots would be useful in general for Offline DQM and also crucial in understanding if the conditions loaded by CMSSW changes from one run to the other in RelVal MC validation.

PR validation:

The code changes were validated by running the DQM relval workflow 136.874 using the runTheMatrix script
runTheMatrix.py -l 136.874 --ibeos

The resultant DQM output file was examined by uploading it to an Offline DQM test GUI and confirming the new plots.
The plots were also approved by the ECAL DPG team.

Backport:

This is a backport to the PR in master: #34342
This is done to have the changes available in CMSSW_11_3_X which is currently in production.

[cmsbuild]

A new Pull Request was created by @abhih1 (Abhirami Harilal) for CMSSW_11_3_X.

It involves the following packages:

DQM/EcalMonitorTasks

@andrius-k, @kmaeshima, @ErnestaP, @ahmad3213, @cmsbuild, @jfernan2, @rvenditti can you please review it and eventually sign? Thanks.
@rchatter, @simonepigazzini, @thomreis, @argiro this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

[qliphy]

@abhih1 Please make a PR in master first

[abhih1]

@qliphy My apologies. Here's the PR to the master: #34342

[boudoul]

Hi @abhih1 . Thanks for this developement. However, I'm not sure I got the reason for the backport. Unless I missed something (@qliphy can correct me if I'm wrong) , there is no plan to produce MC relvals to validate 11_3, since it is now in production .

[jfernan2]

@abhih1 could you please confirm if this PR is needed in 11_3_X for CRUZET?

[abhih1]

Hi @abhih1 . Thanks for this developement. However, I'm not sure I got the reason for the backport. Unless I missed something (@qliphy can correct me if I'm wrong) , there is no plan to produce MC relvals to validate 11_3, since it is now in production .

Hi @boudoul, We weren't sure what version of DQM would be used for offline DQM during CRUZET, so we made the backport just in case. I have updated my PR description to reflect that this would be useful for Offline DQM validation as well, not just for RelVal. Sorry for the confusion.

[abhih1]

@jfernan2 Yes please, it would be nice to have them included for CRUZET.

[jfernan2]

OK @abhih1 but from your PR description, the question from @boudoul still holds: if you plan to do RelVal, I am afraid CRUZET won't have MC production, unless @qliphy say the opposite

[cmsbuild]

Pull request #34341 was updated. @andrius-k, @kmaeshima, @ErnestaP, @ahmad3213, @cmsbuild, @jfernan2, @rvenditti can you please check and sign again.

[jfernan2]

please test

[abhih1]

@jfernan2 I have updated my PR description to include that it would useful in Offline DQM as well, not just for RelVal, and so would be nice to validate these plots on offline DQM during CRUZET. Sorry for the confusion.

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c19ca0/16521/summary.html
COMMIT: ba94b0e
CMSSW: CMSSW_11_3_X_2021-07-06-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34341/16521/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially added 48514 lines to the logs
• Reco comparison results: 8 differences found in the comparisons
• DQMHistoTests: Total files compared: 38
• DQMHistoTests: Total histograms compared: 2877978
• DQMHistoTests: Total failures: 9
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2877947
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 93.156 KiB( 37 files compared)
• DQMHistoSizes: changed ( 1000.0,... ): 3.327 KiB Ecal/Trends
• Checked 160 log files, 37 edm output root files, 38 DQM output files
• TriggerResults: no differences found

[qliphy]

@cms-sw/dqm-l2 Any comment? We are about to make 11_3 release soon.

[jfernan2]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_11_3_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_12_0_X is complete. This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[qliphy]

+1