Modernize CondTools/HLT and add unit tests

[mmusich]

PR description:

This is a follow-up to #36129
Goals:

• address comment [AlCa] Move away from edm::EDAnalyzer LegacyModule in AlCaRecoTriggerBitsRcdRead #36129 (review) ;
• modernize code along latest recommendations;
• equip plugins with fillDescriptions methods;
• move plugins from src directory to plugins;
• equip package with unit tests;
• speed-up script for multi-IOV update of the triggerbits by using multiprocessing;

PR validation:

Relies on the updated unit tests.

if this PR is a backport please specify the original PR and why you need to backport that PR:

N/A

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36187/26772

• This PR adds an extra 24KB to repository

[cmsbuild]

A new Pull Request was created by @mmusich (Marco Musich) for master.

It involves the following packages:

• CondTools/HLT (hlt, db)

@malbouis, @cmsbuild, @missirol, @Martin-Grunewald, @ggovi, @francescobrivio, @tvami can you please review it and eventually sign? Thanks.
@silviodonato, @mmusich this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

[mmusich]

@cmsbuild, please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c649ca/20636/summary.html
COMMIT: cd80791
CMSSW: CMSSW_12_2_X_2021-11-19-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36187/20636/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 42
• DQMHistoTests: Total histograms compared: 3327156
• DQMHistoTests: Total failures: 0
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3327134
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 41 files compared)
• Checked 177 log files, 37 edm output root files, 42 DQM output files
• TriggerResults: no differences found

[missirol]

A couple of minor comments/questions.

1. Is CondTools/HLT/BuildFile.xml actually still needed?

2. Optional: given the addition of fillDescriptions, the tweak below (or something similar) would allow to remove the python/ directory altogether (unless there are other reasons to keep it..).

diff --git a/CondTools/HLT/test/AlCaRecoTriggerBitsRcdRead_TEMPL_cfg.py b/CondTools/HLT/test/AlCaRecoTriggerBitsRcdRead_TEMPL_cfg.py
index a3cccaf738a..c7542040823 100644
--- a/CondTools/HLT/test/AlCaRecoTriggerBitsRcdRead_TEMPL_cfg.py
+++ b/CondTools/HLT/test/AlCaRecoTriggerBitsRcdRead_TEMPL_cfg.py
@@ -28,7 +28,8 @@ process.MessageLogger.cout = cms.untracked.PSet(INFO = cms.untracked.PSet(
     ))
 
 # the module writing to DB
-process.load("CondTools.HLT.AlCaRecoTriggerBitsRcdRead_cfi")
+from CondTools.HLT.alCaRecoTriggerBitsRcdRead_cfi import alCaRecoTriggerBitsRcdRead as _alCaRecoTriggerBitsRcdRead
+process.AlCaRecoTriggerBitsRcdRead = _alCaRecoTriggerBitsRcdRead.clone()
 # 'twiki' is default - others are text, python (future: html?)
 #process.AlCaRecoTriggerBitsRcdRead.outputType = 'twiki'
 # If rawFileName stays empty (default), use the message logger for output.
diff --git a/CondTools/HLT/test/AlCaRecoTriggerBitsRcdRead_cfg.py b/CondTools/HLT/test/AlCaRecoTriggerBitsRcdRead_cfg.py
index 5717dd975db..35124d680e2 100644
--- a/CondTools/HLT/test/AlCaRecoTriggerBitsRcdRead_cfg.py
+++ b/CondTools/HLT/test/AlCaRecoTriggerBitsRcdRead_cfg.py
@@ -9,7 +9,8 @@ process.MessageLogger.cout = cms.untracked.PSet(INFO = cms.untracked.PSet(
     ))
 
 # the module writing to DB
-process.load("CondTools.HLT.AlCaRecoTriggerBitsRcdRead_cfi")
+from CondTools.HLT.alCaRecoTriggerBitsRcdRead_cfi import alCaRecoTriggerBitsRcdRead as _alCaRecoTriggerBitsRcdRead
+process.AlCaRecoTriggerBitsRcdRead = _alCaRecoTriggerBitsRcdRead.clone()
 # 'twiki' is default - others are text, python (future: html?)
 #process.AlCaRecoTriggerBitsRcdRead.outputType = 'twiki'
 # If rawFileName stays empty (default), use the message logger for output.
diff --git a/CondTools/HLT/test/AlCaRecoTriggerBitsRcdUpdate_TEMPL_cfg.py b/CondTools/HLT/test/AlCaRecoTriggerBitsRcdUpdate_TEMPL_cfg.py
index 565c9694a8f..8bb16fe8d58 100644
--- a/CondTools/HLT/test/AlCaRecoTriggerBitsRcdUpdate_TEMPL_cfg.py
+++ b/CondTools/HLT/test/AlCaRecoTriggerBitsRcdUpdate_TEMPL_cfg.py
@@ -65,7 +65,8 @@ process.MessageLogger.cout = cms.untracked.PSet(INFO = cms.untracked.PSet(
 ))
 
 # the module writing to DB
-process.load("CondTools.HLT.AlCaRecoTriggerBitsRcdUpdate_cfi")
+from CondTools.HLT.alCaRecoTriggerBitsRcdUpdate_cfi import alCaRecoTriggerBitsRcdUpdate as _alCaRecoTriggerBitsRcdUpdate
+process.AlCaRecoTriggerBitsRcdUpdate = _alCaRecoTriggerBitsRcdUpdate.clone()
 # The IOV that you want to write out, defaut is 1 to -1/inf. 
 process.AlCaRecoTriggerBitsRcdUpdate.firstRunIOV = options.firstRun # docu see...
 #process.AlCaRecoTriggerBitsRcdUpdate.lastRunIOV = -1 # ...cfi
diff --git a/CondTools/HLT/test/AlCaRecoTriggerBitsRcdUpdate_cfg.py b/CondTools/HLT/test/AlCaRecoTriggerBitsRcdUpdate_cfg.py
index 2c83d52e4d2..dcced5416cb 100644
--- a/CondTools/HLT/test/AlCaRecoTriggerBitsRcdUpdate_cfg.py
+++ b/CondTools/HLT/test/AlCaRecoTriggerBitsRcdUpdate_cfg.py
@@ -29,7 +29,8 @@ process.MessageLogger.cout = cms.untracked.PSet(INFO = cms.untracked.PSet(
     ))
 
 # the module writing to DB
-process.load("CondTools.HLT.AlCaRecoTriggerBitsRcdUpdate_cfi")
+from CondTools.HLT.alCaRecoTriggerBitsRcdUpdate_cfi import alCaRecoTriggerBitsRcdUpdate as _alCaRecoTriggerBitsRcdUpdate
+process.AlCaRecoTriggerBitsRcdUpdate = _alCaRecoTriggerBitsRcdUpdate.clone()
 # The IOV that you want to write out, defaut is 1 to -1/inf. 
 #process.AlCaRecoTriggerBitsRcdUpdate.firstRunIOV = 1 # docu see...
 #process.AlCaRecoTriggerBitsRcdUpdate.lastRunIOV = -1 # ...cfi

[mmusich]

If useful, it might be enough to do

it is useful in the context of the unit test, such that also the cmsRun executions are monitored.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36187/26780

• This PR adds an extra 28KB to repository

[cmsbuild]

Pull request #36187 was updated. @malbouis, @cmsbuild, @missirol, @Martin-Grunewald, @ggovi, @francescobrivio, @tvami can you please check and sign again.

[missirol]

please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c649ca/20644/summary.html
COMMIT: 46b0613
CMSSW: CMSSW_12_2_X_2021-11-20-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36187/20644/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 42
• DQMHistoTests: Total histograms compared: 3247021
• DQMHistoTests: Total failures: 0
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3246999
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 41 files compared)
• Checked 177 log files, 37 edm output root files, 42 DQM output files
• TriggerResults: no differences found

[missirol]

+hlt

[tvami]

+db

• code change in line with PR description
• newly introduced unit test runs fine https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c649ca/20644/unitTests/src/CondTools/HLT/test/testWriteReadAlCaRecoTriggerBits/testing.log
• Jenkins tests pass

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

+1