HLT menu development for `12_3_X` [`12_4_X`] #37754

missirol · 2022-04-30T19:38:12Z

PR description:

Development of HLT menus for CMSSW_12_3_X.

This is the PR for the master branch, which will then be backported to 12_3_X (which is currently the release used for HLT-menu development for Run 3).

Includes the integration of the following JIRA tickets.

Part of CMSHLT-2261: [BPH] Update of low-mass dimuon triggers for Run 3.
CMSHLT-2286: [EGM] Removal of "caloTowers" from E/Gamma reconstruction at HLT.
CMSHLT-2293: [EGM] Update for Run-3 E/Gamma supercluster energy regression.

The configurations were created with V03-03-00 of ConfDB, which introduces support for ProcessAcceleratorCUDA.

PR validation:

TSG tests.

If this PR is a backport, please specify the original PR and why you need to backport that PR:

N/A

cmsbuild · 2022-04-30T19:45:28Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37754/29626

This PR adds an extra 820KB to repository

cmsbuild · 2022-04-30T19:45:53Z

A new Pull Request was created by @missirol (Marino Missiroli) for master.

It involves the following packages:

HLTrigger/Configuration (hlt)

@cmsbuild, @missirol, @Martin-Grunewald can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @silviodonato this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

missirol · 2022-04-30T19:48:59Z

please test

cmsbuild · 2022-04-30T23:30:32Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cbee91/24357/summary.html
COMMIT: 8c48380
CMSSW: CMSSW_12_4_X_2022-04-30-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/37754/24357/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 2 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3689356
DQMHistoTests: Total failures: 68
DQMHistoTests: Total nulls: 162
DQMHistoTests: Total successes: 3689104
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -8633.58 KiB( 48 files compared)
DQMHistoSizes: changed ( 11634.0,... ): -1438.902 KiB HLT/HLTEgammaValidation
DQMHistoSizes: changed ( 11634.0,... ): -0.027 KiB HLT/Filters
Checked 205 log files, 45 edm output root files, 49 DQM output files
TriggerResults: found differences in 6 / 48 workflows

missirol · 2022-05-01T09:47:29Z

The large reduction of DQM outputs in HLT/HLTEgammaValidation is related to the changes in this PR to E/Gamma triggers.

One of the ED producers used in those triggers is changed by this PR (EgammaHLTBcHcalIsolationProducersRegional replaced by EgammaHLTHcalVarProducerFromRecHit), and the DQM/Validation client does not recognize the new modules, because it hard-codes the types of the acceptable ED producers (see here, for example).

This patch [*] is a starting point to fix the issue, but this task is left to EGM and/or DQM experts.

In my opinion, this should not delay this PR.

Attn: @cms-sw/dqm-l2 @cms-sw/egamma-pog-l2

[*]

diff --git a/HLTriggerOffline/Egamma/src/EmDQM.cc b/HLTriggerOffline/Egamma/src/EmDQM.cc
index 0a1fb605a98..43c0e8f9f17 100644
--- a/HLTriggerOffline/Egamma/src/EmDQM.cc
+++ b/HLTriggerOffline/Egamma/src/EmDQM.cc
@@ -1442,6 +1442,7 @@ edm::ParameterSet EmDQM::makePSetForEgammaGenericFilter(const std::string &modul
       inputType == "EgammaHLTGsfTrackVarProducer" ||               // GSF track deta and dphi filter
       inputType == "EgammaHLTBcHcalIsolationProducersRegional" ||  // HCAL isolation and
                                                                    // HE
+      inputType == "EgammaHLTHcalVarProducerFromRecHit" ||         // HCAL isolation (H/E) from RecHits
       inputType == "EgammaHLTEcalPFClusterIsolationProducer" ||    // ECAL PF isolation
                                                                    // filter
       inputType == "EgammaHLTHcalPFClusterIsolationProducer" ||    // HCAL PF isolation
@@ -1530,6 +1531,7 @@ edm::ParameterSet EmDQM::makePSetForEgammaGenericQuadraticFilter(const std::stri
       inputType == "EgammaHLTHcalIsolationProducersRegional" ||      // HCAL isolation and HE
       inputType == "EgammaHLTBcHcalIsolationProducersRegional" ||    // HCAL isolation and
                                                                      // HE
+      inputType == "EgammaHLTHcalVarProducerFromRecHit" ||           // HCAL isolation (H/E) from RecHits
       inputType == "EgammaHLTEcalPFClusterIsolationProducer" ||      // ECAL PF isolation
                                                                      // filter
       inputType == "EgammaHLTHcalPFClusterIsolationProducer" ||      // HCAL PF isolation

missirol · 2022-05-01T09:47:56Z

+hlt

differences in outputs of Run-3 wfs are in line with changes to the HLT GRun menu.
HLT menu development for 12_3_X [12_4_X] #37754 (comment) requires a follow-up related to the HLT-EGM Validation sequences.

cmsbuild · 2022-05-01T09:48:10Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2022-05-01T10:17:01Z

The large reduction of DQM outputs in HLT/HLTEgammaValidation is related to the changes in this PR to E/Gamma triggers.

One of the ED producers used in those triggers is changed by this PR (EgammaHLTBcHcalIsolationProducersRegional replaced by EgammaHLTHcalVarProducerFromRecHit), and the DQM/Validation client does not recognize the new modules, because it hard-codes the types of the acceptable ED producers (see here, for example).

This patch [*] is a starting point to fix the issue, but this task is left to EGM and/or DQM experts.

In my opinion, this should not delay this PR.

Attn: @cms-sw/dqm-l2 @cms-sw/egamma-pog-l2

[*]

diff --git a/HLTriggerOffline/Egamma/src/EmDQM.cc b/HLTriggerOffline/Egamma/src/EmDQM.cc
index 0a1fb605a98..43c0e8f9f17 100644
--- a/HLTriggerOffline/Egamma/src/EmDQM.cc
+++ b/HLTriggerOffline/Egamma/src/EmDQM.cc
@@ -1442,6 +1442,7 @@ edm::ParameterSet EmDQM::makePSetForEgammaGenericFilter(const std::string &modul
       inputType == "EgammaHLTGsfTrackVarProducer" ||               // GSF track deta and dphi filter
       inputType == "EgammaHLTBcHcalIsolationProducersRegional" ||  // HCAL isolation and
                                                                    // HE
+      inputType == "EgammaHLTHcalVarProducerFromRecHit" ||         // HCAL isolation (H/E) from RecHits
       inputType == "EgammaHLTEcalPFClusterIsolationProducer" ||    // ECAL PF isolation
                                                                    // filter
       inputType == "EgammaHLTHcalPFClusterIsolationProducer" ||    // HCAL PF isolation
@@ -1530,6 +1531,7 @@ edm::ParameterSet EmDQM::makePSetForEgammaGenericQuadraticFilter(const std::stri
       inputType == "EgammaHLTHcalIsolationProducersRegional" ||      // HCAL isolation and HE
       inputType == "EgammaHLTBcHcalIsolationProducersRegional" ||    // HCAL isolation and
                                                                      // HE
+      inputType == "EgammaHLTHcalVarProducerFromRecHit" ||           // HCAL isolation (H/E) from RecHits
       inputType == "EgammaHLTEcalPFClusterIsolationProducer" ||      // ECAL PF isolation
                                                                      // filter
       inputType == "EgammaHLTHcalPFClusterIsolationProducer" ||      // HCAL PF isolation

@missirol the fix that you propose is quite logical and straightforward.

I would rather be in favour to include it here, and in the backport PR as well, so that (1) the new menu and the needed dqm update can go on together, and (2) the comparisons in the PR tests remain meaningful.

(By the way, since you are there please also move up the commented "HE" and "filter" in the lines above and below, for a better reading; if the code-format checks split the line again, you can replace "isolation" with "isol" (e.g.), which is still a nicer read than having the last word of the line split)

missirol · 2022-05-01T11:40:43Z

I would rather be in favour to include it here [..]

Okay, done. References in the DQM module to plugin types that don't exist anymore have been removed as well.

cmsbuild · 2022-05-01T11:45:02Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37754/29636

This PR adds an extra 832KB to repository

cmsbuild · 2022-05-01T11:45:24Z

Pull request #37754 was updated. @Martin-Grunewald, @emanueleusai, @ahmad3213, @cmsbuild, @missirol, @jfernan2, @pmandrik, @micsucmed, @rvenditti can you please check and sign again.

missirol · 2022-05-01T11:45:33Z

please test

cmsbuild · 2022-05-01T16:18:22Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cbee91/24362/summary.html
COMMIT: 5e18d42
CMSSW: CMSSW_12_4_X_2022-05-01-0000/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/37754/24362/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3704128
DQMHistoTests: Total failures: 74
DQMHistoTests: Total nulls: 162
DQMHistoTests: Total successes: 3703870
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.162 KiB( 48 files compared)
DQMHistoSizes: changed ( 11634.0,... ): -0.027 KiB HLT/Filters
Checked 205 log files, 45 edm output root files, 49 DQM output files
TriggerResults: found differences in 6 / 48 workflows

missirol · 2022-05-01T16:29:59Z

Based on the outputs of the PR tests, the update of the DQM module worked as expected (no unexpected differences).

@cms-sw/dqm-l2 , this PR now requires your signature.

missirol · 2022-05-01T16:30:10Z

+hlt

differences in outputs of Run-3 wfs are in line with changes to the HLT GRun menu.
includes a related update to a DQM/Validation plugin.

missirol · 2022-05-02T10:03:53Z

@cms-sw/dqm-l2 , please consider reviewing this PR (it contains a simple update to 1 DQM module).

There are more updates to the HLT menu in the pipeline, so it would be good to integrate this PR and test it in IBs sooner rather than later.

emanueleusai · 2022-05-02T13:56:13Z

+1

cmsbuild · 2022-05-02T13:56:34Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2022-05-02T14:02:41Z

+1

HLT menu development for 12_3_X (branch: 12_4_X)

8c48380

cmsbuild added this to the CMSSW_12_4_X milestone Apr 30, 2022

cmsbuild added code-checks-pending hlt-pending orp-pending pending-signatures tests-pending labels Apr 30, 2022

missirol mentioned this pull request Apr 30, 2022

HLT menu development for 12_3_X #37755

Merged

cmsbuild added code-checks-approved and removed code-checks-pending labels Apr 30, 2022

cmsbuild added tests-started and removed tests-pending labels Apr 30, 2022

cmsbuild added tests-approved and removed tests-started labels Apr 30, 2022

cmsbuild added fully-signed hlt-approved and removed hlt-pending pending-signatures labels May 1, 2022

update of EmDQM (EGM-validation plugin)

5e18d42

cmsbuild removed hlt-approved fully-signed tests-approved labels May 1, 2022

cmsbuild added the tests-pending label May 1, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels May 1, 2022

cmsbuild added tests-started and removed tests-pending labels May 1, 2022

cmsbuild added tests-approved and removed tests-started labels May 1, 2022

cmsbuild added hlt-approved and removed hlt-pending labels May 1, 2022

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels May 2, 2022

cmsbuild added orp-approved and removed orp-pending labels May 2, 2022

cmsbuild merged commit d8a5096 into cms-sw:master May 2, 2022

missirol deleted the HLTdevelopment123X_for124X branch May 2, 2022 14:03

This was referenced May 3, 2022

Added IOV synchronization callbacks to ActivityRegistry #37771

Merged

fix use after scope in DMRChecker::bookResidualsHistogram #37796

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HLT menu development for `12_3_X` [`12_4_X`] #37754

HLT menu development for `12_3_X` [`12_4_X`] #37754

missirol commented Apr 30, 2022

cmsbuild commented Apr 30, 2022

cmsbuild commented Apr 30, 2022

missirol commented Apr 30, 2022

cmsbuild commented Apr 30, 2022

missirol commented May 1, 2022

missirol commented May 1, 2022

cmsbuild commented May 1, 2022

perrotta commented May 1, 2022 •

edited

missirol commented May 1, 2022

cmsbuild commented May 1, 2022

cmsbuild commented May 1, 2022

missirol commented May 1, 2022

cmsbuild commented May 1, 2022

missirol commented May 1, 2022

missirol commented May 1, 2022

missirol commented May 2, 2022

emanueleusai commented May 2, 2022

cmsbuild commented May 2, 2022

perrotta commented May 2, 2022

HLT menu development for 12_3_X [12_4_X] #37754

HLT menu development for 12_3_X [12_4_X] #37754

Conversation

missirol commented Apr 30, 2022

PR description:

PR validation:

If this PR is a backport, please specify the original PR and why you need to backport that PR:

cmsbuild commented Apr 30, 2022

cmsbuild commented Apr 30, 2022

missirol commented Apr 30, 2022

cmsbuild commented Apr 30, 2022

Comparison Summary

missirol commented May 1, 2022

missirol commented May 1, 2022

cmsbuild commented May 1, 2022

perrotta commented May 1, 2022 • edited

missirol commented May 1, 2022

cmsbuild commented May 1, 2022

cmsbuild commented May 1, 2022

missirol commented May 1, 2022

cmsbuild commented May 1, 2022

Comparison Summary

missirol commented May 1, 2022

missirol commented May 1, 2022

missirol commented May 2, 2022

emanueleusai commented May 2, 2022

cmsbuild commented May 2, 2022

perrotta commented May 2, 2022

HLT menu development for `12_3_X` [`12_4_X`] #37754

HLT menu development for `12_3_X` [`12_4_X`] #37754

perrotta commented May 1, 2022 •

edited