updates to Top-HLT sequence in DQMOffline (backport of #26540)

[missirol]

PR description:

updates to the Top-HLT sequence in DQMOffline/Trigger/ to use the modifiers Run2_HLTconditions_201* (introduced in #26355), to select the correct HLT paths (and b-tagging algorithm) in different eras (Run2_2016, Run2_2017, Run2_2018).

It includes one bugfix to the plot bjetPhi_* of the TopMonitor plugin (differences in the comparisons are expected only for these histograms).

PR validation:

The code has been tested with runTheMatrix.py -l 10024.0 and by running the offline-DQM on 2017, 2018 datasets.

if this PR is a backport please specify the original PR:

this is a backport of #26540

[cmsbuild]

A new Pull Request was created by @missirol (Marino Missiroli) for CMSSW_10_6_X.

It involves the following packages:

Configuration/Eras
DQMOffline/Trigger

@andrius-k, @kmaeshima, @schneiml, @cmsbuild, @franzoni, @jfernan2, @fioriNTU, @fabiocos, @davidlange6 can you please review it and eventually sign? Thanks.
@battibass, @makortel, @mtosi, @jhgoh, @Martin-Grunewald, @calderona, @HuguesBrun, @trocino, @folguera, @rociovilar this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

[jfernan2]

please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/383/console Started: 2019/05/21 18:29

[cmsbuild]

+1
Tested at: 883cd8e
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-620dea/383/summary.html

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
844c744
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-26861/383/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-26861/383/git-merge-result

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-620dea/383/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 4 differences found in the comparisons
• DQMHistoTests: Total files compared: 33
• DQMHistoTests: Total histograms compared: 3207060
• DQMHistoTests: Total failures: 96
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3206630
• DQMHistoTests: Total skipped: 334
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -6397.848 KiB( 32 files compared)
• DQMHistoSizes: changed ( 136.731,... ): -1599.462 KiB HLT/TOP
• Checked 137 log files, 14 edm output root files, 33 DQM output files

[missirol]

the only differences I could spot in the comparisons are related to the fix of the bjetPhi_* plots in the TopMonitor plugin (e.g. here), plus some skipped comparisons which seem unrelated to this PR (e.g. here)

[jfernan2]

@missirol @sscruz This PR is removing ~1250 histograms from HLT/TOP in baseline in some of the workflows, the original PR too. Can you confirm this is expected?
https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_10_6_X_2019-05-21-1100+620dea/31815/dqm-histo-comparison-summary.html

[missirol]

Hi @jfernan2 , sorry for not having mentioned this.

We expect that with this PR several histograms are removed (wrt baseline) only when running with --era Run2_2016; the affected histograms are related to trigger paths that were not in the 2016 HLT menu (without the PR, those plots would just be empty, at least the numerators and efficiency plots); just for reference, the change comes from here

We checked the affected workflows (136.731, 1306.0, 1330.0, 25202.0) and they all use --era Run2_2016, and the DQM plots affected are the ones we expect. @sscruz

[jfernan2]

+1

[missirol]

Hi, I was just wondering if there are any outstanding issues with this backport, or the review could proceed?

[fabiocos]

@missirol backports to production cycles are normally integrated after validation in master, and anyway when a build is planned

[fabiocos]

@missirol BTW I understand that this backport is needed for the validation of the UL processing, am I correct?

[missirol]

@fabiocos yes, correct (these are the changes that were used by Top to provide feedback on the validation of UL-HLT17 RelVal samples in 10_6_0_pre2, for example here). Thanks to the modifiers, the HLT-Top DQM content will be the intended one for all 3 years in UL samples.

[fabiocos]

+operations

[fabiocos]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_10_6_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_11_0_X is complete. This pull request will be automatically merged.