Adding Online-Vs-Offline Track-level monitoring to DQM for BTV@HLT #30366

johnalison · 2020-06-25T11:42:23Z

PR description:

Adding Relative Online-Offline Track Efficiencies and Fake rates to the BTVHLTOffline DQM Monitoring

This commit:

Adds 16 histograms in BTVHLTOfflineSource: 2 (efficiencies and fake rates) x 4 variables (pt / eta / ip3d / ip3dSig) x 2 (numerator / denominator)
Adds logic to retrieve online and offline tracks and fill numerator/denominator hists
Adds Harvesting configuration to make the ratios

To the products needed in the step2 input I add the following output lines to HLTDebugFEVT
in HLTrigger/Configuration/python/HLTrigger_EventContent_cff.py
(This change in NOT included in this commit)

   'keep *_hltImpactParameterTagInfos_*_*',

   'keep *_hltDeepBLifetimeTagInfosPF_*_*',

   'keep *_hltDeepSecondaryVertexTagInfosPF_*_*',

   'keep *_hltInclusiveSecondaryVertexFinderTagInfos_*_*',

   'keep *_hltMergedTracksForBTag_*_*',

   'keep *_hltFastPixelBLifetimeL3Associator_*_*',

PR validation:

Standard runTheMaxtrix and look at the DQM plots produced

…e BTVHLTOffline DQM Monitoring This commit: - Adds 16 histograms in BTVHLTOfflineSource: 2 (efficencies and fake rates) x 4 variables (pt / eta / ip3d / ip3dSig) x 2 (numerator / denominator) - Adds logic to retrieve online and offline tracks and fill numerator/denominator hists - Adds Harvesting configuration to make the ratios To the products needed in the step2 input I add the following output lines to HLTDebugFEVT in HLTrigger/Configuration/python/HLTrigger_EventContent_cff.py (This change in NOT included in this commit) + 'keep *_hltImpactParameterTagInfos_*_*', + 'keep *_hltDeepBLifetimeTagInfosPF_*_*', + 'keep *_hltDeepSecondaryVertexTagInfosPF_*_*', + 'keep *_hltInclusiveSecondaryVertexFinderTagInfos_*_*', + 'keep *_hltMergedTracksForBTag_*_*', + 'keep *_hltFastPixelBLifetimeL3Associator_*_*',

cmsbuild · 2020-06-25T11:42:49Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-06-25T12:19:29Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-30366/16423

This PR adds an extra 28KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File DQMOffline/Trigger/plugins/BTVHLTOfflineSource.cc modified in PR(s): [RFC] Test clang-tidy --checks modernize-use-emplace #29937

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-30366/16423/code-format.patch
e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-30366/16423/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

cmsbuild · 2020-06-25T12:34:21Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-06-25T13:00:07Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-30366/16425

This PR adds an extra 32KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File DQMOffline/Trigger/plugins/BTVHLTOfflineSource.cc modified in PR(s): [RFC] Test clang-tidy --checks modernize-use-emplace #29937

cmsbuild · 2020-06-25T13:00:31Z

A new Pull Request was created by @johnalison (John Alison) for master.

It involves the following packages:

DQMOffline/Trigger

@andrius-k, @kmaeshima, @schneiml, @cmsbuild, @jfernan2, @fioriNTU can you please review it and eventually sign? Thanks.
@missirol, @mtosi, @Fedespring, @calderona, @HuguesBrun, @jhgoh, @cericeci, @trocino, @folguera, @rociovilar this is something you requested to watch as well.
@silviodonato, @dpiparo you are the release manager for this.

cms-bot commands are listed here

mtosi · 2020-06-25T13:05:42Z

DQMOffline/Trigger/python/BTaggingMonitoring_Client_cff.py

+        "HLT/BTV/HLT_Mu8_TrkIsoVVL_Ele23_CaloIdL_TrackIdL_IsoVL_DZ_PFDiJet30_PFBtagDeepCSV_1p5*",
+        "HLT/BTV/HLT_Mu8_TrkIsoVVL_Ele23_CaloIdL_TrackIdL_IsoVL_DZ_CaloDiJet30_CaloBtagDeepCSV_1p5*",
+        "HLT/BTV/HLT_PFHT400_SixPFJet32_DoublePFBTagDeepCSV_2p94*",
+    ),


if you run this code on the HLTMonitor PD, we should make sure that not only the relevant collections are stored, but also the PD needs to be populated by the needed HLT paths

Hi Mia,

How can I check this ? (BTW What does PD mean here ? )

jfernan2 · 2020-06-25T14:06:26Z

@johnalison If you are developing code for BTV, could you please add yourself along with your git username in the comments to the corresponding e-group in
https://twiki.cern.ch/twiki/bin/viewauth/CMS/DQMContacts#B_Physics
for further contact?
Thanks

johnalison · 2020-06-25T14:19:34Z

@jfernan2 Done

jfernan2 · 2020-06-25T14:52:39Z

Thanks @johnalison

The comment by @mtosi is quite important: you should make sure the Primary Datasets you plan to use contain the HLT paths you want to monitor, and at present only GRun HLT Menu is used in Run3/Phase2 workflows/samples
Which workflows did you use to validate this PR with runTheMatrix?

Regarding which HLT Menu/conditions are being run please have a look at the following config files:
https://github.com/cms-sw/cmssw/blob/832adca964762d9156d6c9090ff7986b18c61041/Configuration/HLT/python/autoCondHLT.py
https://github.com/cms-sw/cmssw/blob/14873dd0c2c0ed7294dfdb4840155fa1ad865df8/Configuration/HLT/python/autoHLT.py

johnalison · 2020-06-25T16:01:42Z

@jfernan2 I subscribed to

HLT Validation Expert and
HLT DQM And Validation Code Developers

Looks like these are "Waiting approval" in the e-groups system

Waiting for approval

johnalison · 2020-06-25T16:08:36Z

@jfernan2 I have not thought at all about which Primary Datasets to use.

These changes only add histograms to the modules that are already being run. The HLT paths being monitored are unchanged, so I assume there is no problem here.
(Sorry this is the first time Im making a DQM change so maybe Im missing something ....)

I developed/tested the code with:

runTheMatrix.py -l 11634

jfernan2 · 2020-06-25T16:52:59Z

please test
Thanks @johnalison

cmsbuild · 2020-06-25T16:53:20Z

The tests are being triggered in jenkins.

CMSSW_11_2_X_2020-06-25-1100/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/7407/console Started: 2020/06/25 20:14

jfernan2 · 2020-07-21T09:37:36Z

please test with #30833
Crash seems unrelated

cmsbuild · 2020-07-21T09:38:03Z

The tests are being triggered in jenkins.
Tested with other pull request(s) #30833

CMSSW_11_2_X_2020-07-20-2300/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/8162/console Started: 2020/07/21 15:07

cmsbuild · 2020-07-21T14:26:51Z

+1
Tested at: 525c6de
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9aecf1/8162/summary.html
CMSSW: CMSSW_11_2_X_2020-07-20-2300
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-07-21T14:26:58Z

Comparison job queued.

cmsbuild · 2020-07-21T17:26:05Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9aecf1/8162/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 2 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2525994
DQMHistoTests: Total failures: 5
DQMHistoTests: Total nulls: 9
DQMHistoTests: Total successes: 2525933
DQMHistoTests: Total skipped: 47
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 44.66 KiB( 33 files compared)
DQMHistoSizes: changed ( 10224.0 ): 0.004 KiB MessageLogger/Warnings
DQMHistoSizes: changed ( 11634.0,... ): 22.328 KiB HLT/BTV
Checked 144 log files, 17 edm output root files, 34 DQM output files

jfernan2 · 2020-07-21T17:55:19Z

+1
Plots now correctly filled

cmsbuild · 2020-07-21T17:55:45Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo (and backports should be raised in the release meeting by the corresponding L2)

silviodonato · 2020-07-22T08:14:20Z

+1
(#30833 already merged)

cmsbuild added this to the CMSSW_11_2_X milestone Jun 25, 2020

cmsbuild added code-checks-pending comparison-pending dqm-pending orp-pending pending-signatures tests-pending labels Jun 25, 2020

cmsbuild added code-checks-rejected and removed code-checks-pending labels Jun 25, 2020

Fix code style

ae59ce8

cmsbuild added code-checks-pending and removed code-checks-rejected labels Jun 25, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Jun 25, 2020

mtosi reviewed Jun 25, 2020

View reviewed changes

cmsbuild added tests-started and removed tests-pending labels Jun 25, 2020

cmsbuild added comparison-notrun tests-rejected and removed tests-started labels Jul 20, 2020

cmsbuild added comparison-pending tests-pending and removed comparison-notrun tests-rejected labels Jul 21, 2020

cmsbuild added tests-started and removed tests-pending labels Jul 21, 2020

cmsbuild added tests-approved and removed tests-started labels Jul 21, 2020

cmsbuild added comparison-available and removed comparison-pending labels Jul 21, 2020

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels Jul 21, 2020

cmsbuild added orp-approved and removed orp-pending labels Jul 22, 2020

cmsbuild merged commit cd02974 into cms-sw:master Jul 22, 2020

cmsbuild mentioned this pull request Jul 22, 2020

drop wrong default params in TLorentzVector in TestCorrections.C #30872

Merged

jfernan2 mentioned this pull request Aug 9, 2020

Add track-to-track validation tool plugin. Added instances for general HLT track monitoring and in BTV specific paths #31040

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding Online-Vs-Offline Track-level monitoring to DQM for BTV@HLT #30366

Adding Online-Vs-Offline Track-level monitoring to DQM for BTV@HLT #30366

johnalison commented Jun 25, 2020

cmsbuild commented Jun 25, 2020

cmsbuild commented Jun 25, 2020

cmsbuild commented Jun 25, 2020

cmsbuild commented Jun 25, 2020

cmsbuild commented Jun 25, 2020

mtosi Jun 25, 2020

johnalison Jun 25, 2020

jfernan2 commented Jun 25, 2020

johnalison commented Jun 25, 2020

jfernan2 commented Jun 25, 2020 •

edited

johnalison commented Jun 25, 2020

johnalison commented Jun 25, 2020 •

edited

jfernan2 commented Jun 25, 2020

cmsbuild commented Jun 25, 2020 •

edited

jfernan2 commented Jul 21, 2020

cmsbuild commented Jul 21, 2020 •

edited

cmsbuild commented Jul 21, 2020

cmsbuild commented Jul 21, 2020

cmsbuild commented Jul 21, 2020

jfernan2 commented Jul 21, 2020

cmsbuild commented Jul 21, 2020

silviodonato commented Jul 22, 2020

Adding Online-Vs-Offline Track-level monitoring to DQM for BTV@HLT #30366

Adding Online-Vs-Offline Track-level monitoring to DQM for BTV@HLT #30366

Conversation

johnalison commented Jun 25, 2020

PR description:

PR validation:

cmsbuild commented Jun 25, 2020

cmsbuild commented Jun 25, 2020

cmsbuild commented Jun 25, 2020

cmsbuild commented Jun 25, 2020

cmsbuild commented Jun 25, 2020

mtosi Jun 25, 2020

Choose a reason for hiding this comment

johnalison Jun 25, 2020

Choose a reason for hiding this comment

jfernan2 commented Jun 25, 2020

johnalison commented Jun 25, 2020

jfernan2 commented Jun 25, 2020 • edited

johnalison commented Jun 25, 2020

Waiting for approval

johnalison commented Jun 25, 2020 • edited

jfernan2 commented Jun 25, 2020

cmsbuild commented Jun 25, 2020 • edited

jfernan2 commented Jul 21, 2020

cmsbuild commented Jul 21, 2020 • edited

cmsbuild commented Jul 21, 2020

cmsbuild commented Jul 21, 2020

cmsbuild commented Jul 21, 2020

jfernan2 commented Jul 21, 2020

cmsbuild commented Jul 21, 2020

silviodonato commented Jul 22, 2020

jfernan2 commented Jun 25, 2020 •

edited

johnalison commented Jun 25, 2020 •

edited

cmsbuild commented Jun 25, 2020 •

edited

cmsbuild commented Jul 21, 2020 •

edited