TkDQM: Add standalone monitoring modules for OT Rechits and new resolution plots for ITRechits #33482

sroychow · 2021-04-20T23:35:03Z

PR description:

This PR adds

Modules for OT Rechit monitoring and validation. These can be run only with a standalone config - DQM/SiTrackerPhase2/test/dqmstep_phase2tk_cfg.py. The input to the config should be a GEN-SIM-RECO file. It runs the IT and OT rechit producers together with the DQM & Validation step.
Fixes an axis range in OT cluster validation module.
For IT rechits, plots of cluster res vs cluster size(in X & Y) are added.
For IT rechits, a scatter plot of res Y vs res X is added

PR validation:

Validated with wf 23234.0, 34634.0.

cc @gabriel-rmrz @mmusich @tsusa @emiglior

cmsbuild · 2021-04-20T23:41:59Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33482/22208

This PR adds an extra 32KB to repository

cmsbuild · 2021-04-20T23:42:20Z

A new Pull Request was created by @sroychow (Suvankar Roy Chowdhury) for master.

It involves the following packages:

DQM/SiTrackerPhase2
Validation/SiTrackerPhase2V

@andrius-k, @kmaeshima, @ErnestaP, @ahmad3213, @cmsbuild, @jfernan2, @rvenditti can you please review it and eventually sign? Thanks.
@arossi83, @sroychow this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

mmusich · 2021-04-21T05:11:12Z

@cmsbuild, please test

mmusich

@sroychow thanks a lot for this!
I left few mostly technical comments below about modernized event and ES data access.

DQM/SiTrackerPhase2/plugins/Phase2OTMonitorRecHit.cc

DQM/SiTrackerPhase2/test/dqmstep_phase2tk_cfg.py

DQM/SiTrackerPhase2/test/harvestingstep_phase2tk_cfg.py

Validation/SiTrackerPhase2V/plugins/Phase2OTValidateRecHit.cc

DQM/SiTrackerPhase2/plugins/Phase2OTMonitorRecHit.cc

Validation/SiTrackerPhase2V/plugins/Phase2OTValidateRecHit.cc

cmsbuild · 2021-04-21T07:48:53Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-dfd2d0/14373/summary.html
COMMIT: 1611ce3
CMSSW: CMSSW_12_0_X_2021-04-20-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33482/14373/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 38
DQMHistoTests: Total histograms compared: 2877046
DQMHistoTests: Total failures: 7
DQMHistoTests: Total nulls: 148
DQMHistoTests: Total successes: 2876869
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 3668.408 KiB( 37 files compared)
DQMHistoSizes: changed ( 23234.0,... ): 375.258 KiB TrackerPhase2ITRecHitV/EndCap_Side1
DQMHistoSizes: changed ( 23234.0,... ): 375.258 KiB TrackerPhase2ITRecHitV/EndCap_Side2
DQMHistoSizes: changed ( 23234.0,... ): 166.586 KiB TrackerPhase2ITRecHitV/Barrel
Checked 160 log files, 37 edm output root files, 38 DQM output files
TriggerResults: no differences found

sroychow · 2021-04-21T11:15:36Z

@mmusich thanks for your comments. Addressed the comments and updated the other modules to the recommended syntax of getting Handle and ES data.

mmusich · 2021-04-21T11:16:39Z

Addressed the comments and updated the other modules to the recommended syntax of getting Handle and ES data.

thank you!

cmsbuild · 2021-04-21T11:21:14Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33482/22216

This PR adds an extra 52KB to repository

cmsbuild · 2021-04-21T20:55:27Z

Pull request #33482 was updated. @andrius-k, @kmaeshima, @ErnestaP, @ahmad3213, @cmsbuild, @jfernan2, @rvenditti can you please check and sign again.

mmusich · 2021-04-21T21:08:17Z

@sroychow there's still a couple of clang-tidy warnings: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-dfd2d0/14391/llvm-analysis/runCMSClangTidyChecks.log.
Also since you are at it can you squash the last 8 commits?

bux fix in accessing handles simplify product access update procduct access to recommended format formatting fix fix bug in reading digisimlinks update handle access

cmsbuild · 2021-04-21T21:51:28Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33482/22228

This PR adds an extra 56KB to repository

cmsbuild · 2021-04-21T21:51:48Z

Pull request #33482 was updated. @andrius-k, @kmaeshima, @ErnestaP, @ahmad3213, @cmsbuild, @jfernan2, @rvenditti can you please check and sign again.

mmusich · 2021-04-21T21:57:35Z

@cmsbuild, please test

cmsbuild · 2021-04-22T02:18:56Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-dfd2d0/14401/summary.html
COMMIT: c0c9c58
CMSSW: CMSSW_12_0_X_2021-04-21-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33482/14401/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 7 differences found in the comparisons
DQMHistoTests: Total files compared: 38
DQMHistoTests: Total histograms compared: 2877046
DQMHistoTests: Total failures: 13
DQMHistoTests: Total nulls: 148
DQMHistoTests: Total successes: 2876863
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 3668.408 KiB( 37 files compared)
DQMHistoSizes: changed ( 23234.0,... ): 375.258 KiB TrackerPhase2ITRecHitV/EndCap_Side1
DQMHistoSizes: changed ( 23234.0,... ): 375.258 KiB TrackerPhase2ITRecHitV/EndCap_Side2
DQMHistoSizes: changed ( 23234.0,... ): 166.586 KiB TrackerPhase2ITRecHitV/Barrel
Checked 160 log files, 37 edm output root files, 38 DQM output files
TriggerResults: no differences found

jfernan2 · 2021-04-23T12:42:09Z

+1

cmsbuild · 2021-04-23T12:42:31Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2021-04-24T07:20:37Z

+1

davidlange6 · 2021-04-27T09:49:32Z

hi @sroychow @cms-sw/dqm-l2 - it looks like this PR adds a dependency on Validation to DQM/SiTrackerPhase2. Could that be fixed? Possibly by moving stuff that needs MC information to Validation. Thanks!

sroychow · 2021-04-28T06:07:06Z

@davidlange6 Thanks! should be addressed by #33557. Please have a look.

sroychow added 3 commits April 21, 2021 01:01

add new resolution plots and fix range

436a934

add standalone modules for monitoring and validation phase2OTrechits

45e79cc

code format

1611ce3

cmsbuild added this to the CMSSW_12_0_X milestone Apr 20, 2021

cmsbuild added code-checks-pending dqm-pending orp-pending pending-signatures tests-pending labels Apr 20, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Apr 20, 2021

cmsbuild added tests-started and removed tests-pending labels Apr 21, 2021

mmusich reviewed Apr 21, 2021

View reviewed changes

cmsbuild added tests-approved and removed tests-started labels Apr 21, 2021

address comments on the PR

bc8b5d4

cmsbuild added code-checks-pending tests-pending and removed code-checks-approved tests-approved labels Apr 21, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Apr 21, 2021

cmsbuild added the code-checks-approved label Apr 21, 2021

update other modules to latest syntax of gethandle and esdata

c0c9c58

bux fix in accessing handles simplify product access update procduct access to recommended format formatting fix fix bug in reading digisimlinks update handle access

sroychow force-pushed the phase2OTdqm branch from 867ea6f to c0c9c58 Compare April 21, 2021 21:45

cmsbuild added code-checks-pending and removed code-checks-approved labels Apr 21, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Apr 21, 2021

cmsbuild added tests-started and removed tests-pending labels Apr 21, 2021

cmsbuild added tests-approved and removed tests-started labels Apr 22, 2021

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels Apr 23, 2021

cmsbuild added orp-approved and removed orp-pending labels Apr 24, 2021

cmsbuild merged commit 0c38ca4 into cms-sw:master Apr 24, 2021

sroychow mentioned this pull request Apr 28, 2021

TkDQM phase2: Remove validation header from DQM package #33557

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TkDQM: Add standalone monitoring modules for OT Rechits and new resolution plots for ITRechits #33482

TkDQM: Add standalone monitoring modules for OT Rechits and new resolution plots for ITRechits #33482

sroychow commented Apr 20, 2021

cmsbuild commented Apr 20, 2021

cmsbuild commented Apr 20, 2021

mmusich commented Apr 21, 2021

mmusich left a comment

cmsbuild commented Apr 21, 2021

sroychow commented Apr 21, 2021

mmusich commented Apr 21, 2021

cmsbuild commented Apr 21, 2021

cmsbuild commented Apr 21, 2021

mmusich commented Apr 21, 2021

cmsbuild commented Apr 21, 2021

cmsbuild commented Apr 21, 2021

mmusich commented Apr 21, 2021

cmsbuild commented Apr 22, 2021

jfernan2 commented Apr 23, 2021

cmsbuild commented Apr 23, 2021

qliphy commented Apr 24, 2021

davidlange6 commented Apr 27, 2021

sroychow commented Apr 28, 2021

TkDQM: Add standalone monitoring modules for OT Rechits and new resolution plots for ITRechits #33482

TkDQM: Add standalone monitoring modules for OT Rechits and new resolution plots for ITRechits #33482

Conversation

sroychow commented Apr 20, 2021

PR description:

PR validation:

cmsbuild commented Apr 20, 2021

cmsbuild commented Apr 20, 2021

mmusich commented Apr 21, 2021

mmusich left a comment

Choose a reason for hiding this comment

cmsbuild commented Apr 21, 2021

Comparison Summary

sroychow commented Apr 21, 2021

mmusich commented Apr 21, 2021

cmsbuild commented Apr 21, 2021

cmsbuild commented Apr 21, 2021

mmusich commented Apr 21, 2021

cmsbuild commented Apr 21, 2021

cmsbuild commented Apr 21, 2021

mmusich commented Apr 21, 2021

cmsbuild commented Apr 22, 2021

Comparison Summary

jfernan2 commented Apr 23, 2021

cmsbuild commented Apr 23, 2021

qliphy commented Apr 24, 2021

davidlange6 commented Apr 27, 2021

sroychow commented Apr 28, 2021