Add DRnR plots to PixelPhase1 DQM #34516

arossi83 · 2021-07-16T11:40:09Z

PR description:

The main target of this PR is the addition of DRnR plots to the PixelPhase1 DQM. Everything has been packed inside a dedicated harvester and also other residuals plots creation has been moved here for consistency

PR validation:

Running wf 136.892 and standalone pixel online DQM client

if this PR is a backport please specify the original PR and why you need to backport that PR:

not a backport and no backport forseen

arossi83 · 2021-07-16T11:41:13Z

@mmusich @sroychow @dmeuser FYI

mmusich · 2021-07-16T11:41:41Z

DQM/SiPixelPhase1Track/plugins/SiPixelPhase1ResidualsExtra.cc

+      }
+      for (int j : {1, 2, 3, 4, 6, 7, 8, 9}) {
+	if (me_x->getBinEntries(j, i) < minHits_){
+	  std::cout<<"Barrel Bin "<<j<<","<<i<<" - Entries "<<me2_x->getBinEntries(j, i)<<std::endl;


no cout please

mmusich · 2021-07-16T11:41:53Z

DQM/SiPixelPhase1Track/plugins/SiPixelPhase1ResidualsExtra.cc

+	  me2_y->setBinEntries(me2_y->getBin(j, i),1);
+	}
+	else{
+	  std::cout<<"Endcap Bin "<<j<<","<<i<<" - Entries "<<me2_x->getBinEntries(j, i)<<std::endl;


no cout please

mmusich · 2021-07-16T11:42:06Z

DQM/SiPixelPhase1Track/plugins/SiPixelPhase1TrackResiduals.cc

@@ -81,6 +83,14 @@ namespace {
        histo[RESIDUAL_X].fill(it.resXprime, id, &iEvent);
        histo[RESIDUAL_Y].fill(it.resYprime, id, &iEvent);

+	if (it.resXprimeErr!=0 && it.resXprimeErr!=0){
+	  std::cout<<it.resXprime<<"  -  "<<it.resXprimeErr<<"  -  "<<it.resXprime/it.resXprimeErr<<std::endl;


no cout please

cmsbuild · 2021-07-16T11:45:58Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34516/24018

This PR adds an extra 24KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File DQM/SiPixelPhase1Config/python/SiPixelPhase1OfflineDQM_harvesting_cff.py modified in PR(s): DQM Pixel Phase 1: Several new Efficiency trend plots #34384
- File DQM/SiPixelPhase1Config/python/SiPixelPhase1OfflineDQM_source_cff.py modified in PR(s): DQM Pixel Phase 1: Several new Efficiency trend plots #34384

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34516/24018/code-format.patch
e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34516/24018/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

cmsbuild · 2021-07-16T12:01:55Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34516/24020

This PR adds an extra 24KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File DQM/SiPixelPhase1Config/python/SiPixelPhase1OfflineDQM_harvesting_cff.py modified in PR(s): DQM Pixel Phase 1: Several new Efficiency trend plots #34384
- File DQM/SiPixelPhase1Config/python/SiPixelPhase1OfflineDQM_source_cff.py modified in PR(s): DQM Pixel Phase 1: Several new Efficiency trend plots #34384

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34516/24020/code-format.patch
e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34516/24020/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

mmusich · 2021-07-16T12:11:42Z

code-checks

cmsbuild · 2021-07-16T12:18:39Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34516/24021

This PR adds an extra 28KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File DQM/SiPixelPhase1Config/python/SiPixelPhase1OfflineDQM_harvesting_cff.py modified in PR(s): DQM Pixel Phase 1: Several new Efficiency trend plots #34384
- File DQM/SiPixelPhase1Config/python/SiPixelPhase1OfflineDQM_source_cff.py modified in PR(s): DQM Pixel Phase 1: Several new Efficiency trend plots #34384

cmsbuild · 2021-07-16T12:18:58Z

A new Pull Request was created by @arossi83 (Alessandro Rossi) for master.

It involves the following packages:

DQM/SiPixelPhase1Config (dqm)
DQM/SiPixelPhase1Summary (dqm)
DQM/SiPixelPhase1Track (dqm)

@andrius-k, @kmaeshima, @ErnestaP, @ahmad3213, @cmsbuild, @jfernan2, @rvenditti can you please review it and eventually sign? Thanks.
@arossi83, @hdelanno, @sroychow, @fioriNTU, @jandrea, @idebruyn, @threus this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy, @perrotta you are the release manager for this.

cms-bot commands are listed here

mmusich · 2021-07-16T12:19:51Z

DQM/SiPixelPhase1Track/interface/SiPixelPhase1ResidualsExtra.h

+#include "FWCore/Framework/interface/EventSetup.h"
+#include "DataFormats/Common/interface/Handle.h"
+#include "FWCore/Framework/interface/LuminosityBlock.h"
+#include "FWCore/Framework/interface/ESHandle.h"


that's not needed.

mmusich · 2021-07-16T12:20:04Z

DQM/SiPixelPhase1Track/interface/SiPixelPhase1ResidualsExtra.h

+  ~SiPixelPhase1ResidualsExtra() override;
+
+protected:
+  // BeginJob


remove commented code

mmusich · 2021-07-16T12:21:38Z

DQM/SiPixelPhase1Track/plugins/SiPixelPhase1TrackResiduals.cc

@@ -81,6 +92,13 @@ namespace {
        histo[RESIDUAL_X].fill(it.resXprime, id, &iEvent);
        histo[RESIDUAL_Y].fill(it.resYprime, id, &iEvent);

+        if (it.resXprimeErr != 0 && it.resXprimeErr != 0) {


I can't understand the logic here, isn't this duplicated twice?

@arossi83 can you please clarify this, or remove the duplication?

Sorry, I just realize where the duplication is, obviously is a typo. I just fixed it!

mmusich · 2021-07-16T12:24:35Z

DQM/SiPixelPhase1Track/plugins/SiPixelPhase1TrackResiduals.cc

+        if (it.resXprimeErr != 0 && it.resXprimeErr != 0) {
+          histo[NormRes_X].fill(it.resXprime / it.resXprimeErr, id, &iEvent);
+          histo[NormRes_Y].fill(it.resYprime / it.resYprimeErr, id, &iEvent);
+          histo[DRNR_X].fill(it.resXprime / it.resXprimeErr, id, &iEvent);


how is DRNR_X different from NormRes_X ?

No difference at this level, the differences are introduced in the harvesting step. I separate the plots here in order to have a proper histogram name on the DQMIO root file for DRnR. Summarizing: here DRNR_X is used to fill a 2D Profile which will show NormRes mean value for each bin, in the harvesting the content of this 2D profile is changed replacing the Mean value with RMS for each bin(module).

Can't the histogram be booked directly in the harvesting step?

In principle we can have the MEs booked and filled in the step3 but to do that we need to implement a method to evaluate RMS of single module in a running way (storing average, average of the square and number of entries) and be sure to be multithread and Concurrent LS safe. In the way things are implemented the 2DProfile ME take care of everything and the "cost" is to have an apparently duplicated entry on step3.

cmsbuild · 2021-07-20T07:56:52Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34516/24093

This PR adds an extra 12KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File DQM/SiPixelPhase1Config/python/SiPixelPhase1OfflineDQM_harvesting_cff.py modified in PR(s): DQM Pixel Phase 1: Several new Efficiency trend plots #34384
- File DQM/SiPixelPhase1Config/python/SiPixelPhase1OfflineDQM_source_cff.py modified in PR(s): DQM Pixel Phase 1: Several new Efficiency trend plots #34384

cmsbuild · 2021-07-20T07:57:13Z

Pull request #34516 was updated. @andrius-k, @kmaeshima, @ErnestaP, @ahmad3213, @cmsbuild, @jfernan2, @rvenditti can you please check and sign again.

mmusich · 2021-07-20T08:34:43Z

@cmsbuild, please test

cmsbuild · 2021-07-21T01:36:01Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9f6227/17011/summary.html
COMMIT: 80973f9
CMSSW: CMSSW_12_0_X_2021-07-19-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34516/17011/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 39
DQMHistoTests: Total histograms compared: 2995596
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2995573
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 1646.008 KiB( 38 files compared)
DQMHistoSizes: changed ( 10024.0,... ): 117.572 KiB PixelPhase1/Tracks
Checked 165 log files, 37 edm output root files, 39 DQM output files
TriggerResults: no differences found

jfernan2 · 2021-07-21T07:30:38Z

+1

cmsbuild · 2021-07-21T07:31:04Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy, @perrotta (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2021-07-21T09:32:19Z

+1

Alessandro Rossi and others added 2 commits July 16, 2021 13:28

Adding DRnR distribution to PixelPhase1 DQM

fb01d11

Merge branch 'cms-sw:master' into PixelDRnR_v2

c8c147c

cmsbuild added this to the CMSSW_12_0_X milestone Jul 16, 2021

cmsbuild added code-checks-pending dqm-pending orp-pending pending-signatures tests-pending labels Jul 16, 2021

mmusich reviewed Jul 16, 2021

View reviewed changes

cmsbuild added code-checks-rejected and removed code-checks-pending labels Jul 16, 2021

Remove some leftover cout

f6c6da4

cmsbuild added code-checks-pending and removed code-checks-rejected labels Jul 16, 2021

Code formatting

496a495

cmsbuild added code-checks-rejected and removed code-checks-pending labels Jul 16, 2021

cmsbuild added code-checks-pending and removed code-checks-rejected labels Jul 16, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 16, 2021

mmusich reviewed Jul 16, 2021

View reviewed changes

cmsbuild added dqm-pending pending-signatures labels Jul 20, 2021

Fixing a wrong selection rule

80973f9

cmsbuild added code-checks-pending tests-pending and removed code-checks-approved tests-approved labels Jul 20, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 20, 2021

cmsbuild added tests-started and removed tests-pending labels Jul 20, 2021

cmsbuild added tests-approved and removed tests-started labels Jul 21, 2021

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels Jul 21, 2021

cmsbuild added orp-approved and removed orp-pending labels Jul 21, 2021

cmsbuild merged commit c6d0025 into cms-sw:master Jul 21, 2021

This was referenced Jul 21, 2021

scram-tools repo to provide scram toolfile definitions cms-sw/cmsdist#7147

Merged

support global::OutputModule<ExternalWork> #34579

Merged

Remove EventSetup use from PdtEntry #34581

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add DRnR plots to PixelPhase1 DQM #34516

Add DRnR plots to PixelPhase1 DQM #34516

arossi83 commented Jul 16, 2021

arossi83 commented Jul 16, 2021

mmusich Jul 16, 2021

arossi83 Jul 16, 2021

mmusich Jul 16, 2021

arossi83 Jul 16, 2021

mmusich Jul 16, 2021

arossi83 Jul 16, 2021

cmsbuild commented Jul 16, 2021

cmsbuild commented Jul 16, 2021

mmusich commented Jul 16, 2021

cmsbuild commented Jul 16, 2021

cmsbuild commented Jul 16, 2021

mmusich Jul 16, 2021

arossi83 Jul 19, 2021 •

edited

mmusich Jul 16, 2021

arossi83 Jul 19, 2021

mmusich Jul 16, 2021

mmusich Jul 20, 2021

arossi83 Jul 20, 2021

mmusich Jul 16, 2021

arossi83 Jul 19, 2021

mmusich Jul 19, 2021

arossi83 Jul 19, 2021 •

edited

cmsbuild commented Jul 20, 2021

cmsbuild commented Jul 20, 2021

mmusich commented Jul 20, 2021

cmsbuild commented Jul 21, 2021

jfernan2 commented Jul 21, 2021

cmsbuild commented Jul 21, 2021

qliphy commented Jul 21, 2021

Add DRnR plots to PixelPhase1 DQM #34516

Add DRnR plots to PixelPhase1 DQM #34516

Conversation

arossi83 commented Jul 16, 2021

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

arossi83 commented Jul 16, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cmsbuild commented Jul 16, 2021

cmsbuild commented Jul 16, 2021

mmusich commented Jul 16, 2021

cmsbuild commented Jul 16, 2021

cmsbuild commented Jul 16, 2021

Choose a reason for hiding this comment

arossi83 Jul 19, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arossi83 Jul 19, 2021 • edited

Choose a reason for hiding this comment

cmsbuild commented Jul 20, 2021

cmsbuild commented Jul 20, 2021

mmusich commented Jul 20, 2021

cmsbuild commented Jul 21, 2021

Comparison Summary

jfernan2 commented Jul 21, 2021

cmsbuild commented Jul 21, 2021

qliphy commented Jul 21, 2021

arossi83 Jul 19, 2021 •

edited

arossi83 Jul 19, 2021 •

edited