[12.2.X] run cosmics track refitter in order to fix to Strip Hit residuals #37318

mmusich · 2022-03-23T13:06:47Z

backport of #37316

PR description:

Follow-up to PR #35811.
Noticed that even in the CRAFT'22 DQM (post PR #35811 has been merged) data the Strip End Caps (TID / TEC) residuals are still empty (see e.g.: https://tinyurl.com/yd83lqlz and https://tinyurl.com/y8286jpa).
This has been traced down to the fact that in cosmics, unlike collisions (see:

cmssw/DQM/SiStripMonitorClient/python/SiStripSourceConfigTier0_cff.py

Lines 152 to 156 in 8204ef9

    
           from DQM.TrackerMonitorTrack.MonitorTrackResiduals_cfi import * 
        
           MonitorTrackResiduals.trajectoryInput = 'refittedForPixelDQM' 
        
           MonitorTrackResiduals.Tracks          = 'refittedForPixelDQM' 
        
           MonitorTrackResiduals.Mod_On        = False 
        
           MonitorTrackResiduals.genericTriggerEventPSet = genericTriggerEventFlag4HLTdb

) the bare track collection (without the trajectory in the event) is fed to TrackerValidationVariables (making it impossible to get precise residuals in the endcap).
This is fixed in this PR introducing the cosmics track refitter in the DQM configuration.

PR validation:

Run: runTheMatrix.py -l 138.2 -j 8 --command='-n 1000'

with the following patch:

diff --git a/DQM/TrackerMonitorTrack/src/MonitorTrackResiduals.cc b/DQM/TrackerMonitorTrack/src/MonitorTrackResiduals.cc
index bb58285028f..4b9eb6d0e8d 100644
--- a/DQM/TrackerMonitorTrack/src/MonitorTrackResiduals.cc
+++ b/DQM/TrackerMonitorTrack/src/MonitorTrackResiduals.cc
@@ -274,14 +274,25 @@ void MonitorTrackResidualsBase<pixel_or_strip>::analyze(const edm::Event &iEvent
 
       auto subdetandlayer = findSubdetAndLayer(RawId, tTopo);
       auto histos = m_SubdetLayerResiduals[subdetandlayer];
+      std::cout << " subdet: " << subdetandlayer.first << " layer: " << subdetandlayer.second << std::endl;
+
       // fill if its error is not zero
       if (it.resXprimeErr != 0 && histos.x.base) {
         histos.x.base->Fill(it.resXprime);
         histos.x.normed->Fill(it.resXprime / it.resXprimeErr);
+
+       if(subdetandlayer.first=="TEC" || subdetandlayer.first=="TID" ){
+         std::cout << "x-residual: " << it.resXprime << " / " << it.resXprimeErr << std::endl;
+       }
         if (!isPixel)
           tkhisto_ResidualsMean->fill(RawId, it.resXprime);
       }
       if (it.resYprimeErr != 0 && histos.y.base) {
+
+       if(subdetandlayer.first=="TEC" || subdetandlayer.first=="TID" ){
+         std::cout << "y-residual:" << it.resYprime << " / " << it.resYprimeErr << std::endl;
+       }
+
         histos.y.base->Fill(it.resYprime);
         histos.y.normed->Fill(it.resYprime / it.resYprimeErr);
       }

and now one gets:

Begin processing the 18th record. Run 343498, Event 18792252, LumiSection 1151 on stream 0 at 23-Mar-2022 13:02:01.846 CET
 subdet: TEC layer: 1
x-residual: -0.0151313 / 0.0473088
 subdet: TEC layer: 2
x-residual: 0.0107512 / 0.0143062
 subdet: TEC layer: 3
x-residual: -0.00317227 / 0.00859425
 subdet: TEC layer: 3
x-residual: -0.00389814 / 0.00671078
 subdet: TEC layer: 3
x-residual: 0.00796366 / 0.00832214
 subdet: TEC layer: 4
x-residual: 0.00557164 / 0.0108646
 subdet: TEC layer: 5
x-residual: -0.00106809 / 0.00630571
 subdet: TEC layer: 5
x-residual: 0.00326104 / 0.00766269
 subdet: TEC layer: 6
x-residual: 0.00382722 / 0.010256
 subdet: TEC layer: 6
x-residual: -0.00890858 / 0.00832553
 subdet: TEC layer: 6
x-residual: -0.00241953 / 0.00897227
 subdet: TEC layer: 6
x-residual: -0.00600596 / 0.00603685
 subdet: TEC layer: 6
x-residual: 0.00968534 / 0.00783616
 subdet: TEC layer: 7
x-residual: 0.00883887 / 0.0321769

whereas before it was:

Begin processing the 18th record. Run 343498, Event 18792252, LumiSection 1151 on stream 0 at 23-Mar-2022 12:25:17.472 CET
 subdet: TEC layer: 7
x-residual: -998001 / 999
 subdet: TEC layer: 6
x-residual: -998001 / 999
 subdet: TEC layer: 6
x-residual: -998001 / 999
 subdet: TEC layer: 6
x-residual: -998001 / 999
 subdet: TEC layer: 6
x-residual: -998001 / 999
 subdet: TEC layer: 6
x-residual: -998001 / 999
 subdet: TEC layer: 5
x-residual: -998001 / 999
 subdet: TEC layer: 5
x-residual: -998001 / 999
 subdet: TEC layer: 4
x-residual: -998001 / 999
 subdet: TEC layer: 3
x-residual: -998001 / 999
 subdet: TEC layer: 3
x-residual: -998001 / 999
 subdet: TEC layer: 3
x-residual: -998001 / 999
 subdet: TEC layer: 2
x-residual: -998001 / 999
 subdet: TEC layer: 1
x-residual: -998001 / 999

Example of obtained plots:

CMSSW_12_4_X_2022-03-22-1100 as-is	CMSSW_12_4_X_2022-03-22-1100 + this PR

if this PR is a backport please specify the original PR and why you need to backport that PR:

Backport of #37316

cmsbuild · 2022-03-23T13:07:07Z

A new Pull Request was created by @mmusich (Marco Musich) for CMSSW_12_2_X.

It involves the following packages:

DQM/SiStripMonitorClient (dqm)

@emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @pmandrik, @pbo0, @rvenditti can you please review it and eventually sign? Thanks.
@arossi83, @hdelanno, @sroychow, @fioriNTU, @jandrea, @idebruyn, @threus, @venturia this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

Backported from run cosmics track refitter in order to fix to Strip Hit residuals #37316

jfernan2 · 2022-03-23T15:05:07Z

I understand this should be merged in Online DQM at P5

jfernan2 · 2022-03-23T15:05:58Z

test parameters:

addpkg = DQM/Integration

mmusich · 2022-03-23T15:07:00Z

type bug-fix

jfernan2 · 2022-03-23T15:10:37Z

please test

cmsbuild · 2022-03-23T20:53:43Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-458507/23326/summary.html
COMMIT: bd6811f
CMSSW: CMSSW_12_2_X_2022-03-23-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/37318/23326/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 42
DQMHistoTests: Total histograms compared: 3251324
DQMHistoTests: Total failures: 57
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3251245
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 41 files compared)
Checked 177 log files, 37 edm output root files, 42 DQM output files
TriggerResults: no differences found

jfernan2 · 2022-03-24T07:36:10Z

+1

cmsbuild · 2022-03-24T07:36:35Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_2_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_12_4_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

francescobrivio · 2022-03-24T12:39:49Z

urgent

might be needed for deployment at P5 if a new 12_2 release will be cut shortly

francescobrivio · 2022-03-24T16:02:00Z

I understand this should be merged in Online DQM at P5

I just read this message from @jfernan2.
Indeed if this only touches DQM we could deploy it in P5 without the need for and urgent 12_2 release,
@mmusich what do you think?

perrotta · 2022-03-24T16:39:16Z

I understand this should be merged in Online DQM at P5

I just read this message from @jfernan2. Indeed if this only touches DQM we could deploy it in P5 without the need for and urgent 12_2 release, @mmusich what do you think?

Well, we propably do need to prepare a new 12_2 release with a bug fix needed to solve some issue that showed up in production (several jobs are failing), see #37334
You can profit of that 12_2 build anyhow.

francescobrivio · 2022-03-24T16:54:03Z

I understand this should be merged in Online DQM at P5

I just read this message from @jfernan2. Indeed if this only touches DQM we could deploy it in P5 without the need for and urgent 12_2 release, @mmusich what do you think?

Well, we propably do need to prepare a new 12_2 release with a bug fix needed to solve some issue that showed up in production (several jobs are failing), see #37334 You can profit of that 12_2 build anyhow.

Indeed you are right Andrea!
In addition I discussed privately with Marco and this PR does not affect online DQM, but only offline DQM, so the fix would actually be in Tier0 rather than P5. I'll report this in the JointOps meeting tomorrow.

mmusich · 2022-03-24T17:14:50Z

Indeed if this only touches DQM we could deploy it in P5 without the need for and urgent 12_2 release,

@francescobrivio @perrotta @jfernan2 the concerned module is not run in the online DQM clients, therefore the proposed update pertains exclusively to offline DQM. As such to absorb the changes a new release containing the fix shall be deployed at Tier-0.

perrotta · 2022-03-24T19:31:23Z

+1

run cosmics track refitter in order to fix to Strip Hit residuals

bd6811f

cmsbuild added this to the CMSSW_12_2_X milestone Mar 23, 2022

cmsbuild added backport dqm-pending orp-pending pending-signatures tests-pending labels Mar 23, 2022

cmsbuild added bug-fix tests-started and removed tests-pending labels Mar 23, 2022

cmsbuild added tests-approved and removed tests-started labels Mar 23, 2022

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels Mar 24, 2022

cmsbuild added backport-ok and removed backport labels Mar 24, 2022

cmsbuild added the urgent label Mar 24, 2022

francescobrivio mentioned this pull request Mar 24, 2022

[12.3.X] run cosmics track refitter in order to fix to Strip Hit residuals #37317

Merged

cmsbuild added orp-approved and removed orp-pending labels Mar 24, 2022

cmsbuild merged commit 3d10973 into cms-sw:CMSSW_12_2_X Mar 24, 2022

mmusich deleted the fixStripEndcapResidualsDQM_12_2_X branch March 24, 2022 19:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[12.2.X] run cosmics track refitter in order to fix to Strip Hit residuals #37318

[12.2.X] run cosmics track refitter in order to fix to Strip Hit residuals #37318

mmusich commented Mar 23, 2022

cmsbuild commented Mar 23, 2022 •

edited

jfernan2 commented Mar 23, 2022 •

edited

jfernan2 commented Mar 23, 2022

mmusich commented Mar 23, 2022

jfernan2 commented Mar 23, 2022

cmsbuild commented Mar 23, 2022

jfernan2 commented Mar 24, 2022

cmsbuild commented Mar 24, 2022

francescobrivio commented Mar 24, 2022

francescobrivio commented Mar 24, 2022

perrotta commented Mar 24, 2022

francescobrivio commented Mar 24, 2022

mmusich commented Mar 24, 2022

perrotta commented Mar 24, 2022

	from DQM.TrackerMonitorTrack.MonitorTrackResiduals_cfi import *
	MonitorTrackResiduals.trajectoryInput = 'refittedForPixelDQM'
	MonitorTrackResiduals.Tracks = 'refittedForPixelDQM'
	MonitorTrackResiduals.Mod_On = False
	MonitorTrackResiduals.genericTriggerEventPSet = genericTriggerEventFlag4HLTdb

[12.2.X] run cosmics track refitter in order to fix to Strip Hit residuals #37318

[12.2.X] run cosmics track refitter in order to fix to Strip Hit residuals #37318

Conversation

mmusich commented Mar 23, 2022

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Mar 23, 2022 • edited

jfernan2 commented Mar 23, 2022 • edited

jfernan2 commented Mar 23, 2022

mmusich commented Mar 23, 2022

jfernan2 commented Mar 23, 2022

cmsbuild commented Mar 23, 2022

Comparison Summary

jfernan2 commented Mar 24, 2022

cmsbuild commented Mar 24, 2022

francescobrivio commented Mar 24, 2022

francescobrivio commented Mar 24, 2022

perrotta commented Mar 24, 2022

francescobrivio commented Mar 24, 2022

mmusich commented Mar 24, 2022

perrotta commented Mar 24, 2022

cmsbuild commented Mar 23, 2022 •

edited

jfernan2 commented Mar 23, 2022 •

edited