HCAL: Applying the fix on pulse containment correction algorithm both for Run3 trigger primitives and Method 0 #35762

TaeunKwon · 2021-10-21T10:33:29Z

PR description:

In the PR #34572 and #34805, the pulse containment correction(PCC) algorithm was updated to make the timePhase definition in HcalPulseContainment similar to SIM step for Run3 Trigger Primitives(TPs). This update was originally designed to make a better agreement between TP energy for the PFA1' scheme and MAHI energy, but recently we found that it also has affected TP energy for the PFA2 scheme(old TP algorithm), by reducing its energy scale by 3~4%. And because we haven't applied this update in the PCC algorithm for RecHits(Method 0), currently there exists a discrepancy between PFA2 and Method 0, which we don't want. Therefore, this PR applies the update on the PCC algorithm described in PR #34572 also for method 0, by adding a shared configurable parameter switching on the PCC update both in TP and Method 0. The impact of this PR is shown in this presentation.

PR validation:

A basic technical test was performed: runTheMatrix.py -l limited

if this PR is a backport please specify the original PR and why you need to backport that PR:

This PR is not a backport.

cmsbuild · 2021-10-21T10:39:27Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-35762/26106

This PR adds an extra 24KB to repository

cmsbuild · 2021-10-21T10:39:48Z

A new Pull Request was created by @TaeunKwon for master.

It involves the following packages:

CalibCalorimetry/HcalPlugins (alca)
CalibCalorimetry/HcalTPGEventSetup (l1, alca)
RecoLocalCalo/HcalRecAlgos (reconstruction)
RecoLocalCalo/HcalRecProducers (reconstruction)
SimCalorimetry/HcalTrigPrimProducers (l1)

@malbouis, @yuanchao, @cmsbuild, @rekovic, @slava77, @jpata, @tvami, @cecilecaillol, @francescobrivio can you please review it and eventually sign? Thanks.
@bsunanda, @rchatter, @rovere, @argiro, @apsallid, @tocheng, @thomreis, @simonepigazzini, @mmusich, @abdoulline, @mariadalfonso this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

tvami · 2021-10-21T10:52:24Z

@cmsbuild , please test

cmsbuild · 2021-10-21T14:13:16Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-547dbb/19793/summary.html
COMMIT: d30db88
CMSSW: CMSSW_12_1_X_2021-10-20-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/35762/19793/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 76 differences found in the comparisons
DQMHistoTests: Total files compared: 40
DQMHistoTests: Total histograms compared: 2751113
DQMHistoTests: Total failures: 100
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 2750990
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.004 KiB( 39 files compared)
DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
Checked 170 log files, 37 edm output root files, 40 DQM output files
TriggerResults: no differences found

tvami · 2021-10-21T14:57:54Z

+alca

changes in AlCa files are trivial, just introduce a new boolean and it's switched on for Run-3
Jenkins diffs are visible in HCAL, which is expected (+MsgLogger)

CalibCalorimetry/HcalPlugins/python/Hcal_PCCUpdate_cff.py

slava77 · 2021-10-23T02:38:15Z

+reconstruction

for #35762 d30db88

code changes are in line with the PR description
jenkins tests pass and comparisons with the baseline show differences only in Run-3 and later workflows in the HBHE eraw (M0) energy, which reduces slightly; this is consistent with the expectation of a few % reduction as mentioned in https://indico.cern.ch/event/1084544/contributions/4559681/attachments/2324926/3959938/Commissioning_1TS_Scheme_21Oct08.pdf

Co-authored-by: Slava Krutelyov <slava77@gmail.com>

cmsbuild · 2021-10-25T14:14:51Z

Pull request #35762 was updated. @malbouis, @yuanchao, @rekovic, @slava77, @jpata, @tvami, @cecilecaillol, @francescobrivio can you please check and sign again.

cmsbuild · 2021-10-25T15:38:08Z

-1

Failed Tests: RelVals
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-547dbb/19903/summary.html
COMMIT: 1cbf578
CMSSW: CMSSW_12_1_X_2021-10-24-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/35762/19903/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

2018.1

----- Begin Fatal Exception 25-Oct-2021 16:44:28 CEST-----------------------
An exception of category 'NoRecord' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'prevalidation_step'
   [2] Prefetching for module MultiTrackValidator/'trackValidatorBuilding'
   [3] Prefetching for module TrackProducer/'detachedTripletStepTracks'
   [4] Prefetching for module TrackCandidateProducer/'detachedTripletStepTrackCandidates'
   [5] Prefetching for module FastTrackerRecHitMaskProducer/'detachedTripletStepMasks'
   [6] Prefetching for module TrackProducer/'detachedQuadStepTracks'
   [7] Prefetching for module TrackCandidateProducer/'detachedQuadStepTrackCandidates'
   [8] Prefetching for module FastTrackerRecHitMaskProducer/'detachedQuadStepMasks'
   [9] Prefetching for module TrackProducer/'lowPtTripletStepTracks'
   [10] Prefetching for module TrackCandidateProducer/'lowPtTripletStepTrackCandidates'
   [11] Prefetching for module FastTrackerRecHitMaskProducer/'lowPtTripletStepMasks'
   [12] Prefetching for module TrackProducer/'highPtTripletStepTracks'
   [13] Prefetching for module TrackCandidateProducer/'highPtTripletStepTrackCandidates'
   [14] Prefetching for module FastTrackerRecHitMaskProducer/'highPtTripletStepMasks'
   [15] Prefetching for module TrackProducer/'lowPtQuadStepTracks'
   [16] Prefetching for module TrackCandidateProducer/'lowPtQuadStepTrackCandidates'
   [17] Prefetching for module FastTrackerRecHitMaskProducer/'lowPtQuadStepMasks'
   [18] Calling method for module TrackTfClassifier/'initialStep'
Exception Message:
No "TfGraphRecord" record found in the EventSetup.

 The Record is delivered by an ESSource or ESProducer but there is no valid IOV for the synchronization value.
 Please check 
   a) if the synchronization value is reasonable and report to the hypernews if it is not.
   b) else check that all ESSources have been properly configured. 
----- End Fatal Exception -------------------------------------------------

perrotta · 2021-10-25T15:44:17Z

@cmsbuild , please test with #35801

You were right @tvami , CMSSW_12_1_X_2021-10-25-1100 was not picked up yet by the tests

cmsbuild · 2021-10-25T19:04:51Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-547dbb/19923/summary.html
COMMIT: 1cbf578
CMSSW: CMSSW_12_1_X_2021-10-25-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/35762/19923/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 70 differences found in the comparisons
DQMHistoTests: Total files compared: 41
DQMHistoTests: Total histograms compared: 2797338
DQMHistoTests: Total failures: 99
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 2797216
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.004 KiB( 40 files compared)
DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
Checked 173 log files, 37 edm output root files, 41 DQM output files
TriggerResults: no differences found

tvami · 2021-10-25T19:55:55Z

+alca

see HCAL: Applying the fix on pulse containment correction algorithm both for Run3 trigger primitives and Method 0 #35762 (comment)

rekovic · 2021-10-26T09:26:02Z

+1

slava77 · 2021-10-26T12:44:57Z

+reconstruction

for #35762 1cbf578

code changes since the last reco signoff were only in CalibCalorimetry/HcalPlugins

cmsbuild · 2021-10-26T12:45:21Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2021-10-26T18:54:26Z

+1

applyFixPCC both in RH and TP

d30db88

cmsbuild added this to the CMSSW_12_1_X milestone Oct 21, 2021

cmsbuild added alca-pending code-checks-pending l1-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Oct 21, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 21, 2021

cmsbuild added tests-started and removed tests-pending labels Oct 21, 2021

cmsbuild added tests-approved and removed tests-started labels Oct 21, 2021

cmsbuild added alca-approved and removed alca-pending labels Oct 21, 2021

slava77 reviewed Oct 23, 2021

View reviewed changes

CalibCalorimetry/HcalPlugins/python/Hcal_PCCUpdate_cff.py Outdated Show resolved Hide resolved

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Oct 23, 2021

Update CalibCalorimetry/HcalPlugins/python/Hcal_PCCUpdate_cff.py

1cbf578

Co-authored-by: Slava Krutelyov <slava77@gmail.com>

cmsbuild removed alca-approved reconstruction-approved tests-approved labels Oct 25, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 25, 2021

cmsbuild added tests-rejected and removed tests-started labels Oct 25, 2021

cmsbuild added requires-external tests-started and removed tests-rejected labels Oct 25, 2021

cmsbuild added tests-approved and removed tests-started labels Oct 25, 2021

cmsbuild added alca-approved and removed alca-pending labels Oct 25, 2021

cmsbuild added l1-approved and removed l1-pending labels Oct 26, 2021

cmsbuild added fully-signed reconstruction-approved and removed reconstruction-pending pending-signatures labels Oct 26, 2021

cmsbuild added orp-approved and removed orp-pending labels Oct 26, 2021

cmsbuild merged commit 6c16996 into cms-sw:master Oct 26, 2021

cmsbuild mentioned this pull request Oct 27, 2021

default initialize HiPuRhoProducer::postOrphan_ #35857

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HCAL: Applying the fix on pulse containment correction algorithm both for Run3 trigger primitives and Method 0 #35762

HCAL: Applying the fix on pulse containment correction algorithm both for Run3 trigger primitives and Method 0 #35762

TaeunKwon commented Oct 21, 2021

cmsbuild commented Oct 21, 2021

cmsbuild commented Oct 21, 2021

tvami commented Oct 21, 2021

cmsbuild commented Oct 21, 2021

tvami commented Oct 21, 2021

slava77 commented Oct 23, 2021

cmsbuild commented Oct 25, 2021

cmsbuild commented Oct 25, 2021

perrotta commented Oct 25, 2021

cmsbuild commented Oct 25, 2021

tvami commented Oct 25, 2021

rekovic commented Oct 26, 2021

slava77 commented Oct 26, 2021

cmsbuild commented Oct 26, 2021

perrotta commented Oct 26, 2021

HCAL: Applying the fix on pulse containment correction algorithm both for Run3 trigger primitives and Method 0 #35762

HCAL: Applying the fix on pulse containment correction algorithm both for Run3 trigger primitives and Method 0 #35762

Conversation

TaeunKwon commented Oct 21, 2021

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Oct 21, 2021

cmsbuild commented Oct 21, 2021

tvami commented Oct 21, 2021

cmsbuild commented Oct 21, 2021

Comparison Summary

tvami commented Oct 21, 2021

slava77 commented Oct 23, 2021

cmsbuild commented Oct 25, 2021

cmsbuild commented Oct 25, 2021

RelVals

perrotta commented Oct 25, 2021

cmsbuild commented Oct 25, 2021

Comparison Summary

tvami commented Oct 25, 2021

rekovic commented Oct 26, 2021

slava77 commented Oct 26, 2021

cmsbuild commented Oct 26, 2021

perrotta commented Oct 26, 2021