[HBHE] Remove Phase 1 HE from HBHE noise filter consideration #22337

jaehyeok · 2018-02-24T01:34:56Z

With full depths in Phase 1 HE we found that some variables used in the HBHE noise filter decision need to be updated.

Phase 1 HE (SiPM+QIE11) has more channels than phase 0 HE, i.e., 46 vs 18 channels per RM. Some of the variables in HBHE noise filters use rechit multiplicity per RM, so cut values should be updated accordingly to account for the increase of rechit multiplicity. But, we know that the QIE11 channels don’t have coherent noise [1], so we decided to exclude HE channels in the construction of filter variables; we use only HB channels to calculate the variables to cut on.

We use HBHERecHitAuxSetter::OFF_TDC_TIME bit in HBHERecHit.auxPhase1() to exclude QIE11 channels because the bit is set to 1 for QIE11 channels and to 0 for QIE8 channels.

This update will have a visible effect in MC reconstructed with full HE depth segmentation and a small effect in data.

More details can be found in [2].

[1] https://twiki.cern.ch/twiki/pub/CMSPublic/HcalDPGResultsCMSDPS2017042/HEP17noise.pdf
[2] https://indico.cern.ch/event/708228/contributions/2907553/attachments/1605989/2548603/20180223_Jae_HCAL_RemoveHEForNoise.pdf

cmsbuild · 2018-02-24T01:35:17Z

The code-checks are being triggered in jenkins.

cmsbuild · 2018-02-24T01:37:16Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-22337/3554

cmsbuild · 2018-02-24T01:37:37Z

A new Pull Request was created by @jaehyeok for master.

It involves the following packages:

DataFormats/METReco
RecoMET/METProducers

@perrotta, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@TaiSakuma, @gouskos, @ahinzmann, @mmarionncern, @rappoccio, @rovere, @jdamgov, @jdolen, @nhanvtran, @gkasieczka, @schoef, @mariadalfonso, @seemasharmafnal this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

perrotta · 2018-02-24T10:34:44Z

please test

cmsbuild · 2018-02-24T10:35:03Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/26293/console Started: 2018/02/24 11:35

cmsbuild · 2018-02-24T12:24:35Z

+1
Tested at: 7c7b2a1
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22337/26293/summary.html

cmsbuild · 2018-02-24T12:24:38Z

Comparison job queued.

cmsbuild · 2018-02-24T14:02:14Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22337/26293/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/build/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-22337/136.8311_RunJetHT2017F_reminiaod+RunJetHT2017F_reminiaod+REMINIAOD_data2017+HARVEST2017_REMINIAOD_data2017

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 12 differences found in the comparisons
DQMHistoTests: Total files compared: 29
DQMHistoTests: Total histograms compared: 2500248
DQMHistoTests: Total failures: 42
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2500030
DQMHistoTests: Total skipped: 176
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 1.09999999997 KiB( 23 files compared)
Checked 118 log files, 9 edm output root files, 29 DQM output files

deguio · 2018-02-26T12:06:23Z

thanks @jaehyeok
I think it is worth backporting this to 10_0_x
could you do it?
thanks!

slava77 · 2018-02-26T14:46:41Z

@deguio
why would we need this in 10_0_X?

mariadalfonso · 2018-02-26T16:26:06Z

@slava77
As far as we understand, trigger studies will be done in the 10_0_X so this recHit filters will impact the METtrigger paths ( via recHit cleaning) for both online/offline and data/MC.

slava77 · 2018-03-02T00:59:05Z

@cmsbuild please test

cmsbuild · 2018-03-02T00:59:21Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-22337/3651

cmsbuild · 2018-03-02T01:01:01Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/26419/console Started: 2018/03/02 02:01

cmsbuild · 2018-03-02T01:01:02Z

Pull request #22337 was updated. @perrotta, @cmsbuild, @slava77 can you please check and sign again.

jaehyeok · 2018-03-02T01:05:27Z

@igv4321

Thanks. I updated the code with CaloRecHitAuxSetter::getBit function.

cmsbuild · 2018-03-02T03:19:29Z

+1
Tested at: 0b3598e
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22337/26419/summary.html

cmsbuild · 2018-03-02T03:19:32Z

Comparison job queued.

cmsbuild · 2018-03-02T04:42:09Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22337/26419/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 8 differences found in the comparisons
DQMHistoTests: Total files compared: 29
DQMHistoTests: Total histograms compared: 2479021
DQMHistoTests: Total failures: 31
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2478814
DQMHistoTests: Total skipped: 176
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 1.17000000002 KiB( 23 files compared)
Checked 118 log files, 9 edm output root files, 29 DQM output files

slava77 · 2018-03-02T22:53:21Z

+1

for #22337 0b3598e

improved since the last signoff [HBHE] Remove Phase 1 HE from HBHE noise filter consideration #22337 (comment) to disable the noise filter only for uncollapsed (default in reco) phase-1 HBHE
jenkins tests pass and comparisons with the baseline show differences only in 2018/2019 tests (the changes previously present in the 2017 are gone as expected for the collapsed reco scenario of the plan-1)

cmsbuild · 2018-03-02T22:53:37Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

fabiocos · 2018-03-04T17:38:35Z

@slava77 @perrotta in my understanding this PR changes the RECO results at low level for HCAL compared to 10_0_X. Do I understand that anyway this is a needed input to the tuning of high level reconstruction, to be added on top of the 10_0_X RECO? And its backport to 10_0_X is also required for HLT tuning?

slava77 · 2018-03-04T17:47:05Z

@slava77 @perrotta in my understanding this PR changes the RECO results at low level for HCAL compared to 10_0_X. Do I understand that anyway this is a needed input to the tuning of high level reconstruction, to be added on top of the 10_0_X RECO? And its backport to 10_0_X is also required for HLT tuning?

There is no backport to 10_0_X anymore.
The changes in this PR do not affect HCAL hit reconstruction, which could affect the planned calibrations from 10_1_X.

fabiocos · 2018-03-04T17:54:09Z

@slava77 ok, thanks for the clarification

fabiocos · 2018-03-04T17:54:14Z

+1

jaehyeok added 4 commits February 17, 2018 02:18

Remove HE rechits from noise filter consideration

4c12162

update comments

caa3d78

Remove the explicit 26, but uses HBHERecHitAuxSetter::OFF_TDC_TIME

23f5623

move the OFF_TDC_TIME statement after laser filter variables are filled

7c7b2a1

cmsbuild added this to the CMSSW_10_1_X milestone Feb 24, 2018

cmsbuild added code-checks-pending comparison-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Feb 24, 2018

cmsbuild added code-checks-approved and removed code-checks-pending labels Feb 24, 2018

cmsbuild added tests-started and removed tests-pending labels Feb 24, 2018

cmsbuild added tests-approved and removed tests-started labels Feb 24, 2018

cmsbuild added comparison-available and removed comparison-pending labels Feb 24, 2018

code cleaning

0b3598e

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 2, 2018

cmsbuild added tests-started and removed tests-pending labels Mar 2, 2018

cmsbuild added tests-approved and removed tests-started labels Mar 2, 2018

cmsbuild added comparison-available and removed comparison-pending labels Mar 2, 2018

cmsbuild added fully-signed reconstruction-approved and removed pending-signatures reconstruction-pending labels Mar 2, 2018

cmsbuild added orp-approved and removed orp-pending labels Mar 4, 2018

cmsbuild merged commit c6be741 into cms-sw:master Mar 4, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[HBHE] Remove Phase 1 HE from HBHE noise filter consideration #22337

[HBHE] Remove Phase 1 HE from HBHE noise filter consideration #22337

jaehyeok commented Feb 24, 2018

cmsbuild commented Feb 24, 2018

cmsbuild commented Feb 24, 2018

cmsbuild commented Feb 24, 2018

perrotta commented Feb 24, 2018

cmsbuild commented Feb 24, 2018 •

edited

cmsbuild commented Feb 24, 2018

cmsbuild commented Feb 24, 2018

cmsbuild commented Feb 24, 2018

deguio commented Feb 26, 2018

slava77 commented Feb 26, 2018

mariadalfonso commented Feb 26, 2018

slava77 commented Mar 2, 2018

cmsbuild commented Mar 2, 2018

cmsbuild commented Mar 2, 2018 •

edited

cmsbuild commented Mar 2, 2018

jaehyeok commented Mar 2, 2018

cmsbuild commented Mar 2, 2018

cmsbuild commented Mar 2, 2018

cmsbuild commented Mar 2, 2018

slava77 commented Mar 2, 2018

cmsbuild commented Mar 2, 2018

fabiocos commented Mar 4, 2018

slava77 commented Mar 4, 2018

fabiocos commented Mar 4, 2018

fabiocos commented Mar 4, 2018

[HBHE] Remove Phase 1 HE from HBHE noise filter consideration #22337

[HBHE] Remove Phase 1 HE from HBHE noise filter consideration #22337

Conversation

jaehyeok commented Feb 24, 2018

cmsbuild commented Feb 24, 2018

cmsbuild commented Feb 24, 2018

cmsbuild commented Feb 24, 2018

perrotta commented Feb 24, 2018

cmsbuild commented Feb 24, 2018 • edited

cmsbuild commented Feb 24, 2018

cmsbuild commented Feb 24, 2018

cmsbuild commented Feb 24, 2018

deguio commented Feb 26, 2018

slava77 commented Feb 26, 2018

mariadalfonso commented Feb 26, 2018

slava77 commented Mar 2, 2018

cmsbuild commented Mar 2, 2018

cmsbuild commented Mar 2, 2018 • edited

cmsbuild commented Mar 2, 2018

jaehyeok commented Mar 2, 2018

cmsbuild commented Mar 2, 2018

cmsbuild commented Mar 2, 2018

cmsbuild commented Mar 2, 2018

slava77 commented Mar 2, 2018

cmsbuild commented Mar 2, 2018

fabiocos commented Mar 4, 2018

slava77 commented Mar 4, 2018

fabiocos commented Mar 4, 2018

fabiocos commented Mar 4, 2018

cmsbuild commented Feb 24, 2018 •

edited

cmsbuild commented Mar 2, 2018 •

edited