HBHE M2 = HPD old code cleanup

[mariadalfonso]

Given that we are switching the legacy re-reco to the "Phase1"code with the #17676.
I also removed the various if statement for the HPD-QIE8.

Now take the noise from the database instead of a flat numbers from the python code
the ADCquantization noise use the exact definition instead of a parametrized version

expect minor changes in the noise hits

[cmsbuild]

A new Pull Request was created by @mariadalfonso for master.

It involves the following packages:

RecoLocalCalo/HcalRecAlgos

@cmsbuild, @cvuosalo, @slava77, @davidlange6 can you please review it and eventually sign? Thanks.
@argiro this is something you requested to watch as well.
@Muzaffar, @davidlange6, @smuzaffar you are the release manager for this.

cms-bot commands are listed here #13028

[slava77]

@mariadalfonso
if I understand correctly, this PR changes behavior and needs #17676 to be merged to proceed with tests.
Please confirm.
Thank you.

[abdoulline]

90X #17676 -> now 91X #17688 for this PR

[mariadalfonso]

@slava77
To see the effect of this PR on some of the workflow we do not need to wait for the #17676 ; but this PR should be merged only if the #17676, that is when the completely dismiss the "run1/run2-2016" code in favor of the "Phase1" for all the era.

[slava77]

@cmsbuild please test
.. to be re-tested after #17676

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/18064/console Started: 2017/03/02 15:32
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/18109/console Started: 2017/03/03 03:01

[cmsbuild]

+1
Tested at: 9eb66e9
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17680/18109/summary.html

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17680/18109/summary.html

[mariadalfonso]

@slava77
What I do with this PR, should I forward port to the 91X (as the 17676 ->17688) ?

[slava77]

@mariadalfonso this is already a 91X PR.
I didn't get to #17688 yet.
Once it's signed off, this PR will be reviewed.

[slava77]

@cmsbuild please test

the last set of tests has expired

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/18535/console Started: 2017/03/18 05:10

[slava77]

@cmsbuild please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/18570/console Started: 2017/03/20 20:42

[slava77]

@kpedro88 @abdoulline
maybe you can comment on the difference in chi2 between HB and HE
is digitization/quantization scale different or something else similar in effect on the pulse shape

Looking vs energy, it doesn't appear as if the effect is noise related
(the plot is from wf 10071, 2017 flat-pt QCD )

[mariadalfonso]

@slava77
This PR takes

1. ADC quantization from exact definitions, some test can be found in HBHE: use proper quantization noise in the M2 #17037
2. pedestal width from the DB HBHE M2: noise definition cleanup #17129

From plots in these PR, I do not expect changes from 1) and 2) in the chi2

3. change also the gaussian constraint on the pedestal

[cmsbuild]

+1
Tested at: 9eb66e9
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17680/18570/summary.html

[cmsbuild]

Comparison job queued.

[abdoulline]

@slava77 the difference I see between HB and HE (e.g. for the same end RecHit energy) comes from a different sampling (approx. = absorber/scintillator ratio) (1) pfotostatistics. Factor ~1.8 more light (p.e.) in HB for the same "end" RecHit energy. So, relative photostatistics error is bigger in HE [*] (2) similarly ADC quantization scale is lower in HE by factor ~1.8. less fC = lower ADC scale for the same "end" RecHit energy wrt HB - means bigger relative quantization error in HE. Well, it might mean that if something is changed, Chi2 effect should be less pronounced in HE (with its bigger error terms) ? S. [*] simHitToPhotoelectrons is similar for HB and HE https://github.com/cms-sw/cmssw/blob/CMSSW_9_0_X/SimCalorimetry/HcalSimAlgos/src/HcalSimParameters.cc#L56 (samplingFactor ~1.8 difference compensated by ~1.8 different fCtoGeV) Means Poisson smearing is more "crude" for HE as ~same "npe" generated here for the same SimHit energy in HB and HE https://github.com/cms-sw/cmssw/blob/CMSSW_9_0_X/SimCalorimetry/CaloSimAlgos/src/CaloHitResponse.cc#L166 but in HE SimHit energy is ~1.8 smaller than in HB for the same resulting RecHit.

…

On Mon, 20 Mar 2017, Slava Krutelyov wrote: @kpedro88 @abdoulline maybe you can comment on the difference in chi2 between HB and HE is digitization/quantization scale different or something else similar in effect on the pulse shape Looking vs energy, it doesn't appear as if the effect is noise related wf10071_chi2_vs_e_hb wf10071_chi2_vs_e_he — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.[AEx02s6gOIVFIDc1A4Z4sBH29D1FlXgFks5rnrgfgaJpZM4MQ1kM.gif]

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17680/18570/summary.html

[mariadalfonso]

@abdoulline

(1) pfotostatistics. Factor ~1.8 more light (p.e.) in HB for the same
"end" RecHit energy. So, relative photostatistics error is bigger in HE [*]

This PhotoStat term didn't change in this PR, as was already active in the release for both the HB and HE and HEP17

(2) similarly ADC quantization scale is lower in HE by factor ~1.8.
less fC = lower ADC scale for the same "end" RecHit energy wrt HB -
means bigger relative quantization error in HE.

Well, it might mean that if something is changed, Chi2 effect should be
less pronounced in HE (with its bigger error terms) ?

S.

[abdoulline]

Yes, I know that photostat. term didn't change in this PR, Maria. I was just answering Slava's question about a difference in general between HE and HB.

…

On Tue, 21 Mar 2017, mariadalfonso wrote: @abdoulline (1) pfotostatistics. Factor ~1.8 more light (p.e.) in HB for the same "end" RecHit energy. So, relative photostatistics error is bigger in HE [*] This PhotoStat term didn't change in this PR, as was already active in the release for both the HB and HE and HEP17 (2) similarly ADC quantization scale is lower in HE by factor ~1.8. less fC = lower ADC scale for the same "end" RecHit energy wrt HB - means bigger relative quantization error in HE. Well, it might mean that if something is changed, Chi2 effect should be less pronounced in HE (with its bigger error terms) ? S. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.[AEx02jMnhQv9EDfmzX_oDLAhfF1w9IJqks5rnxB1gaJpZM4MQ1kM.gif]

[slava77]

+1

for #17680 9eb66e9

• code changes are as described
• jenkins tests pass and comparisons showed differences only in phase-1 workflows in the state before use HBHE Phase1 reco for Run2/legacy, unify M0 parameters #17688 is merged; most recent comparisons show differences in other workflows as well [consistent with observations in local tests]
• local tests on top of use HBHE Phase1 reco for Run2/legacy, unify M0 parameters #17688 show somewhat expected behavior

#17680 (comment) :
changes are small enough to not affect whatever lower level calibrations are to be done with 90X. So, I think that this can go to 91X.

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @Muzaffar, @davidlange6, @smuzaffar

[davidlange6]

+1