dead ES regions with correct energy each plane

[argiro]

same as #17145 for 80x

[cmsbuild]

A new Pull Request was created by @argiro for CMSSW_8_0_X.

It involves the following packages:

RecoParticleFlow/PFClusterTools

@cmsbuild, @cvuosalo, @slava77, @davidlange6 can you please review it and eventually sign? Thanks.
@mmarionncern, @lgray, @rafaellopesdesa, @bachtis, @cbernet this is something you requested to watch as well.
@davidlange6, @smuzaffar you are the release manager for this.

cms-bot commands are listed here #13028

[slava77]

@argiro
this change technically affects the standard workflow; although practically they are protected by the values in current ESEEIntercalibConstants (while e.g. ESEEIntercalibConstants_LG_2016 will change that).
Please clarify if dead ES channels are included in the current MC campaign.

If there is a plan to change ESEEIntercalibConstants to ESEEIntercalibConstants_LG_2016 for the legacy rereco, this PR may have to wait until shortly before the legacy rereco release is built

@franzoni @arunhep @mmusich

[slava77]

@cmsbuild please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/17369/console Started: 2017/01/22 04:16

[cmsbuild]

+1
Tested at: 9cb1c72
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17219/17369/summary.html

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17219/17369/summary.html

[argiro]

after the software change, we plan on modifying the database record. ES channels in 2016 data taking are not included in the current MC so we do not plan to touch the MC tag.

…

On 20 Jan 2017, at 14:41, Slava Krutelyov ***@***.***> wrote: @argiro this change technically affects the standard workflow; although practically they are protected by the values in current ESEEIntercalibConstants (while e.g. ESEEIntercalibConstants_LG_2016 will change that). Please clarify if dead ES channels are included in the current MC campaign. If there is a plan to change ESEEIntercalibConstants to ESEEIntercalibConstants_LG_2016 for the legacy rereco, this PR may have to wait until shortly before the legacy rereco release is built @franzoni @arunhep @mmusich — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[slava77]

On 1/22/17 11:52 AM, argiro wrote: after the software change, we plan on modifying the database record. ES channels in 2016 data taking are not included in the current MC so we do not plan to touch the MC tag.

Thank you for the clarification. So, either this PR will wait until shortly before the legacy rereco release is built or the conditions are not to be updated until that moment. It's probably more practical to have this PR in place to be able to run tests of new conditions with the release.

…

> On 20 Jan 2017, at 14:41, Slava Krutelyov ***@***.***> wrote: > > @argiro > this change technically affects the standard workflow; although practically they are protected by the values in current ESEEIntercalibConstants (while e.g. ESEEIntercalibConstants_LG_2016 will change that). > Please clarify if dead ES channels are included in the current MC campaign. > > If there is a plan to change ESEEIntercalibConstants to ESEEIntercalibConstants_LG_2016 for the legacy rereco, this PR may have to wait until shortly before the legacy rereco release is built > > @franzoni @arunhep @mmusich > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub, or mute the thread. > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#17219 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEdcbnOGliK2vCYIC0TdGs-AHvYfB4UDks5rU7OXgaJpZM4LpFTw>.

[slava77]

backport of #17145

[slava77]

+1

for #17219 9cb1c72

• same change as dead ES regions with corrected energy for each plane. Fixes inconsistency in regression training #17145 72dba60. These changes become relevant only after ESEEIntercalibConstants is changed, which is expected to happen for the legacy rereco
• jenkins tests pass and comparisons with baseline show no differences

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_8_0_X IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @smuzaffar

[slava77]

On 1/31/17 9:50 AM, David Lange wrote: Right, but lets test in 90x…

Please clarify what should be tested in 90X. #17145 was in 90X for 13 days. So, integration level tests have passed. Will that need to be just default relvals with the old payload or a new calibration testing? For the latter, there will have to be some coordination (IIUC, there is no GT with the new payload)

…

> On Jan 31, 2017, at 9:35 AM, Slava Krutelyov ***@***.***> wrote: > > @davidlange6 > please notice the request. > > defaults are unchanged with current GT. > The code is needed to further test the new calibrations > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub, or mute the thread. > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#17219 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEdcbiRpluQJaoF326tRIZTUAvk-aX84ks5rXvXUgaJpZM4LpFTw>.

[davidlange6]

you suggested some testing was waiting on this to be integrated. I guess that is not the case. However, it sounds like getting this new code and new calibrations all to run and work in 90x is still to be accomplished. (as much as we all like integrating code in the dark its probably best to use the development release for our development)

…

On Jan 31, 2017, at 10:07 AM, Slava Krutelyov ***@***.***> wrote: On 1/31/17 9:50 AM, David Lange wrote: > Right, but lets test in 90x… Please clarify what should be tested in 90X. #17145 was in 90X for 13 days. So, integration level tests have passed. Will that need to be just default relvals with the old payload or a new calibration testing? For the latter, there will have to be some coordination (IIUC, there is no GT with the new payload) > >> On Jan 31, 2017, at 9:35 AM, Slava Krutelyov ***@***.***> wrote: >> >> @davidlange6 >> please notice the request. >> >> defaults are unchanged with current GT. >> The code is needed to further test the new calibrations >> >> — >> You are receiving this because you were mentioned. >> Reply to this email directly, view it on GitHub, or mute the thread. >> > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#17219 (comment)>, or > mute the thread > <https://github.com/notifications/unsubscribe-auth/AEdcbiRpluQJaoF326tRIZTUAvk-aX84ks5rXvXUgaJpZM4LpFTw>. > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[slava77]

On 1/31/17 10:16 AM, David Lange wrote: you suggested some testing was waiting on this to be integrated. I guess that is not the case. However, it sounds like getting this new code and new calibrations all to run and work in 90x is still to be accomplished. (as much as we all like integrating code in the dark its probably best to use the development release for our development)

OK. Just to be clear, what specifically does the 80X (this PR) should wait for? - new GT in 90X with the new payload or something else?

…

> On Jan 31, 2017, at 10:07 AM, Slava Krutelyov ***@***.***> wrote: > > On 1/31/17 9:50 AM, David Lange wrote: > > Right, but lets test in 90x… > > Please clarify what should be tested in 90X. > #17145 was in 90X for 13 days. > So, integration level tests have passed. > > Will that need to be just default relvals with the old payload > or a new calibration testing? > For the latter, there will have to be some coordination > (IIUC, there is no GT with the new payload) > > > > >> On Jan 31, 2017, at 9:35 AM, Slava Krutelyov ***@***.***> wrote: > >> > >> @davidlange6 > >> please notice the request. > >> > >> defaults are unchanged with current GT. > >> The code is needed to further test the new calibrations > >> > >> — > >> You are receiving this because you were mentioned. > >> Reply to this email directly, view it on GitHub, or mute the thread. > >> > > > > — > > You are receiving this because you were mentioned. > > Reply to this email directly, view it on GitHub > > <#17219 (comment)>, or > > mute the thread > > <https://github.com/notifications/unsubscribe-auth/AEdcbiRpluQJaoF326tRIZTUAvk-aX84ks5rXvXUgaJpZM4LpFTw>. > > > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub, or mute the thread. > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#17219 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEdcbhHgR6t875eFkewzND8SOHI1S5RGks5rXvwLgaJpZM4LpFTw>.

[slava77]

From the discussion in the ORP I understood that before this can be integrated in 80X, the 90X should have the new ES calibration payload included in tests (standard release GT).

[argiro]

a payload is under preparation

[franzoni]

during the discussion at the alca meeting yesterday
( slide 5 of https://indico.cern.ch/event/592613/contributions/2448614/ )
we agreed with @shervin86 and ming that they would follow up clarifying what needs to be done at GT level (email thread preferable to this chat, if you may)

How were the changes already deployed in 90x, w/o the corresponding change in GT ?
@arunhep may want to follow this one too.

[slava77]

@argiro @franzoni @shervin86
please clarify on the status of the payloads for 80X and for 90X

[cmkuo]

there was a confusion between me and the ECAL tag creator so the previous validation failed.
We sorted out the problem and a new tag was created. Shervin is producing ntuples for validation.
We expect the validation to be done in 1-2 days. Then we can give you the tag for 90X and also 80X if the fix is included.

[cmkuo]

@slava77 I would like to know whether we need a fully validated GT in order to include this fix for 90X. Or a partially validated GT which will not break the SW is OK to allow this fix to be included in 90X ?

[slava77]

On 2/16/17 6:39 AM, cmkuo wrote: @slava77 <https://github.com/slava77> I would like to know whether we need a fully validated GT in order to include this fix for 90X. Or a partially validated GT which will not break the SW is OK to allow this fix to be included in 90X ?

I expect that the GT will need to be good for a RelVal production with the given 90X release. This 80X PR can be included in 80X after the 90X validation. And I think it can be added by itself first, without an accompanying 80X GT.

…

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#17219 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEdcbhyow1OjtgVox_JQ-gqVwsziSuu_ks5rdF-TgaJpZM4LpFTw>.

[cmkuo]

@slava77 @shervin86 @franzoni @argiro

ESEEIntercalibConstants_Legacy2016_v3 with the payload for ES dead region calibration is fully validated.

The attached plots show the validation of one of runs in 2016D which had hardware problem to cause a dead region in the first ES plane.

Color codes : green -> MC, blue -> current tag, red -> ESEEIntercalibConstants_Legacy2016_v3

Feb_DoubleEG_3cat_ES2016D_ESEnP1_Dead.png shows the ES energy on the first plane.
You can see before the dead region calibration (blue), the ES energy does not agree with MC.
You may ask why the ES energy is not exactly zero. First of all, the x-axis does not start from 0.
Secondly, if the electrons hit the border of the dead region, the clustering can pick up the energy from the vicinity of active sensors. After the dead region calibration, the ES energy on the first plane agrees with MC.

Feb_DoubleEG_3cat_ES2016D_ESEn_Dead shows the sum of the energy for two ES planes.
You can see before the dead region calibration (blue), the energy comes from the second (active) plane. After the dead region calibration, the total ES energy agrees with MC.

[slava77]

On 2/18/17 1:42 AM, cmkuo wrote: @slava77 <https://github.com/slava77> @shervin86 <https://github.com/shervin86> @franzoni <https://github.com/franzoni> @argiro <https://github.com/argiro> ESEEIntercalibConstants_Legacy2016_v3 with the payload for ES dead region calibration is fully validated.

Hi Ming, Thank you for sharing the plots. It looks like this payload can be added to the 90X already. Will it make it to 900pre5? Please clarify. Thank you.

…

--slava

The attached plots show the validation of one of runs in 2016D which had hardware problem to cause a dead region in the first ES plane. Color codes : green -> MC, blue -> current tag, red -> ESEEIntercalibConstants_Legacy2016_v3 Feb_DoubleEG_3cat_ES2016D_ESEnP1_Dead.png shows the ES energy on the first plane. You can see before the dead region calibration (blue), the ES energy does not agree with MC. You may ask why the ES energy is not exactly zero. First of all, the x-axis does not start from 0. Secondly, if the electrons hit the border of the dead region, the clustering can pick up the energy from the vicinity of active sensors. After the dead region calibration, the ES energy on the first plane agrees with MC. Feb_DoubleEG_3cat_ES2016D_ESEn_Dead shows the sum of the energy for two ES planes. You can see before the dead region calibration (blue), the energy comes from the second (active) plane. After the dead region calibration, the total ES energy agrees with MC. feb_doubleeg_3cat_es2016d_esenp1_dead <https://cloud.githubusercontent.com/assets/5124244/23092171/7329cf0e-f5c5-11e6-887c-cb13bc56cbe4.png> feb_doubleeg_3cat_es2016d_esen_dead <https://cloud.githubusercontent.com/assets/5124244/23092172/73852778-f5c5-11e6-8414-09cb90412a0c.png> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#17219 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEdcblVK7XFafUDCHupmWdANpwt1bJUcks5rdrz4gaJpZM4LpFTw>.

[cmkuo]

@slava77 What do you mean make it to 900pre5 ? Do you mean to include this tag to GT for 900pre5 ?
I am in contact with Pierre Depasse to have this tag in 90X. Whether this tag can be included in GT for 900pre5, I think it's on the hand of @franzoni and Arun

[slava77]

On 2/18/17 6:27 AM, cmkuo wrote: @slava77 <https://github.com/slava77> What do you mean make it to 900pre5 ? Do you mean to include this tag to GT for 900pre5 ?

Yes, to have it by default in the pre5 relvals.

…

I am in contact with Pierre Depasse to have this tag in 90X. Whether this tag can be included in GT for 900pre5, I think it's on the hand of @franzoni <https://github.com/franzoni> and Arun — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#17219 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEdcbqxQgI856W0KU3SeIWBSk96K6Bgoks5rdv-4gaJpZM4LpFTw>.

[franzoni]

thanks @cmkuo for the plots,
stunning improvements !

How does the required update to conditions interplay with the code changes of this PR and the (already-merged) 90x (#17145) ?
That remains unclear to me. May you please explain ?
Is this PR and its 90x counterpart #17145 de facto inconsistent, and needing the an update to conditions, in order to be as good as the plots you show ?

The procedure to update a tag in GT is described here: https://twiki.cern.ch/twiki/bin/viewauth/CMS/AlCaDB#Policy_about_updates_in_the_GTs - @depasse is familiar with it all.
Integration in 90x can happen at the next PR by AlCa, which I am planning to use primarily for the morphing of the 2017 MC into HCAL plan1 (and could pick up the ES tag).

When you announce the tag, can you please explain, in comparison to the tag used in the offline queues up to now:

• the change in IOV sctructure, where an additional IOV is added (is that the run interval with the ES inefficiencies) with since 272760
• for the common IOV's that follow the newly introduced one, the payloads change; is that the result of the re-measurement of legacy ? If what was the case, we would need to discuss how to handle those payloads: we are purposefully not deploying any legacy-related condition updates in 90x/80x quite yet, we'll do once everything is validated. Please help us understand what the chances you propose are, we'll find a way of complementing this PR (and the 90x counterpart dead ES regions with corrected energy for each plane. Fixes inconsistency in regression training #17145 ) once we know that's cleared

[slava77]

On 2/18/17 3:52 PM, Giovanni Franzoni wrote: thanks @cmkuo <https://github.com/cmkuo> for the plots, stunning improvements ! How does the required update to conditions interplay with the code changes of this PR and the (already-merged) 90x (#17145 <#17145>) ? That remains unclear to me. May you please explain ? Is this PR and its 90x counterpart #17145 <#17145> de facto inconsistent, and needing the an update to conditions, in order to be as good as the plots you show ?

[not to be taken as a response for the EGM group]. This 80X PR is used for a discussion that would justify this PR to be merged in 80X. A production release (80X) can change only once the requested change is validated in a development (90X) release. The validation in 90X should include not just the #17145 code change but also the associated GT/payloads change that enable the code changes (current 90X payloads have values that make no difference between the code from #17145 and before it, virtually disabling the usefulness of the merged 90X PR).

…

The procedure to update a tag in GT is described here: https://twiki.cern.ch/twiki/bin/viewauth/CMS/AlCaDB#Policy_about_updates_in_the_GTs - @depasse <https://github.com/depasse> is familiar with it all. Integration in 90x can happen at the next PR by AlCa, which I am planning to use primarily for the morphing of the 2017 MC into HCAL plan1 (and could pick up the ES tag). When you announce the tag, can you please explain, in comparison to the tag used in the offline queues up to now: * the change in IOV sctructure <https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/tags/ESEEIntercalibConstants_Legacy2016_v3/ESEEIntercalibConstants_LG_offline_data_default>, where an additional IOV is added (is that the run interval with the ES inefficiencies) with since 272760 * for the common IOV's that follow the newly introduced one, the payloads change; is that the result of the re-measurement of legacy ? If what was the case, we would need to discuss how to handle those payloads: we are purposefully not deploying any legacy-related condition updates in 90x/80x quite yet, we'll do once everything is validated. Please help us understand what the chances you propose are, we'll find a way of complementing this PR (and the 90x counterpart #17145 <#17145> ) once we know that's cleared — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#17219 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEdcbi6hYEOjpbh81VDIkZWSkb4HA3QRks5rd4QggaJpZM4LpFTw>.

[franzoni]

Thanks @slava77 , helpful clarifications.

@cmkuo @depasse
I've learnt there's 1-2 days before the HCAL plan1 ingredients will become available in 90x for integration (I won't want to have other items to integrate together with HCAL plan1, since it'll be complex and priority for 900_pre5 as well as MWGR2).
If you can address the questions above and you can queue ESEEIntercalibConstants_Legacy2016_v3 where you propose to deploy it, we can exploit such time window to get it in autoCond.py

[cmkuo]

@franzoni

I would like to add something on top of what Slava said.
I would like to have this fix in legacy 80X CMSSW to cure the dead ES region.
If this fix is back ported to 80X for legacy re-reco, this tag will be the one for legacy re-reco for EE-ES inter-calibration.

ok, we will add the explanation when the tag is announced.
But anyway let me explain it here first.

In this new tag, there are three IOVs for 2016 data and include not only the dead region calibration but also the data/MC discrepancy for 2016 G and H as I presented in AlCa meeting on Feb 1st.
The first IOV covers 2016 B-F.
The second IOV is for 2016G.
The third IOV is for 2016H.
This tag should be used together with ESChannelStatus_V03_offline and ESIntercalibConstants_Run1_Run2_V07_offline.
So these are the tags for legacy re-reco.

If you do not want to deploy the legacy-related condition updates in 90X/80X now,
we need @depasse to take the current ESEEIntercalibConstants in 90X and just
change the payloads for the dead ES region calibration. In this case, the IOV structure
will not be changed.

[depasse]

Hi, ESEEIntercalibConstants_Legacy2016_v3 (4 IOVs, 3 for 2016, as explained in the request comment) was requested this morning for 80X and 90X. I don't know what you mean by "just change the payloads for the dead ES region calibration". Actually is included : ESEEIntercalibConstants_LG_offline_data_default with 3 IOVs (without Run2016B-F). Cheers. Pierre

…

________________________________ From: cmkuo [notifications@github.com] Sent: 19 February 2017 15:00 To: cms-sw/cmssw Cc: Pierre Depasse; Mention Subject: Re: [cms-sw/cmssw] dead ES regions with correct energy each plane (#17219) @franzoni<https://github.com/franzoni> I would like to add something on top of what Slava said. I would like to have this fix in legacy 80X CMSSW to cure the dead ES region. If this fix is back ported to 80X for legacy re-reco, this tag will be the one for legacy re-reco for EE-ES inter-calibration. ok, we will add the explanation when the tag is announced. But anyway let me explain it here first. In this new tag, there are three IOVs for 2016 data and include not only the dead region calibration but also the data/MC discrepancy for 2016 G and H as I presented in AlCa meeting on Feb 1st. The first IOV covers 2016 B-F. The second IOV is for 2016G. The third IOV is for 2016H. This tag should be used together with ESChannelStatus_V03_offline and ESIntercalibConstants_Run1_Run2_V07_offline. So these are the tags for legacy re-reco. If you do not want to deploy the legacy-related condition updates in 90X/80X now, we need @depasse<https://github.com/depasse> to take the current ESEEIntercalibConstants in 90X and just change the payloads for the dead ES region calibration. In this case, the IOV structure will not be changed. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#17219 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AEaCW8wAm1Dla4khAvzHYY5KnagWUe6vks5reEsYgaJpZM4LpFTw>.

[cmkuo]

@depasse since @franzoni (aka ALCA) does not want to deploy legacy re-reco to 80X/90X now, we need to do a minimum payload update. In this case, we should only update the values relevant to dead region calibration. @franzoni please let us know if you want such tag so that I can tell @depasse what to change.

[cmkuo]

@franzoni can you please take a look at the previous comment ?

[franzoni]

@cmkuo
taken note.

Thanks for the explanations.

For 90x, to start with:
are these two

• ESChannelStatus_V03_offline
• ESIntercalibConstants_Run1_Run2_V07_offline

the only updates needed to deploy the dead plane behaviour and guarantee consistency? What role does ESEEIntercalibConstants_LG_offline_data_default play ?

@cmkuo maybe we can have a brief skype, to clear up the details ?

[cmkuo]

@slava77 @depasse @argiro @franzoni @shervin86
I just had a Skype chat with Giovanni and Arun. The summary is as follows.

1. We need ESChannelStatus to trigger the dead region calibration. The ESChannelStatus tag (ESChannelStatus_V03_offline) with the channel status history in 2016 was already deployed in 2016
2. However, the dead region calibration has never been in reconstruction even though the ESChannelStatus is up-to-date because the software is not updated and also the calibration constants in ESEEIntercalibConstants are set to 1 which is not correct
3. ESEEIntercalibConstants_Legacy2016_v3 includes the correct constants for the dead ES region calibration, but also the constants for the active region for legacy re-reco
4. ALCA convenors agreed to deploy ESEEIntercalibConstants_Legacy2016_v3 although it kind of violates their policy. However, if we try to remove legacy re-reco constants and make a new tag, it will take time to produce a new tag and do another validation, and may generate confusion among different parties.
5. ESIntercalibConstants_Run1_Run2_V07_offline will not be deployed for 90X for the time being because it contains the constants for legacy re-reco and whether we include it or not will not affect the dead region calibration. In addition, as soon as all legacy conditions are accepted by ALCA in the near future, it will be embraced to 90X and 80X.

[franzoni]

Thanks @cmkuo
We'll integrate in 90x after the plan1 hcal will be done

[cmkuo]

@slava77 will you include this fix to the 80X CMSSW for legacy re-reco soon ?

[slava77]

On 4/5/17 1:33 PM, cmkuo wrote: @slava77 <https://github.com/slava77> will you include this fix to the 80X CMSSW for legacy re-reco soon ?

that's the plan

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#17219 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEdcbieXS4J8G0ZuEnWfcQ90dvJbI73pks5rs3wMgaJpZM4LpFTw>.

-