Mustache SuperCluster Regression for 800 #13000

matteosan1 · 2016-01-19T21:08:48Z

The PR updates the online and offline regression for mustache SC.
At the moment the corrections are off by default due to the useRegression flag set to false.
Once activated:

Offline regression impact only pixel-matching since it is the only place where the energy is taken directly from the mustache supercluster.
Online regression represent the "highest" level energy corrections used at HLT for electrons and photons.
We plan to switch the corrections on once the payloads will be integrated into the DB.

cmsbuild · 2016-01-19T21:09:07Z

A new Pull Request was created by @matteosan1 (Matteo Sani) for CMSSW_8_0_X.

It involves the following packages:

RecoEcal/EgammaClusterAlgos
RecoEcal/EgammaClusterProducers
RecoEgamma/EgammaTools

@cmsbuild, @cvuosalo, @davidlange6, @slava77 can you please review it and eventually sign? Thanks.
@Sam-Harper, @argiro, @lgray this is something you requested to watch as well.
@slava77, @Degano, @smuzaffar you are the release manager for this.

Following commands in first line of a comment are recognized

+1|approve[d]|sign[ed]: L1/L2's to approve it
-1|reject[ed]: L1/L2's to reject it
assign <category>[,<category>[,...]]: L1/L2's to request signatures from other categories
unassign <category>[,<category>[,...]]: L1/L2's to remove signatures from other categories
hold: L1/all L2's/release manager to mark it as on hold
unhold: L1/user who put this PR on hold
merge: L1/release managers to merge this request
[@cmsbuild,] please test: L1/L2 and selected users to start jenkins tests
[@cmsbuild,] please test with cms-sw/cmsdist#<PR>: L1/L2 and selected users to start jenkins tests using externals from cmsdist

slava77 · 2016-01-19T22:32:08Z

RecoEgamma/EgammaTools/src/SCEnergyCorrectorSemiParm.cc

@@ -0,0 +1,316 @@
+//#include <TFile.h>
+#include "../interface/SCEnergyCorrectorSemiParm.h"


relative paths are not allowed

slava77 · 2016-01-19T22:38:34Z

@cmsbuild please test

cmsbuild · 2016-01-19T22:38:52Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/10595/console

cmsbuild · 2016-01-19T23:43:46Z

-1
Tested at: 73373dc
When I ran the RelVals I found an error in the following worklfows:
134.911 step2

runTheMatrix-results/134.911_RunSinglePh2015D+RunSinglePh2015D+HLTDR2_25ns+RECODR2_25nsreHLT+HARVESTDR2_25nsreHLT/step2_RunSinglePh2015D+RunSinglePh2015D+HLTDR2_25ns+RECODR2_25nsreHLT+HARVESTDR2_25nsreHLT.log
----- Begin Fatal Exception 19-Jan-2016 23:54:31 CET-----------------------
An exception of category 'Configuration' occurred while
   [0] Constructing the EventProcessor
   [1] Validating configuration of module: class=PFECALSuperClusterProducer label='hltParticleFlowSuperClusterECALL1Seeded'
Exception Message:
Illegal parameters found in configuration.  The parameters are named:
 'regressionKeyEB'
 'regressionKeyEE'
You could be trying to use parameter names that are not
allowed for this plugin or they could be misspelled.
----- End Fatal Exception -------------------------------------------------

135.4 step1

runTheMatrix-results/135.4_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS/step1_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS.log
----- Begin Fatal Exception 19-Jan-2016 23:59:08 CET-----------------------
An exception of category 'Configuration' occurred while
   [0] Constructing the EventProcessor
   [1] Validating configuration of module: class=PFECALSuperClusterProducer label='hltParticleFlowSuperClusterECALL1Seeded'
Exception Message:
Illegal parameters found in configuration.  The parameters are named:
 'regressionKeyEB'
 'regressionKeyEE'
You could be trying to use parameter names that are not
allowed for this plugin or they could be misspelled.
----- End Fatal Exception -------------------------------------------------

1306.0 step2

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step2_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log
----- Begin Fatal Exception 20-Jan-2016 00:00:53 CET-----------------------
An exception of category 'Configuration' occurred while
   [0] Constructing the EventProcessor
   [1] Validating configuration of module: class=PFECALSuperClusterProducer label='hltParticleFlowSuperClusterECALL1Seeded'
Exception Message:
Illegal parameters found in configuration.  The parameters are named:
 'regressionKeyEB'
 'regressionKeyEE'
You could be trying to use parameter names that are not
allowed for this plugin or they could be misspelled.
----- End Fatal Exception -------------------------------------------------

1330.0 step2

runTheMatrix-results/1330.0_ZMM_13+ZMM_13+DIGIUP15+RECOUP15+HARVESTUP15/step2_ZMM_13+ZMM_13+DIGIUP15+RECOUP15+HARVESTUP15.log
----- Begin Fatal Exception 20-Jan-2016 00:06:04 CET-----------------------
An exception of category 'Configuration' occurred while
   [0] Constructing the EventProcessor
   [1] Validating configuration of module: class=PFECALSuperClusterProducer label='hltParticleFlowSuperClusterECALL1Seeded'
Exception Message:
Illegal parameters found in configuration.  The parameters are named:
 'regressionKeyEB'
 'regressionKeyEE'
You could be trying to use parameter names that are not
allowed for this plugin or they could be misspelled.
----- End Fatal Exception -------------------------------------------------

25202.0 step2

runTheMatrix-results/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25/step2_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25.log
----- Begin Fatal Exception 20-Jan-2016 00:13:02 CET-----------------------
An exception of category 'Configuration' occurred while
   [0] Constructing the EventProcessor
   [1] Validating configuration of module: class=PFECALSuperClusterProducer label='hltParticleFlowSuperClusterECALL1Seeded'
Exception Message:
Illegal parameters found in configuration.  The parameters are named:
 'regressionKeyEB'
 'regressionKeyEE'
You could be trying to use parameter names that are not
allowed for this plugin or they could be misspelled.
----- End Fatal Exception -------------------------------------------------

50202.0 step2

runTheMatrix-results/50202.0_TTbar_13+TTbar_13+DIGIUP15_PU50+RECOUP15_PU50+HARVESTUP15_PU50/step2_TTbar_13+TTbar_13+DIGIUP15_PU50+RECOUP15_PU50+HARVESTUP15_PU50.log
----- Begin Fatal Exception 20-Jan-2016 00:14:50 CET-----------------------
An exception of category 'Configuration' occurred while
   [0] Constructing the EventProcessor
   [1] Validating configuration of module: class=PFECALSuperClusterProducer label='hltParticleFlowSuperClusterECALL1Seeded'
Exception Message:
Illegal parameters found in configuration.  The parameters are named:
 'regressionKeyEB'
 'regressionKeyEE'
You could be trying to use parameter names that are not
allowed for this plugin or they could be misspelled.
----- End Fatal Exception -------------------------------------------------

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13000/10595/summary.html

cmsbuild · 2016-01-20T09:35:34Z

Pull request #13000 was updated. @cmsbuild, @cvuosalo, @davidlange6, @slava77 can you please check and sign again.

slava77 · 2016-01-20T10:21:23Z

@cmsbuild please test

cmsbuild · 2016-01-20T10:21:39Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/10598/console

slava77 · 2016-01-20T10:22:17Z

RecoEgamma/EgammaTools/src/SCEnergyCorrectorSemiParm.cc

+#include "RecoEcal/EgammaCoreTools/interface/EcalClusterTools.h"
+#include "DataFormats/EcalDetId/interface/EcalSubdetector.h"
+#include "DataFormats/VertexReco/interface/Vertex.h"
+#include "TStreamerInfo.h"


TStreamerInfo looks unnecessary as well

cmsbuild · 2016-01-20T10:35:27Z

-1
Tested at: 306067a
When I ran the RelVals I found an error in the following worklfows:
5.1 step1

runTheMatrix-results/5.1_TTbar+TTbarFS+HARVESTFS/step1_TTbar+TTbarFS+HARVESTFS.log

4.22 step2

runTheMatrix-results/4.22_RunCosmics2011A+RunCosmics2011A+RECOCOSD+ALCACOSD+SKIMCOSD+HARVESTDC/step2_RunCosmics2011A+RunCosmics2011A+RECOCOSD+ALCACOSD+SKIMCOSD+HARVESTDC.log

8.0 step1

runTheMatrix-results/8.0_BeamHalo+BeamHalo+DIGICOS+RECOCOS+ALCABH+HARVESTCOS/step1_BeamHalo+BeamHalo+DIGICOS+RECOCOS+ALCABH+HARVESTCOS.log

9.0 step1

runTheMatrix-results/9.0_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST/step1_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST.log

25.0 step1

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT/step1_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT.log

135.4 step1

runTheMatrix-results/135.4_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS/step1_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS.log

134.911 step2

runTheMatrix-results/134.911_RunSinglePh2015D+RunSinglePh2015D+HLTDR2_25ns+RECODR2_25nsreHLT+HARVESTDR2_25nsreHLT/step2_RunSinglePh2015D+RunSinglePh2015D+HLTDR2_25ns+RECODR2_25nsreHLT+HARVESTDR2_25nsreHLT.log

140.53 step2

runTheMatrix-results/140.53_RunHI2011+RunHI2011+RECOHID11+HARVESTDHI/step2_RunHI2011+RunHI2011+RECOHID11+HARVESTDHI.log

1306.0 step1

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step1_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log

1330.0 step1

runTheMatrix-results/1330.0_ZMM_13+ZMM_13+DIGIUP15+RECOUP15+HARVESTUP15/step1_ZMM_13+ZMM_13+DIGIUP15+RECOUP15+HARVESTUP15.log

101.0 step1

runTheMatrix-results/101.0_SingleElectronE120EHCAL+SingleElectronE120EHCAL/step1_SingleElectronE120EHCAL+SingleElectronE120EHCAL.log

25202.0 step1

runTheMatrix-results/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25/step1_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25.log

50202.0 step1

runTheMatrix-results/50202.0_TTbar_13+TTbar_13+DIGIUP15_PU50+RECOUP15_PU50+HARVESTUP15_PU50/step1_TTbar_13+TTbar_13+DIGIUP15_PU50+RECOUP15_PU50+HARVESTUP15_PU50.log

1000.0 step2

runTheMatrix-results/1000.0_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT/step2_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT.log

1001.0 step2

runTheMatrix-results/1001.0_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4/step2_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4.log

1003.0 step2

runTheMatrix-results/1003.0_RunMinBias2012A+RunMinBias2012A+RECODDQM+HARVESTDDQM/step2_RunMinBias2012A+RunMinBias2012A+RECODDQM+HARVESTDDQM.log

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13000/10598/summary.html

cmsbuild · 2016-02-01T23:49:38Z

+1
Tested at: 09e078a
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13000/10910/summary.html

cmsbuild · 2016-02-02T00:46:36Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13000/10910/summary.html

mmusich · 2016-02-02T08:17:07Z

+1

slava77 · 2016-02-02T16:28:21Z

@mmusich I'm hoping that there is no more update in the autoCond.py planned imminently.
If that changes, this PR will hopefully become unmergeable, but can instead override changes.
So, @matteosan1, it may be better to remove the commit changing autoCond

mmusich · 2016-02-02T16:30:50Z

@slava77 not imminent, but soon we will have to add few ingredients for the final definition 80X asymptotic scenarios, i.e. Ecal noise, Ecal Selective readout, and a bunch of other items, so yes it would be definitely better to remove the commit

slava77 · 2016-02-02T19:12:48Z

@matteosan1 which energies did you plot in #13000 (comment) ?
Are these cluster level or the final candidate (e.g. gedGsfElectron) pt or energy()?

Most of the significant changes that I observe at the high level object level (gedGsfElectron, gedPhoton, and particleFlow candidates) are all localized to pt~10 GeV which seems to be just a cluster threshold effects:
e.g. gamma pt=10 GeV gun events:

vs gamma 35 GeV sample

Similar story for gsfElectrons and PFCandidates (particleFlow) with electron or photon ID.

DQM doesn't seem to have much to offer in terms of ECAL SC reco/true energy plots.
This one in electrons (in ele 35 GeV gun sample) appears to show an improvement of a similar kind posted by Matteo

At the lower level, the supercluster energies are changing noticeably: plots from H->gammagamma (workflow 1332, no pileup)

There is a rather significant change in R9 plots (but I suspect these are plots of an incorrect variable; if the actual "good" R9 is changing: it looks like more clusters are pushed to lower R9 values, which is not so good)
here is a plot in H->gammagamma

Based on the PR description, the effects appear to be somewhat expected, but some comments are needed to confirm the behavior is expected before this can be signed off.

matteosan1 · 2016-02-03T13:02:54Z

Hi Slava,
SC energy plots look consistent to mine, I am a bit surprised of R9 and I am running a simple test to cross-check it.

slava77 · 2016-02-03T13:32:50Z

On 2/3/16 5:02 AM, Matteo Sani wrote:

Hi Slava,
SC energy plots look consistent to mine, I am a bit surprised of R9 and
I am running a simple test to cross-check it.

I recall that in the past we had R9 plot in DQM which was not computed
correctly (using inconsistent numerator and denominator).
It may be worth to check the code that makes the plot (it should be easy
given the path and the histogram name).
Someone in ECAL/EGM should fix it, if it is the case here.

—
Reply to this email directly or view it on GitHub
#13000 (comment).

matteosan1 · 2016-02-03T14:31:56Z

Hi Slava,
I don't see big issues with R9 in my plots (attached, blue is new, red is old). I'll follow up on the DQM issue with ECAL guys.

slava77 · 2016-02-03T15:08:48Z

+1

for #13000 09e078a

code changes are in line with the description (a different SC correction is applied)
jenkins test pass and comparisons with baseline show differences consistent with the local tests summarized earlier in Mustache SuperCluster Regression for 800 #13000 (comment)

@davidlange6
while it's still coherent, please merge this at your earliest.
If there are some issues and this can't be merged imminently, this PR will have to be updated to remove changes in autoCond.py file #13000 (comment)

cmsbuild · 2016-02-03T15:09:07Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_8_0_X IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @slava77, @davidlange6, @Degano, @smuzaffar

davidlange6 · 2016-02-03T16:46:59Z

+1

Mustache SuperCluster Regression for 800

cmsbuild added this to the Next CMSSW_8_0_X milestone Jan 19, 2016

cmsbuild added reconstruction-pending pending-signatures tests-pending orp-pending comparison-pending labels Jan 19, 2016

slava77 reviewed Jan 19, 2016
View reviewed changes

cmsbuild added tests-started and removed tests-pending labels Jan 19, 2016

cmsbuild added tests-rejected tests-pending and removed tests-started tests-rejected labels Jan 19, 2016

cmsbuild added tests-started and removed tests-pending labels Jan 20, 2016

slava77 reviewed Jan 20, 2016
View reviewed changes

cmsbuild added tests-rejected and removed tests-started tests-rejected labels Jan 20, 2016

cmsbuild added tests-started and removed tests-pending labels Feb 1, 2016

cmsbuild added tests-approved and removed tests-started labels Feb 1, 2016

cmsbuild added comparison-available and removed comparison-pending labels Feb 2, 2016

cmsbuild added alca-approved and removed alca-pending labels Feb 2, 2016

cmsbuild added reconstruction-approved fully-signed and removed pending-signatures reconstruction-pending labels Feb 3, 2016

cmsbuild added orp-approved and removed orp-pending labels Feb 3, 2016

cmsbuild added a commit that referenced this pull request Feb 3, 2016

Merge pull request #13000 from matteosan1/mustacheRegression_800

bcb909d

Mustache SuperCluster Regression for 800

cmsbuild merged commit bcb909d into cms-sw:CMSSW_8_0_X Feb 3, 2016

cmsbuild added a commit that referenced this pull request Feb 3, 2016

Merge pull request #13000 from matteosan1/mustacheRegression_800

243a743

matteosan1 deleted the mustacheRegression_800 branch April 3, 2016 13:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mustache SuperCluster Regression for 800 #13000

Mustache SuperCluster Regression for 800 #13000

matteosan1 commented Jan 19, 2016

cmsbuild commented Jan 19, 2016

slava77 Jan 19, 2016

slava77 commented Jan 19, 2016

cmsbuild commented Jan 19, 2016

cmsbuild commented Jan 19, 2016

cmsbuild commented Jan 20, 2016

slava77 commented Jan 20, 2016

cmsbuild commented Jan 20, 2016

slava77 Jan 20, 2016

cmsbuild commented Jan 20, 2016

cmsbuild commented Feb 1, 2016

cmsbuild commented Feb 2, 2016

mmusich commented Feb 2, 2016

slava77 commented Feb 2, 2016

mmusich commented Feb 2, 2016

slava77 commented Feb 2, 2016

matteosan1 commented Feb 3, 2016

slava77 commented Feb 3, 2016

matteosan1 commented Feb 3, 2016

slava77 commented Feb 3, 2016

cmsbuild commented Feb 3, 2016

davidlange6 commented Feb 3, 2016

		@@ -0,0 +1,316 @@
		//#include <TFile.h>
		#include "../interface/SCEnergyCorrectorSemiParm.h"

Mustache SuperCluster Regression for 800 #13000

Mustache SuperCluster Regression for 800 #13000

Conversation

matteosan1 commented Jan 19, 2016

cmsbuild commented Jan 19, 2016

slava77 Jan 19, 2016

Choose a reason for hiding this comment

slava77 commented Jan 19, 2016

cmsbuild commented Jan 19, 2016

cmsbuild commented Jan 19, 2016

cmsbuild commented Jan 20, 2016

slava77 commented Jan 20, 2016

cmsbuild commented Jan 20, 2016

slava77 Jan 20, 2016

Choose a reason for hiding this comment

cmsbuild commented Jan 20, 2016

cmsbuild commented Feb 1, 2016

cmsbuild commented Feb 2, 2016

mmusich commented Feb 2, 2016

slava77 commented Feb 2, 2016

mmusich commented Feb 2, 2016

slava77 commented Feb 2, 2016

matteosan1 commented Feb 3, 2016

slava77 commented Feb 3, 2016

matteosan1 commented Feb 3, 2016

slava77 commented Feb 3, 2016

cmsbuild commented Feb 3, 2016

davidlange6 commented Feb 3, 2016