Add full5x5 shower shape block to non-GED photons producer #12085

harmonicoscillator · 2015-10-24T20:41:06Z

HI noticed that for photons produced with the "standard" photon producer, the full5x5 shower shape info was missing.

HI absolutely needs this info for photon ID purposes in run2 analyses, as we plan on using "standard" photons for run2 analyses.

@ttrk @mandrenguyen @yetkinyilmaz

We would like to request that this be included in the release used for HI data taking, potentially 7_5_3_patch2 or something similar. Apologies for not catching this error earlier.

cmsbuild · 2015-10-24T20:41:17Z

A new Pull Request was created by @richard-cms (R. Alex Barbieri) for CMSSW_8_0_X.

Add full5x5 shower shape block to non-GED photons producer

It involves the following packages:

RecoEgamma/EgammaPhotonProducers

@cmsbuild, @cvuosalo, @slava77 can you please review it and eventually sign? Thanks.
@Sam-Harper, @lgray this is something you requested to watch as well.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.
If you are a L2 or a release manager you can ask for tests by saying 'please test' in the first line of a comment.
@Degano you are the release manager for this.
You can merge this pull request by typing 'merge' in the first line of your comment.

lgray · 2015-10-24T20:43:40Z

Are there non-ged photons that are based off of particle flow clusters? If
not then this PR is not necessary (since full5x5 and 'regular' shower
shapes are the same if there are no fractions)

(Sent from my Nexus 6)
On Oct 24, 2015 10:41 PM, "cmsbuild" notifications@github.com wrote:

A new Pull Request was created by @richard-cms
https://github.com/richard-cms (R. Alex Barbieri) for CMSSW_8_0_X.

Add full5x5 shower shape block to non-GED photons producer

It involves the following packages:

RecoEgamma/EgammaPhotonProducers

@cmsbuild https://github.com/cmsbuild, @cvuosalo
https://github.com/cvuosalo, @slava77 https://github.com/slava77 can
you please review it and eventually sign? Thanks.
@Sam-Harper https://github.com/Sam-Harper, @lgray
https://github.com/lgray this is something you requested to watch as
well.
You can sign-off by replying to this message having '+1' in the first line
of your reply.
You can reject by replying to this message having '-1' in the first line
of your reply.
If you are a L2 or a release manager you can ask for tests by saying
'please test' in the first line of a comment.
@Degano https://github.com/degano you are the release manager for this.
You can merge this pull request by typing 'merge' in the first line of
your comment.

—
Reply to this email directly or view it on GitHub
#12085 (comment).

harmonicoscillator · 2015-10-24T20:44:49Z

There were no errors reported during workflows 140.2 or 33.0.

The actual patch is simply copying the relevant block from PhotonProducerGED, which was added in this commit: 934681b

harmonicoscillator · 2015-10-24T20:47:00Z

@lgray
We saw a significant difference between new sigmaIetatIeta and full5x5 sigmaIetaIeta when testing this patch. We are running two photon reconstructions in HI RECO, one "standard" which is not PF cluster based, and the other which is GED. I think that the new sigmaIetaIeta calculation still attempts to account for the energy fraction (using the GED photon info) during the standard photon RECO.

lgray · 2015-10-24T20:53:12Z

Non-pfcluster-based photons do not use fractions, there is no way to even know about or reconcile with them. Pf and other dedicated em clusterings are incredibly different.

The difference you saw comes from something else, or your explanation of the workflow is incorrect. It's likely inconsistent RecHit flagging or something similar. I encourage that you understand this a bit more before filling these variables with something unknown.

harmonicoscillator · 2015-10-24T21:26:32Z

@lgray, okay we'll try to track down what's actually going on here.

harmonicoscillator · 2015-10-25T03:58:06Z

@lgray
I'm getting very mystified trying to find out how this is happening. We're using identical RecHit flagging for both full5x5 and new sigmaIetaIeta:
https://github.com/richard-cms/cmssw/blob/Full5x5ShowerShapes_76X/RecoEgamma/EgammaPhotonProducers/src/PhotonProducer.cc#L356
https://github.com/richard-cms/cmssw/blob/Full5x5ShowerShapes_76X/RecoEgamma/EgammaPhotonProducers/src/PhotonProducer.cc#L370

However, we really do a difference in the sigmaIetaIeta values. Here's a short comparison on 5 events:

Row * Instance * full5x5_sigmaIetaIeta * sigmaIetaIeta *

   0 \*        0 \* 0.0095054 \* 0.0094380 *

   0 \*        1 \* 0.0240499 \* 0.0240378 *

   1 \*        0 \* 0.0087834 \* 0.0090450 *

   2 \*        0 \* 0.0094981 \* 0.0095428 *

   2 \*        1 \* 0.0085439 \* 0.0069553 *

   3 \*        0 \* 0.0088339 \* 0.0086395 *

   3 \*        1 \* 0.0188216 \* 0.0099303 *

   3 \*        2 \* 0.0240541 \* 0.0242143 *

   4 \*        0 \* 0.0098860 \* 0.0100173 *

If I take a look in the bowels of the island clustering algorithm, I see lines like this one:
https://github.com/cms-sw/cmssw/blob/CMSSW_8_0_X/RecoEcal/EgammaClusterAlgos/src/IslandClusterAlgo.cc#L139
which explicitly set the fraction to 1 for the island basic clusters, as you said.

What could possibly be different between these two sigmaIetaIeta calculations to cause this difference?

slava77 · 2015-10-25T19:11:07Z

@cmsbuild please test

cmsbuild · 2015-10-25T19:16:23Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/9180/console

cmsbuild · 2015-10-25T20:56:09Z

+1
Tested at: f9ac785
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-12085/9180/summary.html

cmsbuild · 2015-10-25T21:54:28Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-12085/9180/summary.html

cvuosalo · 2015-10-27T16:22:15Z

RecoEgamma/EgammaPhotonProducers/src/PhotonProducer.cc

@@ -359,6 +359,19 @@ void PhotonProducer::fillPhotonCollection(edm::Event& evt,
    float sigmaIetaIeta = sqrt(locCov[0]);
    float r9 =e3x3/(scRef->rawEnergy());

+    float full5x5_maxXtal =   noZS::EcalClusterTools::eMax( *(scRef->seed()), &(*hits) );


Before dereferencing hits, please check it is non-null. There is a possibility that hits would equal 0.

So this code was copied directly from https://github.com/CmsHI/cmssw/blob/forest_CMSSW_7_5_3_patch1/RecoEgamma/EgammaPhotonProducers/src/GEDPhotonProducer.cc#L502 which also does not checkout for null so I didn't think about this too hard.

I can add a non-null check to this file, but I'm loathe to change anything in GEDProducer to prevent PR creep.

cvuosalo · 2015-10-27T18:53:58Z

@lgray :
Could you please give feedback about whether you are satisfied with this PR or still have questions about it?

harmonicoscillator · 2015-10-28T20:35:20Z

We spoke with Lindsey/Matteo some offline and are currently attempting some tests to get to the bottom of why the full5x5 and non-full5x5 calculations have differences. We will hopefully have some answers this week.

harmonicoscillator · 2015-10-31T18:11:33Z

Okay, I believe I have found the source of the discrepancy between the noZS (full5x5) and ZS (non-full5x5) results, and it is in the "getFraction" EcalClusterTools function: https://github.com/cms-sw/cmssw/blob/CMSSW_8_0_X/RecoEcal/EgammaCoreTools/interface/EcalClusterTools.h#L257

If noZS is false (non-full5x5), then the getFraction function returns '0' for rechits that it cannot find. The noZS version returns 1.0 in those cases (is this a bug, or intended?). 'getFraction' is called by matrixEnergy, and matrixEnergy is in turn called by e5x5, and the difference in e5x5 causes the difference in sigmaIetaIeta. If I force getFraction to return 1.0 always (the noZS behavior) the noZS and ZS results are the same.

Unfortunately, I do not know if this behavior is a bug or is intended. @lgray or @matteosan1 do you have any ideas on:

What are these rechits that return a fraction 0? They must come from outside the basiccluster?
Is this different behavior for ZS and noZS when the rechit cannot be found intended?

Any thoughts are appreciated. @yenjie, and @ttrk are also interested.

cmsbuild · 2015-11-04T18:56:48Z

Pull request #12085 was updated. @cmsbuild, @cvuosalo, @slava77 can you please check and sign again.

cvuosalo · 2015-11-04T19:05:40Z

@cmsbuild please test

cmsbuild · 2015-11-04T19:06:40Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/9465/console

cvuosalo · 2015-11-04T19:08:24Z

RecoEgamma/EgammaPhotonProducers/src/PhotonProducer.cc

    }
+    if(!hits) continue;


@richard-cms: Better style would be if (hits == 0), since hits is not a boolean, but this way is OK, too.

Okay I switched it to hits == 0.

cmsbuild · 2015-11-04T19:16:43Z

Pull request #12085 was updated. @cmsbuild, @cvuosalo, @slava77 can you please check and sign again.

cmsbuild · 2015-11-04T20:47:37Z

+1
Tested at: 4cca203
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-12085/9465/summary.html

cmsbuild · 2015-11-04T23:02:53Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-12085/9465/summary.html

cvuosalo · 2015-11-04T23:28:42Z

+1

For #12085 3986852

For Heavy Ions, adding full5x5 shower shape to non-GED photons. #12086 and #12087 are the 75X and 76X versions of this PR, and the former has already been approved by Reco.

The code changes are satisfactory, and Jenkins tests against baseline CMSSW_8_0_X_2015-11-04-1100 show no significant differences, except for the expected addition of the shower shapes.

cmsbuild · 2015-11-04T23:31:32Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_8_0_X IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @Degano, @smuzaffar

davidlange6 · 2015-11-05T14:45:03Z

+1

Add full5x5 shower shape block to non-GED photons producer

Add full5x5 shower shape block to non-GED photons producer

f9ac785

cmsbuild added this to the Next CMSSW_8_0_X milestone Oct 24, 2015

cmsbuild added reconstruction-pending pending-signatures tests-pending orp-pending comparison-pending labels Oct 24, 2015

This was referenced Oct 24, 2015

Add full5x5 shower shape block to non-GED photons producer #12086

Merged

Add full5x5 shower shape block to non-GED photons producer #12087

Closed

cmsbuild added tests-started and removed tests-pending labels Oct 25, 2015

cmsbuild added tests-approved and removed tests-started labels Oct 25, 2015

cmsbuild added comparison-available and removed comparison-pending labels Oct 25, 2015

cvuosalo reviewed Oct 27, 2015
View reviewed changes

cmsbuild added tests-pending comparison-pending labels Nov 4, 2015

cvuosalo reviewed Nov 4, 2015
View reviewed changes

cmsbuild added tests-started and removed tests-pending labels Nov 4, 2015

add some null protection

3986852

cmsbuild added tests-pending and removed tests-started labels Nov 4, 2015

cmsbuild added tests-approved and removed tests-pending labels Nov 4, 2015

cmsbuild added comparison-available and removed comparison-pending labels Nov 4, 2015

cmsbuild added reconstruction-approved fully-signed and removed pending-signatures reconstruction-pending labels Nov 4, 2015

cmsbuild added orp-approved and removed orp-pending labels Nov 5, 2015

cmsbuild added a commit that referenced this pull request Nov 5, 2015

Merge pull request #12085 from richard-cms/Full5x5ShowerShapes_76X

94484af

Add full5x5 shower shape block to non-GED photons producer

cmsbuild merged commit 94484af into cms-sw:CMSSW_8_0_X Nov 5, 2015

cvuosalo mentioned this pull request Nov 6, 2015

Fix race condition in DAQ modules for 80X (make new PR) #12277

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add full5x5 shower shape block to non-GED photons producer #12085

Add full5x5 shower shape block to non-GED photons producer #12085

harmonicoscillator commented Oct 24, 2015

cmsbuild commented Oct 24, 2015

lgray commented Oct 24, 2015

harmonicoscillator commented Oct 24, 2015

harmonicoscillator commented Oct 24, 2015

lgray commented Oct 24, 2015

harmonicoscillator commented Oct 24, 2015

harmonicoscillator commented Oct 25, 2015

slava77 commented Oct 25, 2015

cmsbuild commented Oct 25, 2015

cmsbuild commented Oct 25, 2015

cmsbuild commented Oct 25, 2015

cvuosalo Oct 27, 2015

harmonicoscillator Nov 4, 2015

cvuosalo commented Oct 27, 2015

harmonicoscillator commented Oct 28, 2015

harmonicoscillator commented Oct 31, 2015

cmsbuild commented Nov 4, 2015

cvuosalo commented Nov 4, 2015

cmsbuild commented Nov 4, 2015

cvuosalo Nov 4, 2015

harmonicoscillator Nov 4, 2015

cmsbuild commented Nov 4, 2015

cmsbuild commented Nov 4, 2015

cmsbuild commented Nov 4, 2015

cvuosalo commented Nov 4, 2015

cmsbuild commented Nov 4, 2015

davidlange6 commented Nov 5, 2015

Add full5x5 shower shape block to non-GED photons producer #12085

Add full5x5 shower shape block to non-GED photons producer #12085

Conversation

harmonicoscillator commented Oct 24, 2015

cmsbuild commented Oct 24, 2015

lgray commented Oct 24, 2015

harmonicoscillator commented Oct 24, 2015

harmonicoscillator commented Oct 24, 2015

lgray commented Oct 24, 2015

harmonicoscillator commented Oct 24, 2015

harmonicoscillator commented Oct 25, 2015

slava77 commented Oct 25, 2015

cmsbuild commented Oct 25, 2015

cmsbuild commented Oct 25, 2015

cmsbuild commented Oct 25, 2015

cvuosalo Oct 27, 2015

Choose a reason for hiding this comment

harmonicoscillator Nov 4, 2015

Choose a reason for hiding this comment

cvuosalo commented Oct 27, 2015

harmonicoscillator commented Oct 28, 2015

harmonicoscillator commented Oct 31, 2015

cmsbuild commented Nov 4, 2015

cvuosalo commented Nov 4, 2015

cmsbuild commented Nov 4, 2015

cvuosalo Nov 4, 2015

Choose a reason for hiding this comment

harmonicoscillator Nov 4, 2015

Choose a reason for hiding this comment

cmsbuild commented Nov 4, 2015

cmsbuild commented Nov 4, 2015

cmsbuild commented Nov 4, 2015

cvuosalo commented Nov 4, 2015

cmsbuild commented Nov 4, 2015

davidlange6 commented Nov 5, 2015