Combined backport of PR #19205 and #19404: ootPhotons in miniAOD from AOD from either 92X or 80X inputs #19448

kmcdermo · 2017-06-27T14:42:22Z

Backport of both PR #19205 and #19404 for 92X. Both have been merged into 93X.

19205: ootPhotons in miniAOD from AOD (if ootPhotons produced during RECO step)
19404: ootPhotons in miniAOD from 80X AOD (run a era-specific customized ootPhoton RECO step, then run the PAT step to produce ootPhotons in miniAO)

cmsbuild · 2017-06-27T14:42:43Z

A new Pull Request was created by @kmcdermo (Kevin McDermott) for CMSSW_9_2_X.

It involves the following packages:

PhysicsTools/PatAlgos
RecoEgamma/EgammaPhotonProducers

@perrotta, @cmsbuild, @slava77, @monttj, @davidlange6 can you please review it and eventually sign? Thanks.
@rappoccio, @Sam-Harper, @jdolen, @imarches, @ahinzmann, @acaudron, @mmarionncern, @TaiSakuma, @gpetruc, @rafaellopesdesa, @JyothsnaKomaragiri, @nhanvtran, @gouskos, @gkasieczka, @schoef, @ferencek, @mverzett, @mariadalfonso, @lgray, @pvmulder, @seemasharmafnal this is something you requested to watch as well.
@davidlange6 you are the release manager for this.

cms-bot commands are listed here

slava77 · 2017-06-27T14:51:47Z

@kmcdermo
please edit the title of the PR to be more descriptive
Thank you.

kmcdermo · 2017-06-30T10:05:16Z

@slava77
can this PR be tested now for the backport?

perrotta · 2017-06-30T10:36:22Z

please test

cmsbuild · 2017-06-30T10:36:43Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/21023/console Started: 2017/06/30 16:28

arizzi · 2017-06-30T14:15:03Z

@kmcdermo @davidlange6 are the 92X relval already containing the ootPhotons at RECO level so this should work out of the box?

kmcdermo · 2017-06-30T14:19:06Z

@arizzi
yes, indeed, this was included:
175981a

so it will pick up the relvals from 92X with ootPhotons already at AOD.

arizzi · 2017-06-30T14:52:14Z

mhh... then why recent 93X IB (containing the other PR) fail on 92X relval with
30-Jun-2017 16:48:45 CEST Successfully opened file root://eoscms.cern.ch//eos/cms/store/relval/CMSSW_9_2_1/RelValTTbar_13/GEN-SIM-RECO/PU25ns_92X_upgrade2017_realistic_v1-v1/10000/0C07A2DB-594A-E711-A1FF-0025905A6134.root
Begin processing the 1st record. Run 1, Event 7607, LumiSection 153 at 30-Jun-2017 16:49:19.603 CEST
----- Begin Fatal Exception 30-Jun-2017 16:49:32 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
[0] Processing Event run: 1 lumi: 153 event: 7607 stream: 0
[1] Running path 'MINIAODSIMoutput_step'
[2] Prefetching for module PoolOutputModule/'MINIAODSIMoutput'
[3] Calling method for module ReducedEGProducer/'reducedEgamma'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: std::vectorreco::Photon
Looking for module label: ootPhotons
Looking for productInstanceName:

kmcdermo · 2017-06-30T14:57:22Z

@arizzi

Where is this file getting picked up? ootPhotons at AOD were added into 9_2_2, and this is a 9_2_1 sample, so that test needs to be updated.

arizzi · 2017-06-30T15:00:57Z

that was my test, not something central. so not any 92X is ok, should be >=922 (will try this)

cmsbuild · 2017-06-30T15:45:04Z

+1
Tested at: 06ec2b4
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19448/21023/summary.html

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
fc51b5b
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19448/21023/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19448/21023/git-merge-result

cmsbuild · 2017-06-30T15:45:07Z

Comparison job queued.

cmsbuild · 2017-06-30T18:31:35Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19448/21023/summary.html

Comparison Summary:

You potentially added 2 lines to the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 7 differences found in the comparisons
DQMHistoTests: Total files compared: 22
DQMHistoTests: Total histograms compared: 1813696
DQMHistoTests: Total failures: 29444
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 1784086
DQMHistoTests: Total skipped: 166
DQMHistoTests: Total Missing objects: 0
Checked 90 log files, 14 edm output root files, 22 DQM output files

slava77 · 2017-07-01T16:14:08Z

as noted in #19404 (comment)
we will need the legacy reminiAOD part combined with this PR for 92X integration.

…ation to before loops, const auto where possible, use index=-1 in EGM object loops to avoid keeping track of incrementing index

…mma_cfi, make ootPhotons a task in ootPhotonSequence_cff, import task and modify accordingly for legacy task in ootPhotonProducer_cff

…luous modifiers for legacy repro on recHits

cmsbuild · 2017-07-03T21:03:54Z

Pull request #19448 was updated. @perrotta, @cmsbuild, @slava77, @monttj, @davidlange6 can you please check and sign again.

slava77 · 2017-07-03T21:14:00Z

@kmcdermo
is this backport done, or is it still in the middle of commits and tests?

kmcdermo · 2017-07-03T21:14:41Z

@slava77
please let me test locally first to make sure I did not break anything, but otherwise these are all the commits that are needed.

kmcdermo · 2017-07-03T21:41:01Z

@slava77
Okay, tested locally on an 80X AOD file, and it worked (produced ootPhotons) .

Let me know if you want me to squash any of these commits. I just rebased this branch, then cherry-picked the seven commits from 19404 on top.

slava77 · 2017-07-03T21:50:31Z

@cmsbuild please test

cmsbuild · 2017-07-03T21:50:46Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/21109/console Started: 2017/07/03 23:51
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/21112/console Started: 2017/07/04 00:59

cmsbuild · 2017-07-04T00:29:31Z

+1
Tested at: 321d0bb
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19448/21112/summary.html

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
fc51b5b
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19448/21112/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19448/21112/git-merge-result

cmsbuild · 2017-07-04T00:29:34Z

Comparison job queued.

cmsbuild · 2017-07-04T01:54:50Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19448/21112/summary.html

Comparison Summary:

You potentially added 2 lines to the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 7 differences found in the comparisons
DQMHistoTests: Total files compared: 22
DQMHistoTests: Total histograms compared: 1813730
DQMHistoTests: Total failures: 93
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 1813470
DQMHistoTests: Total skipped: 166
DQMHistoTests: Total Missing objects: 0
Checked 90 log files, 14 edm output root files, 22 DQM output files

kmcdermo · 2017-07-05T12:13:31Z

ping on this PR

slava77 · 2017-07-06T16:46:36Z

+1

for #19448 321d0bb

changes in the code in this PR are identical to the combination of Out-of-time photons for MiniAOD #19205 06ec2b4 and ootPhotons for 80X Legacy reprocessing #19404 73da687; both are present in 930pre1 and apparently have not produced any issues there
jenkins tests pass and comparisons with baseline show differences only in the reducedEgamma reducedEBrechits, where some increment is expected; based on a copy of jenkins outputs in 136.731 I also can confirm that the expected new products are present (slimmedOOTPhotons and reducedEgamma_*OOT*Clusters)

davidlange6 · 2017-07-06T20:55:39Z

merge

cmsbuild added this to the CMSSW_9_2_X milestone Jun 27, 2017

cmsbuild added analysis-pending comparison-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Jun 27, 2017

kmcdermo changed the title ~~Backport of PR #19205~~ Backport of PR #19205 (ootPhotons for miniAOD from AOD in 92X) Jun 27, 2017

kmcdermo mentioned this pull request Jun 29, 2017

Out-of-time photons for MiniAOD #19205

Merged

cmsbuild added tests-started and removed tests-pending labels Jun 30, 2017

cmsbuild added tests-approved and removed tests-started labels Jun 30, 2017

cmsbuild added comparison-available and removed comparison-pending labels Jun 30, 2017

Add out-of-time photons to miniAOD

91427fc

kmcdermo added 3 commits July 3, 2017 23:01

make changes as suggested for ReducedEGProducer: move index initializ…

5926a58

…ation to before loops, const auto where possible, use index=-1 in EGM object loops to avoid keeping track of incrementing index

make changes to configs as requested: remove type spec. in reducedEga…

ee00be0

…mma_cfi, make ootPhotons a task in ootPhotonSequence_cff, import task and modify accordingly for legacy task in ootPhotonProducer_cff

move changes to ootPhotonTask to ootPhotonSequence_cff, remove superf…

321d0bb

…luous modifiers for legacy repro on recHits

kmcdermo force-pushed the oot-photons-miniaod branch from 06ec2b4 to 321d0bb Compare July 3, 2017 21:02

cmsbuild added comparison-pending tests-pending and removed comparison-available tests-approved labels Jul 3, 2017

kmcdermo changed the title ~~Backport of PR #19205 (ootPhotons for miniAOD from AOD in 92X)~~ Combined backport of PR #19205 and #19404: ootPhotons in miniAOD from AOD from either 92X or 80X inputs Jul 3, 2017

cmsbuild added tests-started and removed tests-pending labels Jul 3, 2017

cmsbuild added tests-approved and removed tests-started labels Jul 4, 2017

cmsbuild added comparison-available and removed comparison-pending labels Jul 4, 2017

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Jul 6, 2017

cmsbuild merged commit d38320d into cms-sw:CMSSW_9_2_X Jul 6, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Combined backport of PR #19205 and #19404: ootPhotons in miniAOD from AOD from either 92X or 80X inputs #19448

Combined backport of PR #19205 and #19404: ootPhotons in miniAOD from AOD from either 92X or 80X inputs #19448

kmcdermo commented Jun 27, 2017 •

edited

cmsbuild commented Jun 27, 2017

slava77 commented Jun 27, 2017

kmcdermo commented Jun 30, 2017

perrotta commented Jun 30, 2017

cmsbuild commented Jun 30, 2017 •

edited

arizzi commented Jun 30, 2017

kmcdermo commented Jun 30, 2017

arizzi commented Jun 30, 2017

kmcdermo commented Jun 30, 2017

arizzi commented Jun 30, 2017

cmsbuild commented Jun 30, 2017

cmsbuild commented Jun 30, 2017

cmsbuild commented Jun 30, 2017

slava77 commented Jul 1, 2017

cmsbuild commented Jul 3, 2017

slava77 commented Jul 3, 2017

kmcdermo commented Jul 3, 2017

kmcdermo commented Jul 3, 2017

slava77 commented Jul 3, 2017

cmsbuild commented Jul 3, 2017 •

edited

cmsbuild commented Jul 4, 2017

cmsbuild commented Jul 4, 2017

cmsbuild commented Jul 4, 2017

kmcdermo commented Jul 5, 2017

slava77 commented Jul 6, 2017

davidlange6 commented Jul 6, 2017

Combined backport of PR #19205 and #19404: ootPhotons in miniAOD from AOD from either 92X or 80X inputs #19448

Combined backport of PR #19205 and #19404: ootPhotons in miniAOD from AOD from either 92X or 80X inputs #19448

Conversation

kmcdermo commented Jun 27, 2017 • edited

cmsbuild commented Jun 27, 2017

slava77 commented Jun 27, 2017

kmcdermo commented Jun 30, 2017

perrotta commented Jun 30, 2017

cmsbuild commented Jun 30, 2017 • edited

arizzi commented Jun 30, 2017

kmcdermo commented Jun 30, 2017

arizzi commented Jun 30, 2017

kmcdermo commented Jun 30, 2017

arizzi commented Jun 30, 2017

cmsbuild commented Jun 30, 2017

cmsbuild commented Jun 30, 2017

cmsbuild commented Jun 30, 2017

slava77 commented Jul 1, 2017

cmsbuild commented Jul 3, 2017

slava77 commented Jul 3, 2017

kmcdermo commented Jul 3, 2017

kmcdermo commented Jul 3, 2017

slava77 commented Jul 3, 2017

cmsbuild commented Jul 3, 2017 • edited

cmsbuild commented Jul 4, 2017

cmsbuild commented Jul 4, 2017

cmsbuild commented Jul 4, 2017

kmcdermo commented Jul 5, 2017

slava77 commented Jul 6, 2017

davidlange6 commented Jul 6, 2017

kmcdermo commented Jun 27, 2017 •

edited

cmsbuild commented Jun 30, 2017 •

edited

cmsbuild commented Jul 3, 2017 •

edited