Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ootPhotons for 80X Legacy reprocessing #19404

Merged
merged 7 commits into from Jul 3, 2017

Conversation

kmcdermo
Copy link
Contributor

@kmcdermo kmcdermo commented Jun 22, 2017

This PR sits on top of PR #19205 (ootPhotons in miniAOD) with only one new commit for the legacy sequence.

I was able to get this to run locally and produce the ootPhoton collection from 80X AOD (2016G, singlephoton). However, after rebasing, I cannot complete the runTheMatrix.py -l limited -i all because of the same problems with #19205, namely some PhaseII sequence breaking the workflow.

I had to touch a number of files to accommodate the differences between the new 92X interfaces and what is available in the 80X AOD. A number of products have to be regenerated for this sequence to run (see the customization in the ootPhotonProducer_cff.py). I tried following what was done in #19304 for customizing the configuration, as well as following along #19264 for minimally changing any code.

EDIT: I forgot to mention I built this on top of an IB: CMSSW_9_2_X_2017-06-21-2300, so as to include those two PRs listed above.

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @kmcdermo (Kevin McDermott) for master.

It involves the following packages:

PhysicsTools/PatAlgos
RecoEcal/EgammaClusterProducers
RecoEgamma/EgammaIsolationAlgos
RecoEgamma/EgammaPhotonProducers
RecoEgamma/PhotonIdentification
RecoParticleFlow/PFClusterProducer

@perrotta, @cmsbuild, @slava77, @monttj, @davidlange6 can you please review it and eventually sign? Thanks.
@rappoccio, @gouskos, @TaiSakuma, @argiro, @seemasharmafnal, @mmarionncern, @imarches, @ahinzmann, @acaudron, @lgray, @jdolen, @ferencek, @Sam-Harper, @rafaellopesdesa, @nhanvtran, @gkasieczka, @schoef, @JyothsnaKomaragiri, @mverzett, @cbernet, @gpetruc, @mariadalfonso, @pvmulder, @bachtis this is something you requested to watch as well.
@davidlange6 you are the release manager for this.

cms-bot commands are listed here

@slava77
Copy link
Contributor

slava77 commented Jun 22, 2017

@cmsbuild please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 22, 2017

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/20843/console Started: 2017/06/22 15:27

@kmcdermo
Copy link
Contributor Author

@slava77 these tests will fail for the same reason as #19205, namely some PhaseII tracking sequence.

@cmsbuild
Copy link
Contributor

Comparison job queued.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19404/20843/summary.html

Comparison Summary:

  • You potentially added 3 lines to the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 7 differences found in the comparisons
  • DQMHistoTests: Total files compared: 22
  • DQMHistoTests: Total histograms compared: 1802916
  • DQMHistoTests: Total failures: 29459
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 1773291
  • DQMHistoTests: Total skipped: 166
  • DQMHistoTests: Total Missing objects: 0
  • Checked 90 log files, 14 edm output root files, 22 DQM output files

@kmcdermo
Copy link
Contributor Author

I made a simple validation of this sequence by re-processing some signal MC from AODSIM produced through the standard 80X chain, then switching to 92X for the PAT step using this era customization (GMSB, ctau = 3m).

Without any selection on the leading photon, this is the leading photon seed time for each photon collection produced in the PAT step (slimmedPatPhotons from gedPhotons and slimmedOOTPATPhotons from ootPhotons):
leadpho_time_nosel

However, as we have seen before, when we include some selection on the photon (namely, implementing the cut based VID: https://twiki.cern.ch/twiki/bin/view/CMS/CutBasedPhotonIdentificationRun2), the standard collection loses its late time photons, in which case our ootPhotons kick in:
leadpho_time_wsel

I would say this validates this sequence.

@slava77
Copy link
Contributor

slava77 commented Jun 29, 2017

@kmcdermo
this PR can now be advanced to the proper review step, since the #19205 has just been merged.
Please rebase and possibly apply already the updates mentioned for the ReducedEGProducer to continue

Thank you.

@kmcdermo
Copy link
Contributor Author

@slava77

I guess rebase to the latest IB with 19205? should I make a separate PR for the changes requested for the ReducedEGProducer or commit them here?

For the sake of the backport, it would probably be easier with a separate PR

@slava77
Copy link
Contributor

slava77 commented Jun 29, 2017 via email

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 2, 2017

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19404/21065/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 22
  • DQMHistoTests: Total histograms compared: 1756063
  • DQMHistoTests: Total failures: 14918
  • DQMHistoTests: Total nulls: 1
  • DQMHistoTests: Total successes: 1740978
  • DQMHistoTests: Total skipped: 166
  • DQMHistoTests: Total Missing objects: 0
  • Checked 90 log files, 14 edm output root files, 22 DQM output files

ecalSeverityLevel,
particleFlowRecHitPS,
particleFlowClusterPS,
ootPhotonTask,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the ES and *PS modules are needed to make ootPhotonTask to work, they better be added to ootPhotonTask

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well, they are not needed in the standard reco, only in this special case.

or rather, they are already produced in the standard reco chain, and therefore available, whereas here they have to be reproduced.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let me know if I should move these things to the ootPhotonTask

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a fair argument that ootPhotonTask needs these extra modules only in the legacy reminiAOD setup.
The modifier can be applied to ootPhotonTask where it's defined. This PAT producer file isn't supposed to know what else is needed logically upstream of making ootPhoton

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, fair enough, I will make this change.

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 3, 2017

Pull request #19404 was updated. @perrotta, @cmsbuild, @slava77, @monttj, @davidlange6 can you please check and sign again.

@kmcdermo
Copy link
Contributor Author

kmcdermo commented Jul 3, 2017

okay, I have made the changes as recommended, and the tests locally worked.

@slava77
Copy link
Contributor

slava77 commented Jul 3, 2017

@cmsbuild please test workflow 136.7611,1325.5

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 3, 2017

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/21100/console Started: 2017/07/03 17:58

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 3, 2017

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 3, 2017

Comparison job queued.

@slava77
Copy link
Contributor

slava77 commented Jul 3, 2017

+1

for #19404 73da687

  • added modifications for run2_miniAOD_80XLegacy to be able to run OOT photons in miniAOD from 2016 AOD
  • jenkins tests pass including now running reminiAOD workflows 136.7611,1325.5
  • local tests with 136.7611,1325.5 show roughly expected behavior (the baseline used for tests is CMSSW_9_3_X_2017-07-02-2300 with Out-of-time photons for MiniAOD #19205 reverted): OOT photon modules run and the final miniAOD contains slimmedOOTPhotons reco::Photon collection and also additional relevant reducedEgamma clusters and hits. Changes are very similar to what was observed in signoff of Out-of-time photons for MiniAOD #19205
    • 2K event run in 136.7611 brings 127 slimmedOOTPhotons, 185 basic clusters and an increase of 1.3% in the size of the reducedEgamma:reducedEBRecHits. The total added products are about 400 bytes per event.
    • CPU cost per event is somewhat negligible (about 30 ms/event compared to 1.8 s/event in this reminiAOD job in full)

@davidlange6
Copy link
Contributor

merge

@cmsbuild cmsbuild merged commit d5de801 into cms-sw:master Jul 3, 2017
@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 3, 2017

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19404/21100/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 22
  • DQMHistoTests: Total histograms compared: 1756063
  • DQMHistoTests: Total failures: 29449
  • DQMHistoTests: Total nulls: 1
  • DQMHistoTests: Total successes: 1726447
  • DQMHistoTests: Total skipped: 166
  • DQMHistoTests: Total Missing objects: 0
  • Checked 90 log files, 14 edm output root files, 22 DQM output files

cmsbuild added a commit that referenced this pull request Jul 6, 2017
Combined backport of PR #19205 and #19404: ootPhotons in miniAOD from AOD from either 92X or 80X inputs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants