Add puppi multiplicities to MiniAOD #22396

ahinzmann · 2018-02-28T20:15:49Z

PUPPI-weighted multiplicities were found useful to define pileup-safe jet ID criteria and will shortly be recommended for analyses as discussed during this presentation:
https://indico.cern.ch/event/707753/contributions/2905460/attachments/1607520/2551266/JetID_run2017_AK4_PUPPI_recommendations_27022018_v1.pdf
We would therefore like to include them into MiniAOD (and backport to 94 asap).

It was checked that the new userfloats are added correctly to the slimmedJetsPuppi collection running the standard MiniAOD sequence.

Timing and size was checked using test/patMiniAOD_standard_cfg.py with 1000 events of RelValZEE_13/GEN-SIM-RECO/PU25ns_92X_upgrade2017_realistic_v7-v1:

Without this PR:
TimeReport 0.000082 0.000082 0.000082 slimmedJetsPuppi
patJets_slimmedJetsPuppi__PAT. 5769.94 668.333

With this PR:
TimeReport 0.000235 0.000235 0.000235 slimmedJetsPuppi
TimeReport 0.000071 0.000071 0.000071 slimmedJetsPuppiNoMultiplicties
TimeReport 0.000054 0.000054 0.000054 patPuppiJetSpecificProducer
patJets_slimmedJetsPuppi__PAT. 6953.5 705.626

cmsbuild · 2018-02-28T20:16:06Z

The code-checks are being triggered in jenkins.

cmsbuild · 2018-02-28T20:19:46Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-22396/3622

cmsbuild · 2018-02-28T20:20:07Z

A new Pull Request was created by @ahinzmann for master.

It involves the following packages:

PhysicsTools/PatAlgos

@perrotta, @monttj, @cmsbuild, @slava77, @gpetruc, @arizzi can you please review it and eventually sign? Thanks.
@TaiSakuma, @gouskos, @imarches, @acaudron, @mmarionncern, @rappoccio, @jdamgov, @jdolen, @nhanvtran, @gpetruc, @gkasieczka, @schoef, @ferencek, @mverzett, @mariadalfonso, @pvmulder, @seemasharmafnal, @JyothsnaKomaragiri this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

slava77 · 2018-02-28T20:48:48Z

@ahinzmann
please provide estimates of the size on disk needed for this PR per event.
CPU per event would be nice to know as well.

slava77 · 2018-02-28T20:48:56Z

@cmsbuild please test

cmsbuild · 2018-02-28T20:49:06Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/26388/console Started: 2018/02/28 21:50

cmsbuild · 2018-02-28T22:21:37Z

-1

Tested at: 3d0c460

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22396/26388/summary.html

I found follow errors while testing this PR

Failed tests: RelVals

RelVals:

When I ran the RelVals I found an error in the following worklfows:
21234.0 step3

runTheMatrix-results/21234.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D21_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D21+RecoFullGlobal_2023D21+HARVESTFullGlobal_2023D21/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D21_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D21+RecoFullGlobal_2023D21+HARVESTFullGlobal_2023D21.log

20434.0 step3

runTheMatrix-results/20434.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D19_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D19+RecoFullGlobal_2023D19+HARVESTFullGlobal_2023D19/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D19_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D19+RecoFullGlobal_2023D19+HARVESTFullGlobal_2023D19.log

20034.0 step3

runTheMatrix-results/20034.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D17_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D17+RecoFullGlobal_2023D17+HARVESTFullGlobal_2023D17/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D17_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D17+RecoFullGlobal_2023D17+HARVESTFullGlobal_2023D17.log

cmsbuild · 2018-02-28T22:21:39Z

Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped)

ahinzmann · 2018-03-01T15:36:04Z

@slava77 I've added the timing and size to the description of the PR

perrotta · 2018-03-01T16:19:15Z

Crash looks like being correlated:

----- Begin Fatal Exception 28-Feb-2018 22:31:01 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'FEVTDEBUGHLToutput_step'
   [2] Prefetching for module PoolOutputModule/'FEVTDEBUGHLToutput'
   [3] Calling method for module PATJetSelector/'selectedUpdatedPatJetsPuppiJetSpecific'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for a container with elements of type: pat::Jet
Looking for module label: updatedPatJetsPuppiJetSpecific
Looking for productInstanceName: 

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

cmsbuild · 2018-03-05T14:29:14Z

The code-checks are being triggered in jenkins.

cmsbuild · 2018-03-05T14:31:49Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-22396/3739

cmsbuild · 2018-03-05T14:32:08Z

Pull request #22396 was updated. @perrotta, @monttj, @cmsbuild, @slava77, @gpetruc, @arizzi can you please check and sign again.

perrotta · 2018-03-05T14:32:19Z

please test

cmsbuild · 2018-03-05T14:32:28Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/26512/console Started: 2018/03/05 16:25

cmsbuild · 2018-03-05T18:15:10Z

+1
Tested at: 4e2bdd1
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22396/26512/summary.html

cmsbuild · 2018-03-05T18:15:14Z

Comparison job queued.

cmsbuild · 2018-03-05T20:53:58Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22396/26512/summary.html

Comparison Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 121 differences found in the comparisons
DQMHistoTests: Total files compared: 29
DQMHistoTests: Total histograms compared: 2479021
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2478844
DQMHistoTests: Total skipped: 176
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 1.13000000017 KiB( 23 files compared)
Checked 118 log files, 9 edm output root files, 29 DQM output files

perrotta · 2018-03-07T12:49:00Z

+1

Six userfloats added per jet to slimmedJetsPuppi
Differences reported in the jenkins tests refer to the newly added userfloats to the miniAOD output
miniAOD event size on disk increases by ~40 B (on slimmedJetsPuppi) when measured on Zee events with PU (as reported in the PR description)

fabiocos · 2018-03-07T13:08:20Z

+1

perrotta · 2018-03-08T16:48:24Z

@fabiocos : this PR still misses the "analysis" signature and you need a explicit "merge" if you want to merge it

fabiocos · 2018-03-08T22:08:05Z

merge

arizzi · 2018-03-09T07:34:43Z

for the records: me and @gpetruc understood that we have to sign for AnalysisTools related PRs in PhysicsTools/PatAlgos or NanoAODTools only in the MAOD and analysis branches, not on master. Is that right?

fabiocos · 2018-03-09T07:50:04Z

yes, I would like your explicit approval for the AN branch, that is the general request. As far as PhysicsTools/NanoAOD is concerned anyway, I would say that missing an independent analysis signature it makes sense that you sign everywhere, as you are anyway the responsible and the tester of it

add puppi multiplicities to MiniAOD

3d0c460

cmsbuild added this to the CMSSW_10_1_X milestone Feb 28, 2018

cmsbuild added analysis-pending code-checks-pending comparison-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Feb 28, 2018

cmsbuild added code-checks-approved and removed code-checks-pending labels Feb 28, 2018

cmsbuild added tests-started and removed tests-pending labels Feb 28, 2018

cmsbuild added comparison-notrun tests-rejected and removed comparison-pending tests-started labels Feb 28, 2018

pick up selected pat jets

b36e338

cmsbuild removed code-checks-approved comparison-notrun labels Mar 1, 2018

cmsbuild added the tests-pending label Mar 5, 2018

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 5, 2018

cmsbuild added tests-started and removed tests-pending labels Mar 5, 2018

cmsbuild added tests-approved and removed tests-started labels Mar 5, 2018

cmsbuild added comparison-available and removed comparison-pending labels Mar 5, 2018

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Mar 7, 2018

cmsbuild added orp-approved and removed orp-pending labels Mar 7, 2018

ahinzmann mentioned this pull request Mar 7, 2018

Add puppi multiplicities to MiniAOD - 94X backport #22508

Merged

cmsbuild merged commit 7980e47 into cms-sw:master Mar 8, 2018

jmduarte mentioned this pull request May 11, 2018

TensorFlow-based DeepDoubleB tagger integration #23033

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add puppi multiplicities to MiniAOD #22396

Add puppi multiplicities to MiniAOD #22396

ahinzmann commented Feb 28, 2018 •

edited

cmsbuild commented Feb 28, 2018

cmsbuild commented Feb 28, 2018

cmsbuild commented Feb 28, 2018

slava77 commented Feb 28, 2018

slava77 commented Feb 28, 2018

cmsbuild commented Feb 28, 2018 •

edited

cmsbuild commented Feb 28, 2018

cmsbuild commented Feb 28, 2018

ahinzmann commented Mar 1, 2018

perrotta commented Mar 1, 2018

cmsbuild commented Mar 5, 2018

cmsbuild commented Mar 5, 2018

cmsbuild commented Mar 5, 2018

perrotta commented Mar 5, 2018

cmsbuild commented Mar 5, 2018 •

edited

cmsbuild commented Mar 5, 2018

cmsbuild commented Mar 5, 2018

cmsbuild commented Mar 5, 2018

perrotta commented Mar 7, 2018

fabiocos commented Mar 7, 2018

perrotta commented Mar 8, 2018

fabiocos commented Mar 8, 2018

arizzi commented Mar 9, 2018

fabiocos commented Mar 9, 2018

Add puppi multiplicities to MiniAOD #22396

Add puppi multiplicities to MiniAOD #22396

Conversation

ahinzmann commented Feb 28, 2018 • edited

cmsbuild commented Feb 28, 2018

cmsbuild commented Feb 28, 2018

cmsbuild commented Feb 28, 2018

slava77 commented Feb 28, 2018

slava77 commented Feb 28, 2018

cmsbuild commented Feb 28, 2018 • edited

cmsbuild commented Feb 28, 2018

cmsbuild commented Feb 28, 2018

ahinzmann commented Mar 1, 2018

perrotta commented Mar 1, 2018

cmsbuild commented Mar 5, 2018

cmsbuild commented Mar 5, 2018

cmsbuild commented Mar 5, 2018

perrotta commented Mar 5, 2018

cmsbuild commented Mar 5, 2018 • edited

cmsbuild commented Mar 5, 2018

cmsbuild commented Mar 5, 2018

cmsbuild commented Mar 5, 2018

perrotta commented Mar 7, 2018

fabiocos commented Mar 7, 2018

perrotta commented Mar 8, 2018

fabiocos commented Mar 8, 2018

arizzi commented Mar 9, 2018

fabiocos commented Mar 9, 2018

ahinzmann commented Feb 28, 2018 •

edited

cmsbuild commented Feb 28, 2018 •

edited

cmsbuild commented Mar 5, 2018 •

edited