[JME NanoAOD] Backport of #32722 (Changes for event size reduction) to 11_2_X

[nurfikri89]

Backport from #32722

PR description:

This PR is to reduce the size of custom JME NanoAODs. The following changes are made:

• Check that npuProd[ibx].getPU_pT_hats() size is more than 0 in PhysicsTools/NanoAOD/plugins/NPUTablesProducer.cc.
• Switch on storing Parton Shower (PS) weights.
• b-tagging algorithm outputs are saved for AK4 jets with pT > 15 GeV.
• Low-level input variables for PileUp jet ID and Quark-Gluon Likelihood are saved for AK4 jets with pT > 10 GeV.
• Remove AK4 PF and AK8 PF jet collections.
• Add separate AK8 GEN and AK8 Puppi jet collections for JEC studies. Minimal information are saved.
• For reclustered GEN jet collections, the minimum pT at the jet clustering level uses the default value (3 GeV) in RecoJets/JetProducers/python/ak4GenJets_cfi.py.
• For reclustered reco-level jet collections, the minimum pT at the jet clustering level is set to be 0. Then, only jets with (JEC-corrected) pT > 8 GeV are saved. Additionally for MC samples, jets with pT < 8 GeV are saved if they are matched to GEN jets.

With this PR, the event size is now reduced to 6.42 kb/event from 9.32 kb/event. The comparison was made using 10K events from a TTJets RunIISummer19UL17MiniAOD sample. The changes in this PR was discussed in the 13/01/2021 XPOG meeting [1].

[1] https://indico.cern.ch/event/978436/

if this PR is a backport please specify the original PR:

Original PR is #32722

[cmsbuild]

A new Pull Request was created by @nurfikri89 (Nurfikri Norjoharuddeen) for CMSSW_11_2_X.

It involves the following packages:

PhysicsTools/NanoAOD
PhysicsTools/PatAlgos

@perrotta, @gouskos, @cmsbuild, @fgolf, @slava77, @jpata, @mariadalfonso can you please review it and eventually sign? Thanks.
@jdamgov, @emilbols, @gouskos, @swertz, @jdolen, @ahinzmann, @smoortga, @schoef, @rappoccio, @mariadalfonso, @JyothsnaKomaragiri, @nhanvtran, @gkasieczka, @clelange, @hatakeyamak, @ferencek, @gpetruc, @andrzejnovak, @peruzzim, @seemasharmafnal, @mmarionncern this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

• Backported from [JME NanoAOD] Changes for event size reduction #32722

[slava77]

please test workflow 25202.15,10224.15,11024.15

[slava77]

backport of #32722

[slava77]

please test workflow 25202.15,10224.15,11024.15

it looks like the last set of tests got stuck.
@smuzaffar @mrodozov please check what happened in case there is a problem in the test status detection
cms/32760/slc7_amd64_gcc900/relvals Pending — Waiting for tests to start was in the status for at least the last 12 hours while tests in other PRs were running. And the "Details" link https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/12583/ was showing completed results since a while before.

[smuzaffar]

@slava77 , I see that relval/input test was run here https://cmssdt.cern.ch/jenkins/job/ib-run-pr-relvals/701/ . The normal relval test should have been https://cmssdt.cern.ch/jenkins/job/ib-run-pr-relvals/702 but I do not see it in jenkins ( I guess it was accidentally killed/deleted).

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-563b7f/12599/summary.html
COMMIT: 3d31ee2
CMSSW: CMSSW_11_2_X_2021-01-29-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/32760/12599/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-563b7f/10224.15_TTbar_13+2017PU_JMENano+TTbar_13TeV_TuneCUETP8M1_GenSim+DigiPU+RecoFakeHLTPU+HARVESTFakeHLTPU+Nano
• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-563b7f/11024.15_TTbar_13+2018PU_JMENano+TTbar_13TeV_TuneCUETP8M1_GenSim+DigiPU+RecoFakeHLTPU+HARVESTFakeHLTPU+Nano
• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-563b7f/25202.15_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25+NANOUP15MC_PU25_JME

Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 36
• DQMHistoTests: Total histograms compared: 2529619
• DQMHistoTests: Total failures: 1
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2529596
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 35 files compared)
• Checked 151 log files, 37 edm output root files, 36 DQM output files

[slava77]

+1

for #32760 3d31ee2

• the backport has the same diff in the code as [JME NanoAOD] Changes for event size reduction #32722
• the modifications do not affect regular production setup, except for the special JME nanoAOD; so, this should be OK for a backport to 11_2_X
• jenkins tests pass without differences in the (monitored) results

[silviodonato]

@cms-sw/xpog-l2 any objections?

[gouskos]

+xpog

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_11_2_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_11_3_X is complete. This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[silviodonato]

+1