PUPPI MET in RECO #29769

ahinzmann · 2020-05-07T23:59:10Z

Clone of #29254 and #29768, with a fix for MET JER uncertainties to adress issue #29744.

PR description:

This PR computes puppiNoLep weights and puppi MET in RECO and stores the puppiNoLep weight as a value map. PUPPI MET is clustered based on standard PF candidates together with the value map of weights. A new boolean "isWeighted" is added to the reco::PFMet interface to indicate whether weights were applied during jet clustering. The packed candidates pick up the value map such that puppi does not need to be re-computed in MiniAOD.

The same was done for the puppi and PUPPI jets in #28936.

Other changes:

Produce puppiForMET directly in PuppiProducer instead of a two step procedure puppiNoLep->puppiForMET and rename it to puppiNoLep, consistent with the name in PackedCandidates
Implement fillDescriptions for PFMetProducer
Rename PFMET_cfi to pfMet_cfi everywhere (for consistency with the module name)
Fix input jet collection of pfMetEI
Remove pfMetEI from RecoMET_EventContent (however, leave EITopPAG_EventContent_cff untouched)
Resolve issue cleanup miniAOD workflow after moving PUPPI to reco #29264
By moving to the same input PF candidate collection for puppi MET and puppi jets (with just different weights), PUPPI MET significance and unclustered pT computation then don't rely on dR matching as implemented in Fix MET unclustered energy computation #29331 anymore.
Store ref to PF candidate inside pat muons (just like done for all other PAT objects). PUPPI MET significance and unclustered pT computation then don't rely on dR matching as implemented in Fix MET Significance, Covariance Matrix and unclustered energy lepton subtraction #29385 anymore.
Fix particle multiplicity computation in xy-correction for PUPPI MET. Previously, particles with a weight of 0 could also count in the multiplicity as a function of which the xy-corrections are computed. Since xy-correction are not needed/recommended for PUPPI MET, this fix is not backported.
For HI workflows, raise candidate pT threshold for chsMet and trkMet to 999. consistent with pfMet.

PR validation:

runTheMatrix -l limited

if this PR is a backport please specify the original PR and why you need to backport that PR:

No backport foreseen.

cmsbuild · 2020-05-07T23:59:34Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-05-08T00:04:30Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-29769/15217

This PR adds an extra 12KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File CommonTools/ParticleFlow/plugins/Type1PFMET.cc modified in PR(s): PUPPI MET in RECO #29768
- File CommonTools/ParticleFlow/python/EITopPAG_cff.py modified in PR(s): PUPPI MET in RECO #29768
- File CommonTools/ParticleFlow/python/PFBRECO_cff.py modified in PR(s): PUPPI MET in RECO #29768
- File CommonTools/ParticleFlow/python/pfMET_cfi.py modified in PR(s): PUPPI MET in RECO #29768
- File CommonTools/PileupAlgos/plugins/PuppiProducer.cc modified in PR(s): PUPPI MET in RECO #29768, PUPPI tune v14 (phase-2 excluded) #29600
- File CommonTools/PileupAlgos/plugins/PuppiProducer.h modified in PR(s): PUPPI MET in RECO #29768, PUPPI tune v14 (phase-2 excluded) #29600
- File CommonTools/PileupAlgos/python/Puppi_cff.py modified in PR(s): PUPPI MET in RECO #29768, PUPPI tune v14 (phase-2 excluded) #29600
- File CommonTools/PileupAlgos/python/customizePuppiTune_cff.py modified in PR(s): PUPPI MET in RECO #29768, PUPPI tune v14 (phase-2 excluded) #29600
- File CommonTools/PileupAlgos/src/PuppiContainer.cc modified in PR(s): PUPPI MET in RECO #29768
- File CommonTools/PileupAlgos/test/testPUMods.py modified in PR(s): PUPPI MET in RECO #29768
- File DataFormats/METReco/interface/MET.h modified in PR(s): PUPPI MET in RECO #29768
- File DataFormats/METReco/interface/PFMET.h modified in PR(s): PUPPI MET in RECO #29768
- File DataFormats/METReco/src/MET.cc modified in PR(s): PUPPI MET in RECO #29768
- File DataFormats/METReco/src/classes_def.xml modified in PR(s): PUPPI MET in RECO #29768
- File DataFormats/PatCandidates/src/classes_def_objects.xml modified in PR(s): PUPPI MET in RECO #29768
- File ElectroWeakAnalysis/Utilities/python/distPFMET_cfi.py modified in PR(s): PUPPI MET in RECO #29768
- File HLTriggerOffline/Exotica/python/ExoticaValidation_cff.py modified in PR(s): PUPPI MET in RECO #29768
- File JetMETCorrections/Type1MET/plugins/MultShiftMETcorrDBInputProducer.cc modified in PR(s): PUPPI MET in RECO #29768
- File JetMETCorrections/Type1MET/plugins/MultShiftMETcorrDBInputProducer.h modified in PR(s): PUPPI MET in RECO #29768
- File JetMETCorrections/Type1MET/plugins/MultShiftMETcorrInputProducer.cc modified in PR(s): PUPPI MET in RECO #29768
- File JetMETCorrections/Type1MET/plugins/MultShiftMETcorrInputProducer.h modified in PR(s): PUPPI MET in RECO #29768
- File JetMETCorrections/Type1MET/plugins/PFCandMETcorrInputProducer.cc modified in PR(s): PUPPI MET in RECO #29768
- File JetMETCorrections/Type1MET/plugins/PFCandMETcorrInputProducer.h modified in PR(s): PUPPI MET in RECO #29768
- File JetMETCorrections/Type1MET/python/correctionTermsPfMetType1Type2_cff.py modified in PR(s): PUPPI MET in RECO #29768
- File JetMETCorrections/Type1MET/python/pfMETmultShiftCorrectionsDB_cfi.py modified in PR(s): PUPPI MET in RECO #29768
- File JetMETCorrections/Type1MET/python/pfMETmultShiftCorrections_cfi.py modified in PR(s): PUPPI MET in RECO #29768
- File JetMETCorrections/Type1MET/src/AddCorrectionsToGenericMET.cc modified in PR(s): PUPPI MET in RECO #29768
- File PhysicsTools/PatAlgos/plugins/PATMETProducer.cc modified in PR(s): PUPPI MET in RECO #29768
- File PhysicsTools/PatAlgos/plugins/PATMETProducer.h modified in PR(s): PUPPI MET in RECO #29768
- File PhysicsTools/PatAlgos/plugins/PATMuonProducer.cc modified in PR(s): PUPPI MET in RECO #29768
- File PhysicsTools/PatAlgos/plugins/PATPackedCandidateProducer.cc modified in PR(s): PUPPI MET in RECO #29768
- File PhysicsTools/PatAlgos/python/producersLayer1/metProducer_cfi.py modified in PR(s): PUPPI MET in RECO #29768
- File PhysicsTools/PatAlgos/python/slimming/applySubstructure_cff.py modified in PR(s): PUPPI MET in RECO #29768
- File PhysicsTools/PatAlgos/python/slimming/customizeMiniAOD_HcalFixLegacy2016.py modified in PR(s): PUPPI MET in RECO #29768
- File PhysicsTools/PatAlgos/python/slimming/customizeMiniAOD_MuEGFixMoriond2017.py modified in PR(s): PUPPI MET in RECO #29768
- File PhysicsTools/PatAlgos/python/slimming/extraSlimmedMETs_MuEGFixMoriond2017.py modified in PR(s): PUPPI MET in RECO #29768
- File PhysicsTools/PatAlgos/python/slimming/miniAOD_tools.py modified in PR(s): PUPPI MET in RECO #29768, run2 re-miniAOD setup based on UL inputs #29756
- File PhysicsTools/PatAlgos/python/slimming/puppiForMET_cff.py modified in PR(s): PUPPI MET in RECO #29768
- File PhysicsTools/PatAlgos/python/tools/jetTools.py modified in PR(s): PUPPI MET in RECO #29768
- File PhysicsTools/PatAlgos/test/corMETFromMiniAOD.py modified in PR(s): PUPPI MET in RECO #29768
- File PhysicsTools/PatAlgos/test/miniAOD/example_ei.py modified in PR(s): PUPPI MET in RECO #29768
- File PhysicsTools/PatAlgos/test/miniAOD/example_jet_and_met.py modified in PR(s): PUPPI MET in RECO #29768
- File PhysicsTools/PatAlgos/test/produceMETFromAOD.py modified in PR(s): PUPPI MET in RECO #29768
- File PhysicsTools/PatUtils/plugins/ShiftedParticleMETcorrInputProducer.cc modified in PR(s): PUPPI MET in RECO #29768
- File PhysicsTools/PatUtils/plugins/ShiftedParticleMETcorrInputProducer.h modified in PR(s): PUPPI MET in RECO #29768
- File PhysicsTools/PatUtils/plugins/ShiftedParticleProducer.cc modified in PR(s): PUPPI MET in RECO #29768
- File PhysicsTools/PatUtils/plugins/ShiftedParticleProducer.h modified in PR(s): PUPPI MET in RECO #29768
- File PhysicsTools/PatUtils/python/tools/eGammaCorrection.py modified in PR(s): PUPPI MET in RECO #29768
- File PhysicsTools/PatUtils/python/tools/runMETCorrectionsAndUncertainties.py modified in PR(s): PUPPI MET in RECO #29768
- File RecoMET/Configuration/python/RecoMET_EventContent_cff.py modified in PR(s): PUPPI MET in RECO #29768
- File RecoMET/Configuration/python/RecoPFMET_cff.py modified in PR(s): PUPPI MET in RECO #29768
- File RecoMET/METAlgorithms/interface/METAlgo.h modified in PR(s): PUPPI MET in RECO #29768
- File RecoMET/METAlgorithms/interface/METSignificance.h modified in PR(s): PUPPI MET in RECO #29768
- File RecoMET/METAlgorithms/interface/PFSpecificAlgo.h modified in PR(s): PUPPI MET in RECO #29768
- File RecoMET/METAlgorithms/src/METAlgo.cc modified in PR(s): PUPPI MET in RECO #29768
- File RecoMET/METAlgorithms/src/METSignificance.cc modified in PR(s): PUPPI MET in RECO #29768
- File RecoMET/METAlgorithms/src/PFSpecificAlgo.cc modified in PR(s): PUPPI MET in RECO #29768
- File RecoMET/METProducers/interface/METSignificanceProducer.h modified in PR(s): PUPPI MET in RECO #29768
- File RecoMET/METProducers/interface/PFMETProducer.h modified in PR(s): PUPPI MET in RECO #29768
- File RecoMET/METProducers/python/METSignificance_cfi.py modified in PR(s): PUPPI MET in RECO #29768
- File RecoMET/METProducers/python/PFMET_cfi.py modified in PR(s): PUPPI MET in RECO #29768
- File RecoMET/METProducers/python/pfChMet_cfi.py modified in PR(s): PUPPI MET in RECO #29768
- File RecoMET/METProducers/src/METSignificanceProducer.cc modified in PR(s): PUPPI MET in RECO #29768
- File RecoMET/METProducers/src/PFMETProducer.cc modified in PR(s): PUPPI MET in RECO #29768
- File RecoMET/METProducers/test/recoMET_pfMet_cfg.py modified in PR(s): PUPPI MET in RECO #29768

cmsbuild · 2020-05-08T00:04:56Z

A new Pull Request was created by @ahinzmann for master.

It involves the following packages:

CommonTools/ParticleFlow
CommonTools/PileupAlgos
DataFormats/METReco
DataFormats/PatCandidates
ElectroWeakAnalysis/Utilities
HLTriggerOffline/Exotica
JetMETCorrections/Type1MET
PhysicsTools/PatAlgos
PhysicsTools/PatUtils
RecoMET/Configuration
RecoMET/METAlgorithms
RecoMET/METProducers

@perrotta, @andrius-k, @kmaeshima, @schneiml, @cmsbuild, @jfernan2, @fioriNTU, @slava77, @santocch can you please review it and eventually sign? Thanks.
@rappoccio, @gouskos, @hatakeyamak, @emilbols, @peruzzim, @seemasharmafnal, @mmarionncern, @jdolen, @smoortga, @missirol, @ferencek, @rovere, @jdamgov, @nhanvtran, @gkasieczka, @schoef, @mtosi, @mariadalfonso, @clelange, @riga, @JyothsnaKomaragiri, @mverzett, @cbernet, @gpetruc, @andrzejnovak this is something you requested to watch as well.
@silviodonato, @dpiparo you are the release manager for this.

cms-bot commands are listed here

slava77 · 2020-05-08T00:56:47Z

@cmsbuild please test workflow 25202.0, 25213.0, 250206.182, 25403.18, 250203.18, 250402.18, 11025.0, 11634.7, 25403.18, 250402.18, 11834.0, 25211.18, 25403.0, 250205.18

cmsbuild · 2020-05-08T00:57:09Z

The tests are being triggered in jenkins.
Test Parameters:

MATRIX_EXTRAS = 25202.0,25211.18,25403.0,250203.18,250206.182,25213.0,11025.0,250402.18,250205.18,11634.7,11834.0,25403.18
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/6168/console Started: 2020/05/08 05:09

cmsbuild · 2020-05-08T05:05:39Z

+1
Tested at: 01dd1d4
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-dde9e7/6168/summary.html
CMSSW: CMSSW_11_1_X_2020-05-07-2300
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-05-08T05:05:42Z

Comparison job queued.

cmsbuild · 2020-05-08T06:38:02Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-dde9e7/6168/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-dde9e7/250203.18_H125GGgluonfusion_13UP18+H125GGgluonfusion_13UP18+DIGIPRMXUP18_PU25+RECOPRMXUP18_PU25+HARVESTUP18_PU25
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-dde9e7/250205.18_ZTT_13UP18+ZTT_13UP18+DIGIPRMXUP18_PU25+RECOPRMXUP18_PU25+HARVESTUP18_PU25
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-dde9e7/250206.182_ZMM_13UP18_RD+ZMM_13UP18_RD+DIGIPRMXUP18_PU25_RD+RECOPRMXUP18_PU25_L1TMuDQM_RD+HARVESTUP18_PU25_L1TMuDQM_RD
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-dde9e7/250402.18_TTbar_13+FS_TTbar_13_PRMXUP18_PU50+HARVESTUP18FS+MINIAODMCUP18FS
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-dde9e7/25211.18_GluGluHToZZTo4L_M125_Pow_py8_Evt_13UP18+GluGluHToZZTo4L_M125_Pow_py8_Evt_13UP18+DIGIUP18_PU25+RECOUP18_PU25+HARVESTUP18_PU25
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-dde9e7/25403.0_H125GGgluonfusion_13+FS_H125GGgluonfusion_13_UP15_PU25+HARVESTUP15FS+MINIAODMCUP15FS
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-dde9e7/25403.18_H125GGgluonfusion_13+FS_H125GGgluonfusion_13_UP18_PU50+HARVESTUP18FS+MINIAODMCUP18FS

Comparison Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 142 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2697527
DQMHistoTests: Total failures: 105
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2697103
DQMHistoTests: Total skipped: 319
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 147 log files, 16 edm output root files, 34 DQM output files

jfernan2 · 2020-05-08T07:45:49Z

@ahinzmann Is this PR also fixing this small differences created in the past:
#29254 (comment)
Thanks

ahinzmann · 2020-05-08T07:56:59Z

@ahinzmann Is this PR also fixing this small differences created in the past:
#29254 (comment)
Thanks

No

slava77 · 2020-05-08T14:14:59Z

+1

for #29769 01dd1d4

this looks as an appropriate bugfix to the issue in PUPPI MET in RECO #29254
jenkins tests pass and comparisons with the baseline appear to be in the same places as for PUPPI MET in RECO #29254 itself
the extra matrix tests confirm that the crash is not happening anymore in ShiftedParticleMETcorrInputProducer.cc, as expected

jfernan2 · 2020-05-08T14:18:29Z

+1

silviodonato · 2020-05-09T20:27:17Z

merge

santocch · 2020-05-10T20:38:57Z

+1

cmsbuild · 2020-05-10T20:39:24Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

silviodonato and others added 2 commits May 7, 2020 22:15

Revert "Revert "PUPPI MET in RECO""

a97679e

fix met resolution shifts

01dd1d4

cmsbuild added this to the CMSSW_11_1_X milestone May 7, 2020

cmsbuild added analysis-pending code-checks-pending comparison-pending dqm-pending orp-pending pending-signatures reconstruction-pending tests-pending labels May 7, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels May 8, 2020

cmsbuild added tests-started and removed tests-pending labels May 8, 2020

cmsbuild added tests-approved and removed tests-started labels May 8, 2020

cmsbuild added comparison-available and removed comparison-pending labels May 8, 2020

cmsbuild added reconstruction-approved and removed reconstruction-pending labels May 8, 2020

cmsbuild added dqm-approved and removed dqm-pending labels May 8, 2020

cmsbuild added orp-approved and removed orp-pending labels May 9, 2020

cmsbuild merged commit 60e3a2a into cms-sw:master May 9, 2020

cmsbuild added analysis-approved fully-signed and removed analysis-pending pending-signatures labels May 10, 2020

This was referenced May 13, 2020

UL remini updates: fix running PUPPI, add egamma scale and smearing #29826

Merged

Exception: Original collection and shifted collection are of different size in ShiftedParticleMETcorrInputProducer #29744

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PUPPI MET in RECO #29769

PUPPI MET in RECO #29769

ahinzmann commented May 7, 2020

cmsbuild commented May 7, 2020

cmsbuild commented May 8, 2020

cmsbuild commented May 8, 2020

slava77 commented May 8, 2020

cmsbuild commented May 8, 2020 •

edited

cmsbuild commented May 8, 2020

cmsbuild commented May 8, 2020

cmsbuild commented May 8, 2020

jfernan2 commented May 8, 2020 •

edited

ahinzmann commented May 8, 2020

slava77 commented May 8, 2020

jfernan2 commented May 8, 2020

silviodonato commented May 9, 2020

santocch commented May 10, 2020

cmsbuild commented May 10, 2020

PUPPI MET in RECO #29769

PUPPI MET in RECO #29769

Conversation

ahinzmann commented May 7, 2020

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented May 7, 2020

cmsbuild commented May 8, 2020

cmsbuild commented May 8, 2020

slava77 commented May 8, 2020

cmsbuild commented May 8, 2020 • edited

cmsbuild commented May 8, 2020

cmsbuild commented May 8, 2020

cmsbuild commented May 8, 2020

jfernan2 commented May 8, 2020 • edited

ahinzmann commented May 8, 2020

slava77 commented May 8, 2020

jfernan2 commented May 8, 2020

silviodonato commented May 9, 2020

santocch commented May 10, 2020

cmsbuild commented May 10, 2020

cmsbuild commented May 8, 2020 •

edited

jfernan2 commented May 8, 2020 •

edited