Migrate most PhysicsTools/PatAlgos plugins to ESGetToken #34368

guitargeek · 2021-07-06T18:07:46Z

PR description:

Migrate most PhysicsTools/PatAlgos plugins to ESGetToken, as already tried in #34307 but then decided to have it in a separate PR.

The migration of the only missing plugin, the TauJetCorrFactorsProducer is a bit more tricky. This will be done at a later stage.

PR validation:

CMSSW compiles.

if this PR is a backport please specify the original PR and why you need to backport that PR:

No backport intended.

cmsbuild · 2021-07-06T18:15:17Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34368/23744

This PR adds an extra 64KB to repository

cmsbuild · 2021-07-06T18:15:43Z

A new Pull Request was created by @guitargeek (Jonas Rembser) for master.

It involves the following packages:

PhysicsTools/PatAlgos

@perrotta, @jpata, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@mmarionncern, @gouskos, @JyothsnaKomaragiri, @ahinzmann, @schoef, @rappoccio, @swozniewski, @jdamgov, @jdolen, @nhanvtran, @gkasieczka, @clelange, @emilbols, @hatakeyamak, @mbluj, @gpetruc, @andrzejnovak, @mariadalfonso, @seemasharmafnal this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

Dr15Jones · 2021-07-06T20:50:02Z

PhysicsTools/PatAlgos/plugins/PATMuonProducer.cc

  }

  // MC info
  edm::Handle<edm::ValueMap<reco::MuonSimInfo>> simInfo;
  bool simInfoIsAvailalbe = iEvent.getByToken(simInfo_, simInfo);

  // this will be the new object collection
-  std::vector<Muon>* patMuons = new std::vector<Muon>();
+  auto patMuons = std::make_unique<std::vector<Muon>>();


For future reference, this could have just been std::vector<Muon> pathMuons and then call iEvent.emplace(std::move(pathMuons)).

slava77 · 2021-07-07T15:37:50Z

@cmsbuild please test

guitargeek · 2021-07-07T15:48:18Z

Oh sorry I just saw now after pushing that you started the tests! I thought you were waiting for me to respond to @Dr15Jones comments...

cmsbuild · 2021-07-07T15:52:50Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34368/23777

This PR adds an extra 64KB to repository

cmsbuild · 2021-07-07T15:53:09Z

Pull request #34368 was updated. @perrotta, @jpata, @cmsbuild, @slava77 can you please check and sign again.

slava77 · 2021-07-07T16:00:24Z

Oh sorry I just saw now after pushing that you started the tests! I thought you were waiting for me to respond to @Dr15Jones comments...

are you done with the updates now?

guitargeek · 2021-07-07T16:02:13Z

Yes, I'm done now.

slava77 · 2021-07-07T17:40:25Z

@cmsbuild please test

cmsbuild · 2021-07-07T19:55:47Z

-1

Failed Tests: UnitTests RelVals RelVals-INPUT AddOn
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9eaab1/16560/summary.html
COMMIT: c52c527
CMSSW: CMSSW_12_0_X_2021-07-06-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34368/16560/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test runtestPhysicsToolsPatAlgos had ERRORS

RelVals

136.8523

----- Begin Fatal Exception 07-Jul-2021 20:45:11 CEST-----------------------
An exception of category 'InvalidRequest' occurred while
   [0] Processing  Event run: 319450 lumi: 31 event: 42789123 stream: 0
   [1] Running path 'nanoAOD_step'
   [2] Calling method for module JetCorrFactorsProducer/'patJetCorrFactorsReapplyJEC'
Exception Message:
You try to create a CorrectionFactor which is neither flavor dependent nor 
flavor independent. The CorrectionFactor should obey the following rules:  

 * CorrectionFactor is a std::pair<std::string, std::vector<float> >.      
 * The std::string holds the label of the correction level (following the  
   conventions of JetMET.						      
 * The std::vector<float> holds the correction factors, these factors are  
   up to the given level. They include all previous correction steps.      
 * The vector has the length *1* for flavor independent correction factors 
   or *5* for flavor dependent correction factors.			      
 * The expected order of flavor dependent correction factors is: NONE,     
   GLUON, UDS, CHARM, BOTTOM. If follows the JetMET conventions and is     
   in the Flavor enumerator of the JetCorrFactos class.		      
 * For flavor depdendent correction factors the first entry in the vector  
   (corresponding to NONE) is invalid and should be set to -1. It will not 
   be considered by the class structure though.			      

Make sure that all elements of the argument vector to this contructor are  
in accordance with these rules.
----- End Fatal Exception -------------------------------------------------

135.4

----- Begin Fatal Exception 07-Jul-2021 20:46:08 CEST-----------------------
An exception of category 'InvalidRequest' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'validation_step'
   [2] Prefetching for module BDHadronTrackMonitoringAnalyzer/'BDHadronTrackMonitoringAnalyze'
   [3] Prefetching for module PATJetProducer/'patJetsBDHadron'
   [4] Calling method for module JetCorrFactorsProducer/'patJetCorrFactors'
Exception Message:
You try to create a CorrectionFactor which is neither flavor dependent nor 
flavor independent. The CorrectionFactor should obey the following rules:  

 * CorrectionFactor is a std::pair<std::string, std::vector<float> >.      
 * The std::string holds the label of the correction level (following the  
   conventions of JetMET.						      
 * The std::vector<float> holds the correction factors, these factors are  
   up to the given level. They include all previous correction steps.      
 * The vector has the length *1* for flavor independent correction factors 
   or *5* for flavor dependent correction factors.			      
 * The expected order of flavor dependent correction factors is: NONE,     
   GLUON, UDS, CHARM, BOTTOM. If follows the JetMET conventions and is     
   in the Flavor enumerator of the JetCorrFactos class.		      
 * For flavor depdendent correction factors the first entry in the vector  
   (corresponding to NONE) is invalid and should be set to -1. It will not 
   be considered by the class structure though.			      

Make sure that all elements of the argument vector to this contructor are  
in accordance with these rules.
----- End Fatal Exception -------------------------------------------------

136.8311

----- Begin Fatal Exception 07-Jul-2021 20:46:03 CEST-----------------------
An exception of category 'InvalidRequest' occurred while
   [0] Processing  Event run: 305064 lumi: 36 event: 55020723 stream: 0
   [1] Running path 'dqmofflineOnPAT_step'
   [2] Prefetching for module SingleTopTChannelLeptonDQM_miniAOD/'singleTopElectronMediumDQM_miniAOD'
   [3] Prefetching for module PATJetSelector/'slimmedJets'
   [4] Prefetching for module PATJetUpdater/'updatedPatJetsTransientCorrectedSlimmedDeepFlavour'
   [5] Prefetching for module PATJetUpdater/'updatedPatJetsSlimmedDeepFlavour'
   [6] Prefetching for module PATJetSlimmer/'slimmedJetsNoDeepFlavour'
   [7] Prefetching for module PATJetSelector/'selectedPatJets'
   [8] Prefetching for module PATJetProducer/'patJets'
   [9] Calling method for module JetCorrFactorsProducer/'patJetCorrFactors'
Exception Message:
You try to create a CorrectionFactor which is neither flavor dependent nor 
flavor independent. The CorrectionFactor should obey the following rules:  

 * CorrectionFactor is a std::pair<std::string, std::vector<float> >.      
 * The std::string holds the label of the correction level (following the  
   conventions of JetMET.						      
 * The std::vector<float> holds the correction factors, these factors are  
   up to the given level. They include all previous correction steps.      
 * The vector has the length *1* for flavor independent correction factors 
   or *5* for flavor dependent correction factors.			      
 * The expected order of flavor dependent correction factors is: NONE,     
   GLUON, UDS, CHARM, BOTTOM. If follows the JetMET conventions and is     
   in the Flavor enumerator of the JetCorrFactos class.		      
 * For flavor depdendent correction factors the first entry in the vector  
   (corresponding to NONE) is invalid and should be set to -1. It will not 
   be considered by the class structure though.			      

Make sure that all elements of the argument vector to this contructor are  
in accordance with these rules.
----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

RelVals-INPUT

136.72411136.72411_RunJetHT2016B_reminiaodUL+RunJetHT2016B_reminiaodUL+REMINIAOD_data2016UL_HIPM+HARVESTDR2_REMINIAOD_data2016UL_HIPM/step2_RunJetHT2016B_reminiaodUL+RunJetHT2016B_reminiaodUL+REMINIAOD_data2016UL_HIPM+HARVESTDR2_REMINIAOD_data2016UL_HIPM.log
136.76111136.76111_RunJetHT2016E_reminiaodUL+RunJetHT2016E_reminiaodUL+REMINIAOD_data2016UL_HIPM+HARVESTDR2_REMINIAOD_data2016UL_HIPM/step2_RunJetHT2016E_reminiaodUL+RunJetHT2016E_reminiaodUL+REMINIAOD_data2016UL_HIPM+HARVESTDR2_REMINIAOD_data2016UL_HIPM.log
136.7611136.7611_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016_HIPM+HARVESTDR2_REMINIAOD_data2016_HIPM/step2_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016_HIPM+HARVESTDR2_REMINIAOD_data2016_HIPM.log

Expand to see more relval errors ...

AddOn Tests

fastsim

----- Begin Fatal Exception 07-Jul-2021 20:45:33 CEST-----------------------
An exception of category 'InvalidRequest' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 4 stream: 3
   [1] Running path 'validation_step'
   [2] Prefetching for module BDHadronTrackMonitoringAnalyzer/'BDHadronTrackMonitoringAnalyze'
   [3] Prefetching for module PATJetProducer/'patJetsBDHadron'
   [4] Calling method for module JetCorrFactorsProducer/'patJetCorrFactors'
Exception Message:
You try to create a CorrectionFactor which is neither flavor dependent nor 
flavor independent. The CorrectionFactor should obey the following rules:  

 * CorrectionFactor is a std::pair<std::string, std::vector<float> >.      
 * The std::string holds the label of the correction level (following the  
   conventions of JetMET.						      
 * The std::vector<float> holds the correction factors, these factors are  
   up to the given level. They include all previous correction steps.      
 * The vector has the length *1* for flavor independent correction factors 
   or *5* for flavor dependent correction factors.			      
 * The expected order of flavor dependent correction factors is: NONE,     
   GLUON, UDS, CHARM, BOTTOM. If follows the JetMET conventions and is     
   in the Flavor enumerator of the JetCorrFactos class.		      
 * For flavor depdendent correction factors the first entry in the vector  
   (corresponding to NONE) is invalid and should be set to -1. It will not 
   be considered by the class structure though.			      

Make sure that all elements of the argument vector to this contructor are  
in accordance with these rules.
----- End Fatal Exception -------------------------------------------------

pat1

----- Begin Fatal Exception 07-Jul-2021 20:43:08 CEST-----------------------
An exception of category 'InvalidRequest' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 2 stream: 0
   [1] Running path 'p'
   [2] Prefetching for module CandidateSummaryTable/'selectedPatCandidateSummary'
   [3] Prefetching for module PATJetSelector/'selectedPatJets'
   [4] Prefetching for module PATJetProducer/'patJets'
   [5] Calling method for module JetCorrFactorsProducer/'patJetCorrFactors'
Exception Message:
You try to create a CorrectionFactor which is neither flavor dependent nor 
flavor independent. The CorrectionFactor should obey the following rules:  

 * CorrectionFactor is a std::pair<std::string, std::vector<float> >.      
 * The std::string holds the label of the correction level (following the  
   conventions of JetMET.						      
 * The std::vector<float> holds the correction factors, these factors are  
   up to the given level. They include all previous correction steps.      
 * The vector has the length *1* for flavor independent correction factors 
   or *5* for flavor dependent correction factors.			      
 * The expected order of flavor dependent correction factors is: NONE,     
   GLUON, UDS, CHARM, BOTTOM. If follows the JetMET conventions and is     
   in the Flavor enumerator of the JetCorrFactos class.		      
 * For flavor depdendent correction factors the first entry in the vector  
   (corresponding to NONE) is invalid and should be set to -1. It will not 
   be considered by the class structure though.			      

Make sure that all elements of the argument vector to this contructor are  
in accordance with these rules.
----- End Fatal Exception -------------------------------------------------

fastsim1

----- Begin Fatal Exception 07-Jul-2021 20:45:36 CEST-----------------------
An exception of category 'InvalidRequest' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 4 stream: 1
   [1] Running path 'validation_step'
   [2] Prefetching for module BDHadronTrackMonitoringAnalyzer/'BDHadronTrackMonitoringAnalyze'
   [3] Prefetching for module PATJetProducer/'patJetsBDHadron'
   [4] Calling method for module JetCorrFactorsProducer/'patJetCorrFactors'
Exception Message:
You try to create a CorrectionFactor which is neither flavor dependent nor 
flavor independent. The CorrectionFactor should obey the following rules:  

 * CorrectionFactor is a std::pair<std::string, std::vector<float> >.      
 * The std::string holds the label of the correction level (following the  
   conventions of JetMET.						      
 * The std::vector<float> holds the correction factors, these factors are  
   up to the given level. They include all previous correction steps.      
 * The vector has the length *1* for flavor independent correction factors 
   or *5* for flavor dependent correction factors.			      
 * The expected order of flavor dependent correction factors is: NONE,     
   GLUON, UDS, CHARM, BOTTOM. If follows the JetMET conventions and is     
   in the Flavor enumerator of the JetCorrFactos class.		      
 * For flavor depdendent correction factors the first entry in the vector  
   (corresponding to NONE) is invalid and should be set to -1. It will not 
   be considered by the class structure though.			      

Make sure that all elements of the argument vector to this contructor are  
in accordance with these rules.
----- End Fatal Exception -------------------------------------------------

Expand to see more addon errors ...

fastsim2

slava77 · 2021-07-15T12:31:54Z

I don't have time to understand what's going on here, so I'll revert the changes to TauJetCorrFactorsProducer.cc and leave the migration of that plugin out of this PR.

apparently that was not enough

guitargeek · 2021-07-17T09:30:46Z

Right, I think I found the problem now. I naively replaced std::vector<float>(1, 1) with std::vector<float>{1.0, 1.0} in the JetCorrFactorsProducer. I hope it works now!

cmsbuild · 2021-07-17T09:40:21Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34368/24048

This PR adds an extra 20KB to repository

cmsbuild · 2021-07-17T09:40:43Z

Pull request #34368 was updated. @perrotta, @jpata, @cmsbuild, @slava77 can you please check and sign again.

perrotta · 2021-07-17T09:40:55Z

please test

cmsbuild · 2021-07-17T12:53:22Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9eaab1/16940/summary.html
COMMIT: abd333a
CMSSW: CMSSW_12_0_X_2021-07-16-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/34368/16940/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 38
DQMHistoTests: Total histograms compared: 2786302
DQMHistoTests: Total failures: 7
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2786273
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 37 files compared)
Checked 160 log files, 37 edm output root files, 38 DQM output files
TriggerResults: no differences found

slava77 · 2021-07-20T15:47:35Z

+reconstruction

for #34368 abd333a

code changes are technical, they are in line with the PR description
jenkins tests pass and comparisons with the baseline show no differences

cmsbuild · 2021-07-20T15:47:58Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy, @perrotta (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2021-07-20T16:54:24Z

+1

cmsbuild added this to the CMSSW_12_0_X milestone Jul 6, 2021

cmsbuild added code-checks-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Jul 6, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 6, 2021

Dr15Jones reviewed Jul 6, 2021

View reviewed changes

cmsbuild added tests-started code-checks-pending tests-pending and removed tests-pending code-checks-approved tests-started labels Jul 7, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 7, 2021

cmsbuild added tests-started and removed tests-pending labels Jul 7, 2021

cmsbuild removed the tests-started label Jul 7, 2021

cmsbuild added code-checks-pending tests-pending and removed code-checks-approved tests-rejected labels Jul 17, 2021

guitargeek added 2 commits July 17, 2021 11:30

Migrate most PhysicsTools/PatAlgos plugins to ESGetToken

01eb3b5

Use EDPutToken in PATMuonProducer plus other changes

abd333a

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 17, 2021

cmsbuild added tests-started and removed tests-pending labels Jul 17, 2021

cmsbuild added tests-approved and removed tests-started labels Jul 17, 2021

cmsbuild added fully-signed reconstruction-approved and removed pending-signatures reconstruction-pending labels Jul 20, 2021

cmsbuild added orp-approved and removed orp-pending labels Jul 20, 2021

cmsbuild merged commit 67967cd into cms-sw:master Jul 20, 2021

guitargeek deleted the PhysicsTools_PatAlgos_3 branch July 20, 2021 17:29

guitargeek mentioned this pull request Jul 25, 2021

Merge .h and .cc files of plugins in several PhysicsTools packages #34615

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate most PhysicsTools/PatAlgos plugins to ESGetToken #34368

Migrate most PhysicsTools/PatAlgos plugins to ESGetToken #34368

guitargeek commented Jul 6, 2021 •

edited

cmsbuild commented Jul 6, 2021

cmsbuild commented Jul 6, 2021

Dr15Jones Jul 6, 2021

slava77 commented Jul 7, 2021

guitargeek commented Jul 7, 2021

cmsbuild commented Jul 7, 2021

cmsbuild commented Jul 7, 2021

slava77 commented Jul 7, 2021

guitargeek commented Jul 7, 2021

slava77 commented Jul 7, 2021

cmsbuild commented Jul 7, 2021

slava77 commented Jul 15, 2021

guitargeek commented Jul 17, 2021

cmsbuild commented Jul 17, 2021

cmsbuild commented Jul 17, 2021

perrotta commented Jul 17, 2021

cmsbuild commented Jul 17, 2021

slava77 commented Jul 20, 2021

cmsbuild commented Jul 20, 2021

qliphy commented Jul 20, 2021

Migrate most PhysicsTools/PatAlgos plugins to ESGetToken #34368

Migrate most PhysicsTools/PatAlgos plugins to ESGetToken #34368

Conversation

guitargeek commented Jul 6, 2021 • edited

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Jul 6, 2021

cmsbuild commented Jul 6, 2021

Dr15Jones Jul 6, 2021

Choose a reason for hiding this comment

slava77 commented Jul 7, 2021

guitargeek commented Jul 7, 2021

cmsbuild commented Jul 7, 2021

cmsbuild commented Jul 7, 2021

slava77 commented Jul 7, 2021

guitargeek commented Jul 7, 2021

slava77 commented Jul 7, 2021

cmsbuild commented Jul 7, 2021

Unit Tests

RelVals

RelVals-INPUT

AddOn Tests

slava77 commented Jul 15, 2021

guitargeek commented Jul 17, 2021

cmsbuild commented Jul 17, 2021

cmsbuild commented Jul 17, 2021

perrotta commented Jul 17, 2021

cmsbuild commented Jul 17, 2021

Comparison Summary

slava77 commented Jul 20, 2021

cmsbuild commented Jul 20, 2021

qliphy commented Jul 20, 2021

guitargeek commented Jul 6, 2021 •

edited