Reconstruct new electrons with H/E (cone) > 0.15 and H/E (tower) < 0.15: 105X #25699

Sam-Harper · 2019-01-18T10:12:56Z

Dear All,

Due to time zone differences, I'm just going to submit this PR by Brent now. All the work is his, I just did the rebase for speed so we can start the testing again.

rebase of #25440

Best,
Sam

cmsbuild · 2019-01-18T10:13:25Z

The code-checks are being triggered in jenkins.

cmsbuild · 2019-01-18T10:20:07Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-25699/8075

This PR adds an extra 92KB to repository

cmsbuild · 2019-01-18T10:20:34Z

A new Pull Request was created by @Sam-Harper (Sam Harper) for master.

It involves the following packages:

RecoEgamma/EgammaElectronAlgos
RecoEgamma/EgammaElectronProducers
RecoParticleFlow/PFProducer

@perrotta, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@mmarionncern, @jainshilpi, @cbernet, @rovere, @lgray, @bachtis, @hatakeyamak, @varuns23, @seemasharmafnal this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

perrotta · 2019-01-18T10:31:59Z

please test

cmsbuild · 2019-01-18T10:32:22Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/32693/console Started: 2019/01/18 12:35

cmsbuild · 2019-01-18T12:20:07Z

-1

Tested at: d11fef0

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-25699/32693/summary.html

I found follow errors while testing this PR

Failed tests: RelVals AddOn

RelVals:

When I ran the RelVals I found an error in the following workflows:
140.53 step2

runTheMatrix-results/140.53_RunHI2011+RunHI2011+RECOHID11+HARVESTDHI/step2_RunHI2011+RunHI2011+RECOHID11+HARVESTDHI.log

135.4 step1

runTheMatrix-results/135.4_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS/step1_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS.log

5.1 step1

runTheMatrix-results/5.1_TTbar+TTbarFS+HARVESTFS/step1_TTbar+TTbarFS+HARVESTFS.log

1001.0 step2

runTheMatrix-results/1001.0_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVDSIPIXELCALRUN1+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4+ALCAHARVD5/step2_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVDSIPIXELCALRUN1+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4+ALCAHARVD5.log

1000.0 step2

runTheMatrix-results/1000.0_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT/step2_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT.log

140.56 step2

runTheMatrix-results/140.56_RunHI2018+RunHI2018+RECOHID18+HARVESTDHI18/step2_RunHI2018+RunHI2018+RECOHID18+HARVESTDHI18.log

4.53 step3

runTheMatrix-results/4.53_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT/step3_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT.log

136.731 step3

runTheMatrix-results/136.731_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2/step3_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2.log

136.788 step3

runTheMatrix-results/136.788_RunSinglePh2017B+RunSinglePh2017B+HLTDR2_2017+RECODR2_2017reHLT_skimSinglePh_Prompt+HARVEST2017/step3_RunSinglePh2017B+RunSinglePh2017B+HLTDR2_2017+RECODR2_2017reHLT_skimSinglePh_Prompt+HARVEST2017.log

1306.0 step3

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step3_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log

1330.0 step3

runTheMatrix-results/1330.0_ZMM_13+ZMM_13+DIGIUP15+RECOUP15_L1TMuDQM+HARVESTUP15_L1TMuDQM/step3_ZMM_13+ZMM_13+DIGIUP15+RECOUP15_L1TMuDQM+HARVESTUP15_L1TMuDQM.log

10042.0 step3

runTheMatrix-results/10042.0_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017/step3_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017.log

136.85 step3

runTheMatrix-results/136.85_RunEGamma2018A+RunEGamma2018A+HLTDR2_2018+RECODR2_2018reHLT_skimEGamma_Prompt_L1TEgDQM+HARVEST2018_L1TEgDQM/step3_RunEGamma2018A+RunEGamma2018A+HLTDR2_2018+RECODR2_2018reHLT_skimEGamma_Prompt_L1TEgDQM+HARVEST2018_L1TEgDQM.log

9.0 step3

runTheMatrix-results/9.0_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST/step3_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST.log

158.0 step3

runTheMatrix-results/158.0_HydjetQ_B12_5020GeV_2018_ppReco+HydjetQ_B12_5020GeV_2018_ppReco+DIGIHI2018PPRECO+RECOHI2018PPRECO+ALCARECOHI2018PPRECO+HARVESTHI2018PPRECO/step3_HydjetQ_B12_5020GeV_2018_ppReco+HydjetQ_B12_5020GeV_2018_ppReco+DIGIHI2018PPRECO+RECOHI2018PPRECO+ALCARECOHI2018PPRECO+HARVESTHI2018PPRECO.log

25.0 step3

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT/step3_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT.log

11624.0 step3

runTheMatrix-results/11624.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2019_GenSimFull+DigiFull_2019+RecoFull_2019+HARVESTFull_2019+ALCAFull_2019/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2019_GenSimFull+DigiFull_2019+RecoFull_2019+HARVESTFull_2019+ALCAFull_2019.log

10024.0 step3

runTheMatrix-results/10024.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017.log

10824.0 step3

runTheMatrix-results/10824.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+HARVESTFull_2018+ALCAFull_2018+NanoFull_2018/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+HARVESTFull_2018+ALCAFull_2018+NanoFull_2018.log

25202.0 step3

runTheMatrix-results/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25/step3_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25.log

10224.0 step3

runTheMatrix-results/10224.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017PU_GenSimFull+DigiFullPU_2017PU+RecoFullPU_2017PU+HARVESTFullPU_2017PU/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017PU_GenSimFull+DigiFullPU_2017PU+RecoFullPU_2017PU+HARVESTFullPU_2017PU.log

20034.0 step3

runTheMatrix-results/20034.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D17_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D17+RecoFullGlobal_2023D17+HARVESTFullGlobal_2023D17/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D17_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D17+RecoFullGlobal_2023D17+HARVESTFullGlobal_2023D17.log

21234.0 step3

runTheMatrix-results/21234.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D21_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D21+RecoFullGlobal_2023D21+HARVESTFullGlobal_2023D21/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D21_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D21+RecoFullGlobal_2023D21+HARVESTFullGlobal_2023D21.log

27434.0 step3

runTheMatrix-results/27434.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D35_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D35+RecoFullGlobal_2023D35+HARVESTFullGlobal_2023D35/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D35_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D35+RecoFullGlobal_2023D35+HARVESTFullGlobal_2023D35.log

250202.181 step4

runTheMatrix-results/250202.181_TTbar_13UP18+TTbar_13UP18+PREMIXUP18_PU25+DIGIPRMXLOCALUP18_PU25+RECOPRMXUP18_PU25+HARVESTUP18_PU25/step4_TTbar_13UP18+TTbar_13UP18+PREMIXUP18_PU25+DIGIPRMXLOCALUP18_PU25+RECOPRMXUP18_PU25+HARVESTUP18_PU25.log

AddOn:

I found errors in the following addon tests:

cmsDriver.py TTbar_8TeV_TuneCUETP8M1_cfi --conditions auto:run1_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot Realistic8TeVCollision : FAILED - time: date Fri Jan 18 13:10:07 2019-date Fri Jan 18 13:08:53 2019 s - exit: 18688
cmsDriver.py RelVal -s HLT:Fake2,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_Fake2 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --processName=HLTRECO --filein file:RelVal_Raw_Fake2_DATA.root --fileout file:RelVal_Raw_Fake2_DATA_HLT_RECO.root : FAILED - time: date Fri Jan 18 13:10:45 2019-date Fri Jan 18 13:08:58 2019 s - exit: 18688
cmsDriver.py RelVal -s HLT:PRef,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_PRef --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_PRef_DATA.root --fileout file:RelVal_Raw_PRef_DATA_HLT_RECO.root : FAILED - time: date Fri Jan 18 13:11:49 2019-date Fri Jan 18 13:09:03 2019 s - exit: 18688
cmsDriver.py RelVal -s HLT:HIon,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_HIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018_pp_on_AA --processName=HLTRECO --filein file:RelVal_Raw_HIon_DATA.root --fileout file:RelVal_Raw_HIon_DATA_HLT_RECO.root : FAILED - time: date Fri Jan 18 13:13:44 2019-date Fri Jan 18 13:09:05 2019 s - exit: 18688
cmsDriver.py RelVal -s HLT:GRun,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_GRun --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_GRun_MC.root --fileout file:RelVal_Raw_GRun_MC_HLT_RECO.root : FAILED - time: date Fri Jan 18 13:18:30 2019-date Fri Jan 18 13:09:09 2019 s - exit: 18688
cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc_l1stage1 --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_25ns : FAILED - time: date Fri Jan 18 13:10:29 2019-date Fri Jan 18 13:09:10 2019 s - exit: 18688
cmsDriver.py RelVal -s HLT:PRef,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_PRef --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_PRef_MC.root --fileout file:RelVal_Raw_PRef_MC_HLT_RECO.root : FAILED - time: date Fri Jan 18 13:14:57 2019-date Fri Jan 18 13:09:14 2019 s - exit: 18688
cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_2016 : FAILED - time: date Fri Jan 18 13:10:34 2019-date Fri Jan 18 13:09:18 2019 s - exit: 18688
cmsDriver.py RelVal -s HLT:PIon,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_PIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_PIon_DATA.root --fileout file:RelVal_Raw_PIon_DATA_HLT_RECO.root : FAILED - time: date Fri Jan 18 13:11:48 2019-date Fri Jan 18 13:09:22 2019 s - exit: 18688
cmsDriver.py RelVal -s HLT:Fake,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run1_mc_Fake --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --processName=HLTRECO --filein file:RelVal_Raw_Fake_MC.root --fileout file:RelVal_Raw_Fake_MC_HLT_RECO.root : FAILED - time: date Fri Jan 18 13:14:32 2019-date Fri Jan 18 13:09:25 2019 s - exit: 18688
cmsDriver.py RelVal -s HLT:HIon,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_HIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018_pp_on_AA --processName=HLTRECO --filein file:RelVal_Raw_HIon_MC.root --fileout file:RelVal_Raw_HIon_MC_HLT_RECO.root : FAILED - time: date Fri Jan 18 13:16:25 2019-date Fri Jan 18 13:09:28 2019 s - exit: 18688
cmsDriver.py RelVal -s HLT:Fake2,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_Fake2 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --processName=HLTRECO --filein file:RelVal_Raw_Fake2_MC.root --fileout file:RelVal_Raw_Fake2_MC_HLT_RECO.root : FAILED - time: date Fri Jan 18 13:14:26 2019-date Fri Jan 18 13:09:33 2019 s - exit: 18688
cmsDriver.py RelVal -s HLT:Fake1,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_Fake1 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_25ns --processName=HLTRECO --filein file:RelVal_Raw_Fake1_MC.root --fileout file:RelVal_Raw_Fake1_MC_HLT_RECO.root : FAILED - time: date Fri Jan 18 13:15:22 2019-date Fri Jan 18 13:09:34 2019 s - exit: 18688
cmsDriver.py RelVal -s HLT:Fake,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run1_data_Fake --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --processName=HLTRECO --filein file:RelVal_Raw_Fake_DATA.root --fileout file:RelVal_Raw_Fake_DATA_HLT_RECO.root : FAILED - time: date Fri Jan 18 13:11:34 2019-date Fri Jan 18 13:09:41 2019 s - exit: 18688
cmsDriver.py RelVal -s HLT:GRun,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_GRun --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_GRun_DATA.root --fileout file:RelVal_Raw_GRun_DATA_HLT_RECO.root : FAILED - time: date Fri Jan 18 13:16:06 2019-date Fri Jan 18 13:09:46 2019 s - exit: 18688
cmsDriver.py RelVal -s HLT:Fake1,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_Fake1 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_25ns --processName=HLTRECO --filein file:RelVal_Raw_Fake1_DATA.root --fileout file:RelVal_Raw_Fake1_DATA_HLT_RECO.root : FAILED - time: date Fri Jan 18 13:12:38 2019-date Fri Jan 18 13:09:48 2019 s - exit: 18688
cmsDriver.py RelVal -s HLT:PIon,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_PIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_PIon_MC.root --fileout file:RelVal_Raw_PIon_MC_HLT_RECO.root : FAILED - time: date Fri Jan 18 13:14:58 2019-date Fri Jan 18 13:09:50 2019 s - exit: 18688

cmsbuild · 2019-01-18T12:20:11Z

Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped)

Sam-Harper · 2019-01-18T13:27:48Z

My bad, I forgot a change. Fixed now.

perrotta · 2019-01-21T16:18:04Z

Looking at the automatic test outputs, it is evident the larger number of ged gsf tracks and electron reconstructed in basically all tested workflows, which propagate as PAT electrons in miniAOD.
At a first glance, the number of jets doesn't change, but their tag ids do.
Also affected are the BTAG, PAT taus, and PAT MET (quite likely due to the extra electron objects)

Does this comply with what you declared in #25440 (comment) , i.e. "this PR ... should add gedGsfElectrons to the event but should not alter the number of particle flow candidates"?

Please check and report here, so that we can continue with the review.

Sam-Harper · 2019-01-21T16:27:17Z

So I'm doing a check again (right now) to be absolutely sure we can do not change the PF candidates. After that we can discuss proceeding, one option is to merge it into 10_6_X and give the other pogs a bit more time to adapt. But I think for finding bugs and the like, its better to be in 10_5_X.

Sam-Harper · 2019-01-21T18:42:45Z

So I was looking into this. The MET I see changing is the MET uncertainties not the MET itself. I believe this to be because the MET uncertainties have a bug when they scale and shift down electrons. They scale and shift down all electrons, not just the PF electrons which are used in the met collection.

eg process.shiftedPatMETCorrElectronEnDownPuppi = cms.EDProducer("ShiftedParticleMETcorrInputProducer",
srcOriginal = cms.InputTag("selectedPatElectrons"),
srcShifted = cms.InputTag("shiftedPatElectronEnDownPuppi")
)
should really select electrons that are PF electrons. So this (if I understand correctly) is already broken and changing it further should not be an issue.

The rest effected are BTAG IDs and Tau IDs which ideally should not be sensitive to the e/gamma reconstruction preselection but apparently are. I will ping the Tau/BTag convenors with their preferred option

put this into this 10_5 pre release.
put this into 10_6_0_pre1

Actually now I've done some digging on the tau ids, I understand the changes. The H/E used in the tau ID bares little to no resemblance to the one used by E/gamma. Hence the tau ID is overly sensitive to the reco::GsfElectron selection. Probably the same for the b-tag.

Sam-Harper · 2019-01-21T19:10:51Z

So thinking more on this, because we're only changing the IDs and not the reco objects themselves, I think this should really make it into 10_5 as it would allow a release for them to retune those IDs if required. Its not changing the actual reco quantities, just the ID results. Also it doesnt impact the main point of 10_5 which is PF cluster corrections.

Anyways I've pinged everybody with you in +cc

Sam-Harper · 2019-01-23T07:49:50Z

So this is not a problem for taus at all. BTV are still deciding (they were noticed late in the process) but I really dont think its an issue and it should go in, if it does cause a problem, its better to know sooner rather than later.

So to summarise, this might cause some impact for BTV (which may even be positive, we're now better at reconstructing electrons near jets) but nobody else. We will need relvals to evaluate this so it'll have to go into either 10_5_X_pre1 or 10_6_X_pre1 and if in the unlikely event we break BTV in 10_5_X_pre1, is it really going to be unrecoverable? Is pre1 a critical release for BTV?

Sam-Harper · 2019-01-23T11:03:34Z

okay talked to Mauro, its fine, the ones changing are not so important and they dont have major changes to validate in 10_5_0_pre1 so if in the extremely unlikely event it fails validation here, it wont hide another faliure.

perrotta · 2019-01-23T18:19:45Z

I verified the performance with the usual TTbar PU wf 25202

Timing changes only by a tiny amount:

The same excluding the first 1 events
  delta/mean delta/orJob     original                   new       module name
  ---------- ------------     --------                  ----       ------------
   +0.420189      +0.00%         0.64 ms/ev ->         0.98 ms/ev gedGsfElectrons
   +0.299101      +0.00%         0.63 ms/ev ->         0.86 ms/ev gedPhotons
   +0.238370      +0.00%         0.71 ms/ev ->         0.91 ms/ev photonMVAValueMapProducer
   +0.201284      +0.00%         1.03 ms/ev ->         1.26 ms/ev electronMVAValueMapProducer
  ---------- ------------     --------                  ----       ------------
Job total:  8.86164 s/ev ==> 8.95411 s/ev

Additions to the reco output are also manageable:

Compare packed values
-----------------------------------------------------------------
   or, B         new, B      delta, B   delta, %   deltaJ, %    branch 
-----------------------------------------------------------------
   1027.0 ->      1049.8         23      2.2   0.00     recoGsfElectrons_gedGsfElectrons__RECO.
     82.6 ->        84.2          2      2.0   0.00     recoPFCandidatesrecoPFCandidaterecoPFCandidatesrecoPFCandidateedmrefhelperFindUsingAdvanceedmRefsedmValueMap_particleBasedIsolation_gedGsfElectrons_RECO.
   3053.7 ->      3055.0          1      0.0   0.00     recoTracks_conversionStepTracks__RECO.
-------------------------------------------------------------
  2442231 ->     2442254         23             0.0     ALL BRANCHES

as well as in miniAOD

Compare packed values
-----------------------------------------------------------------
   or, B         new, B      delta, B   delta, %   deltaJ, %    branch 
-----------------------------------------------------------------
    420.8 ->       457.9         37      8.4   0.05     recoConversions_reducedEgamma_reducedConversions_RECO.
   2133.0 ->      2177.0         44      2.0   0.06     patElectrons_slimmedElectrons__RECO.
    571.2 ->       584.1         13      2.2   0.02     recoSuperClusters_reducedEgamma_reducedSuperClusters_RECO.
    424.0 ->       434.0         10      2.3   0.01     recoGsfTracks_reducedEgamma_reducedGsfTracks_RECO.
     70.2 ->        71.1          1      1.3   0.00     recoGsfElectronCores_reducedEgamma_reducedGedGsfElectronCores_RECO.
    902.7 ->       927.5         25      2.7   0.04     EcalRecHitsSorted_reducedEgamma_reducedEBRecHits_RECO.
    596.9 ->       602.2          5      0.9   0.01     HBHERecHitsSorted_reducedEgamma_reducedHBHEHits_RECO.
    708.5 ->       724.1         16      2.2   0.02     recoCaloClusters_reducedEgamma_reducedEBEEClusters_RECO.
-------------------------------------------------------------
    69590 ->       69741        151             0.2     ALL BRANCHES

perrotta · 2019-01-23T18:32:35Z

+1

A few additional ged electrons selected, with the intent of improving electron efficiency within jets
The additional electrons are not considered in Particle Flow. Some effect of their presence is visible in Tau ID and BTV, and the two groups have agreed on merging this pull request so that they can fine tune their algorithms, if even needed
For that, I support having this already merged now in 10_5_0_pre2, so that the validation can start at once on RelVals
Another effect is seen in the MET uncertainties, which likely witnesses a bug in the way electrons enter in their calculation, see Reconstruct new electrons with H/E (cone) > 0.15 and H/E (tower) < 0.15: 105X #25699 (comment): this has to be followed up
Jenkins tests pass and show no differences in other parts

cmsbuild · 2019-01-23T18:33:02Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

slava77 · 2019-01-23T18:37:20Z

On 1/23/19 10:19 AM, perrotta wrote: I verified the performance with the usual TTbar PU wf 25202

I guess this is still OK for timing purposes. However, this is a 2016 workflow with PU35 and may show a factor of a few different results (depending on a module) from more typical current reference with PU50 (11024 or its premixed version like 250202.18)

perrotta · 2019-01-23T18:55:55Z

Ok, @slava77 : I am rerunning the tests with 11024, so that numbers can remain here as reference.
I don't expect relevant changes in the conclusions, though

perrotta · 2019-01-23T22:36:50Z

Preformance measurements remade with wf 11024

Reco event size:

Compare packed values
-----------------------------------------------------------------
   or, B         new, B      delta, B   delta, %   deltaJ, %    branch 
-----------------------------------------------------------------
    788.4 ->       792.8          4      0.6   0.00     recoCandidateedmPtrrecoTemplatedSoftLeptonTagInfos_softPFElectronsTagInfos__RECO.
   1070.7 ->      1112.9         42      3.9   0.00     recoGsfElectrons_gedGsfElectrons__RECO.
   9124.2 ->      9134.2         10      0.1   0.00     EcalRecHitsSorted_reducedEcalRecHitsEB__RECO.
     84.7 ->        85.7          1      1.3   0.00     recoPFCandidatesrecoPFCandidaterecoPFCandidatesrecoPFCandidateedmrefhelperFindUsingAdvanceedmRefsedmValueMap_particleBasedIsolation_gedGsfElectrons_RECO.
-------------------------------------------------------------
  3838045 ->     3838097         52             0.0     ALL BRANCHES

miniAOD event size:

Compare packed values
-----------------------------------------------------------------
   or, B         new, B      delta, B   delta, %   deltaJ, %    branch 
-----------------------------------------------------------------
    557.0 ->       575.7         19      3.3   0.02     recoConversions_reducedEgamma_reducedConversions_RECO.
   2184.3 ->      2258.3         74      3.3   0.08     patElectrons_slimmedElectrons__RECO.
    483.6 ->       507.8         24      4.9   0.03     recoSuperClusters_reducedEgamma_reducedSuperClusters_RECO.
    456.2 ->       492.4         36      7.6   0.04     recoGsfTracks_reducedEgamma_reducedGsfTracks_RECO.
    390.5 ->       395.5          5      1.3   0.01     EcalRecHitsSorted_reducedEgamma_reducedEERecHits_RECO.
     71.8 ->        72.7          1      1.3   0.00     recoGsfElectronCores_reducedEgamma_reducedGedGsfElectronCores_RECO.
    815.4 ->       856.6         41      4.9   0.04     EcalRecHitsSorted_reducedEgamma_reducedEBRecHits_RECO.
    374.1 ->       378.8          5      1.3   0.01     EcalRecHitsSorted_reducedEgamma_reducedESRecHits_RECO.
    567.6 ->       579.6         12      2.1   0.01     recoCaloClusters_reducedEgamma_reducedESClusters_RECO.
    873.7 ->       901.4         28      3.1   0.03     HBHERecHitsSorted_reducedEgamma_reducedHBHEHits_RECO.
    648.7 ->       671.4         23      3.4   0.02     recoCaloClusters_reducedEgamma_reducedEBEEClusters_RECO.
-------------------------------------------------------------
    93479 ->       93748        269             0.3     ALL BRANCHES

CPU time

The same excluding the first 1 events
  delta/mean delta/orJob     original                   new       module name
  ---------- ------------     --------                  ----       ------------
   +0.637106      +0.00%         0.68 ms/ev ->         1.32 ms/ev gedGsfElectrons
   +0.571193      +0.00%         0.65 ms/ev ->         1.18 ms/ev gedPhotons
   +0.505899      +0.00%         1.02 ms/ev ->         1.70 ms/ev electronMVAValueMapProducer
   +0.500613      +0.00%         0.68 ms/ev ->         1.13 ms/ev photonMVAValueMapProducer
   +0.316154      +0.01%         3.78 ms/ev ->         5.20 ms/ev ak8PFJetsPuppiSoftDrop
   +0.302332      +0.05%        22.05 ms/ev ->        29.91 ms/ev siPixelClustersPreSplitting
   +0.250454      +0.12%        67.41 ms/ev ->        86.71 ms/ev lowPtTripletStepHitTriplets
   +0.235536      +0.01%         6.32 ms/ev ->         8.01 ms/ev particleFlowRecHitPS
  ---------- ------------     --------                  ----       ------------
Job total:  15.7714 s/ev ==> 15.8399 s/ev

(where most of the differences come from unrelated modules and are likely due to some normalization issue: in any case I don't rerun it, because it confirms anyhow that the additional load due to this PR is really limited)

fabiocos · 2019-01-24T09:14:10Z

code-checks

cmsbuild · 2019-01-24T09:14:40Z

The code-checks are being triggered in jenkins.

cmsbuild · 2019-01-24T09:21:24Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-25699/8161

This PR adds an extra 96KB to repository

fabiocos · 2019-01-24T09:45:27Z

+1

Brent Stone and others added 3 commits January 17, 2019 16:40

reconstruct new electrons with H/E (tower) < 0.15

48d78f4

making cosmetic changes

6b28c78

altering PF preselection to remove newly added electrons

d11fef0

cmsbuild added this to the CMSSW_10_5_X milestone Jan 18, 2019

cmsbuild added code-checks-pending comparison-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Jan 18, 2019

cmsbuild added code-checks-approved and removed code-checks-pending labels Jan 18, 2019

cmsbuild added tests-started and removed tests-pending labels Jan 18, 2019

cmsbuild added comparison-notrun tests-rejected and removed comparison-pending tests-started labels Jan 18, 2019

fixing config oversite

20af78e

cmsbuild removed code-checks-approved comparison-notrun tests-rejected labels Jan 18, 2019

perrotta mentioned this pull request Jan 21, 2019

Reconstruct new electrons with H/E (cone) > 0.15 and H/E (tower) < 0.15 #25440

Closed

cmsbuild added fully-signed reconstruction-approved and removed pending-signatures reconstruction-pending labels Jan 23, 2019

perrotta mentioned this pull request Jan 23, 2019

Do MET uncertainties have a bug when they scale and shift down electrons? #25751

Closed

cmsbuild added code-checks-pending and removed code-checks-approved labels Jan 24, 2019

cmsbuild added code-checks-approved and removed code-checks-pending labels Jan 24, 2019

cmsbuild added orp-approved and removed orp-pending labels Jan 24, 2019

cmsbuild merged commit 23bcf95 into cms-sw:master Jan 24, 2019

perrotta mentioned this pull request Jan 27, 2019

Complete low pT electron chain (2) #25753

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reconstruct new electrons with H/E (cone) > 0.15 and H/E (tower) < 0.15: 105X #25699

Reconstruct new electrons with H/E (cone) > 0.15 and H/E (tower) < 0.15: 105X #25699

Sam-Harper commented Jan 18, 2019

cmsbuild commented Jan 18, 2019

cmsbuild commented Jan 18, 2019

cmsbuild commented Jan 18, 2019

perrotta commented Jan 18, 2019

cmsbuild commented Jan 18, 2019 •

edited

cmsbuild commented Jan 18, 2019

cmsbuild commented Jan 18, 2019

Sam-Harper commented Jan 18, 2019

perrotta commented Jan 21, 2019

Sam-Harper commented Jan 21, 2019

Sam-Harper commented Jan 21, 2019

Sam-Harper commented Jan 21, 2019

Sam-Harper commented Jan 23, 2019 •

edited

Sam-Harper commented Jan 23, 2019

perrotta commented Jan 23, 2019

perrotta commented Jan 23, 2019

cmsbuild commented Jan 23, 2019

slava77 commented Jan 23, 2019 via email

perrotta commented Jan 23, 2019

perrotta commented Jan 23, 2019

fabiocos commented Jan 24, 2019

cmsbuild commented Jan 24, 2019

cmsbuild commented Jan 24, 2019

fabiocos commented Jan 24, 2019

Reconstruct new electrons with H/E (cone) > 0.15 and H/E (tower) < 0.15: 105X #25699

Reconstruct new electrons with H/E (cone) > 0.15 and H/E (tower) < 0.15: 105X #25699

Conversation

Sam-Harper commented Jan 18, 2019

cmsbuild commented Jan 18, 2019

cmsbuild commented Jan 18, 2019

cmsbuild commented Jan 18, 2019

perrotta commented Jan 18, 2019

cmsbuild commented Jan 18, 2019 • edited

cmsbuild commented Jan 18, 2019

cmsbuild commented Jan 18, 2019

Sam-Harper commented Jan 18, 2019

perrotta commented Jan 21, 2019

Sam-Harper commented Jan 21, 2019

Sam-Harper commented Jan 21, 2019

Sam-Harper commented Jan 21, 2019

Sam-Harper commented Jan 23, 2019 • edited

Sam-Harper commented Jan 23, 2019

perrotta commented Jan 23, 2019

perrotta commented Jan 23, 2019

cmsbuild commented Jan 23, 2019

slava77 commented Jan 23, 2019 via email

perrotta commented Jan 23, 2019

perrotta commented Jan 23, 2019

fabiocos commented Jan 24, 2019

cmsbuild commented Jan 24, 2019

cmsbuild commented Jan 24, 2019

fabiocos commented Jan 24, 2019

cmsbuild commented Jan 18, 2019 •

edited

Sam-Harper commented Jan 23, 2019 •

edited