Adapting EGRegression Modifiers to 106X and adding V3 modifier #26892

Sam-Harper · 2019-05-22T15:12:44Z

PR description:

This is a speculative PR which may or may not be final. Its more alerting people to what is coming as it'll look a lot like this with perhaps some variables changed.

First there are bug fixes to the V1 and V2 regression modifiers. These modifiers cache the event setup object. When using ModifiedElectronProducer/ ModifiedPhotonProducer this updates every lumisection. However at somepoint after 102X, the event setup object is deleted and should not be cached. Therefore it was modified to properly cache just the calo geometry. When this code was used in reco, it just happened that setEventContent must have been called every event so we didnt see this bug. This was a fun debug btw.

Then it adds the V3 regression modifier. Currently its fully backwards compatible with the existing regressions but adds features necessary for the new regression. The new regression may need more changes now based on studies ongoing.

So this might be all the code we need but then again might have some more changes coming.

Right now even though its using the V3 regression modifer for the old regressions, no changes are expected.

cmsbuild · 2019-05-22T15:13:22Z

The code-checks are being triggered in jenkins.

cmsbuild · 2019-05-22T15:19:15Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-26892/9927

This PR adds an extra 24KB to repository

cmsbuild · 2019-05-22T15:19:49Z

A new Pull Request was created by @Sam-Harper (Sam Harper) for master.

It involves the following packages:

RecoEgamma/EgammaTools

@perrotta, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@jainshilpi, @varuns23, @lgray this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

slava77 · 2019-05-22T15:23:23Z

@cmsbuild please test

cmsbuild · 2019-05-22T15:24:01Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/428/console Started: 2019/05/22 17:24

cmsbuild · 2019-05-22T16:15:08Z

-1

Tested at: 6fc793e

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-472c6b/428/summary.html

I found follow errors while testing this PR

Failed tests: RelVals AddOn

RelVals:

When I ran the RelVals I found an error in the following workflows:
135.4 step1

runTheMatrix-results/135.4_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS/step1_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS.log

5.1 step1

runTheMatrix-results/5.1_TTbar+TTbarFS+HARVESTFS/step1_TTbar+TTbarFS+HARVESTFS.log

140.53 step2

runTheMatrix-results/140.53_RunHI2011+RunHI2011+RECOHID11+HARVESTDHI/step2_RunHI2011+RunHI2011+RECOHID11+HARVESTDHI.log

140.56 step2

runTheMatrix-results/140.56_RunHI2018+RunHI2018+RECOHID18+HARVESTDHI18/step2_RunHI2018+RunHI2018+RECOHID18+HARVESTDHI18.log

4.53 step3

runTheMatrix-results/4.53_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT/step3_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT.log

136.731 step3

runTheMatrix-results/136.731_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2/step3_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2.log

136.788 step3

runTheMatrix-results/136.788_RunSinglePh2017B+RunSinglePh2017B+HLTDR2_2017+RECODR2_2017reHLT_skimSinglePh_Prompt+HARVEST2017/step3_RunSinglePh2017B+RunSinglePh2017B+HLTDR2_2017+RECODR2_2017reHLT_skimSinglePh_Prompt+HARVEST2017.log

1001.0 step2

runTheMatrix-results/1001.0_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVDSIPIXELCALRUN1+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4+ALCAHARVD5/step2_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVDSIPIXELCALRUN1+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4+ALCAHARVD5.log

1000.0 step2

runTheMatrix-results/1000.0_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT/step2_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT.log

1306.0 step3

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step3_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log

136.85 step3

runTheMatrix-results/136.85_RunEGamma2018A+RunEGamma2018A+HLTDR2_2018+RECODR2_2018reHLT_skimEGamma_Offline_L1TEgDQM+HARVEST2018_L1TEgDQM/step3_RunEGamma2018A+RunEGamma2018A+HLTDR2_2018+RECODR2_2018reHLT_skimEGamma_Offline_L1TEgDQM+HARVEST2018_L1TEgDQM.log

9.0 step3

runTheMatrix-results/9.0_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST/step3_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST.log

25.0 step3

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT/step3_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT.log

1330.0 step3

runTheMatrix-results/1330.0_ZMM_13+ZMM_13+DIGIUP15+RECOUP15_L1TMuDQM+HARVESTUP15_L1TMuDQM/step3_ZMM_13+ZMM_13+DIGIUP15+RECOUP15_L1TMuDQM+HARVESTUP15_L1TMuDQM.log

158.0 step3

runTheMatrix-results/158.0_HydjetQ_B12_5020GeV_2018_ppReco+HydjetQ_B12_5020GeV_2018_ppReco+DIGIHI2018PPRECO+RECOHI2018PPRECO+ALCARECOHI2018PPRECO+HARVESTHI2018PPRECO/step3_HydjetQ_B12_5020GeV_2018_ppReco+HydjetQ_B12_5020GeV_2018_ppReco+DIGIHI2018PPRECO+RECOHI2018PPRECO+ALCARECOHI2018PPRECO+HARVESTHI2018PPRECO.log

10042.0 step3

runTheMatrix-results/10042.0_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017/step3_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017.log

10024.0 step3

runTheMatrix-results/10024.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017.log

11624.0 step3

runTheMatrix-results/11624.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2021_GenSimFull+DigiFull_2021+RecoFull_2021+HARVESTFull_2021+ALCAFull_2021/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2021_GenSimFull+DigiFull_2021+RecoFull_2021+HARVESTFull_2021+ALCAFull_2021.log

25202.0 step3

runTheMatrix-results/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25/step3_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25.log

10824.0 step3

runTheMatrix-results/10824.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+HARVESTFull_2018+ALCAFull_2018+NanoFull_2018/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+HARVESTFull_2018+ALCAFull_2018+NanoFull_2018.log

10224.0 step3

runTheMatrix-results/10224.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017PU_GenSimFull+DigiFullPU_2017PU+RecoFullPU_2017PU+HARVESTFullPU_2017PU/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017PU_GenSimFull+DigiFullPU_2017PU+RecoFullPU_2017PU+HARVESTFullPU_2017PU.log

20034.0 step3

runTheMatrix-results/20034.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D17_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D17+RecoFullGlobal_2023D17+HARVESTFullGlobal_2023D17/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D17_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D17+RecoFullGlobal_2023D17+HARVESTFullGlobal_2023D17.log

21234.0 step3

runTheMatrix-results/21234.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D21_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D21+RecoFullGlobal_2023D21+HARVESTFullGlobal_2023D21/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D21_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D21+RecoFullGlobal_2023D21+HARVESTFullGlobal_2023D21.log

27434.0 step3

runTheMatrix-results/27434.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D35_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D35+RecoFullGlobal_2023D35+HARVESTFullGlobal_2023D35/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D35_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D35+RecoFullGlobal_2023D35+HARVESTFullGlobal_2023D35.log

29034.0 step3

runTheMatrix-results/29034.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D41_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D41+RecoFullGlobal_2023D41+HARVESTFullGlobal_2023D41/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D41_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D41+RecoFullGlobal_2023D41+HARVESTFullGlobal_2023D41.log

250202.181 step4

runTheMatrix-results/250202.181_TTbar_13UP18+TTbar_13UP18+PREMIXUP18_PU25+DIGIPRMXLOCALUP18_PU25+RECOPRMXUP18_PU25+HARVESTUP18_PU25/step4_TTbar_13UP18+TTbar_13UP18+PREMIXUP18_PU25+DIGIPRMXLOCALUP18_PU25+RECOPRMXUP18_PU25+HARVESTUP18_PU25.log

AddOn:

I found errors in the following addon tests:

cmsDriver.py TTbar_8TeV_TuneCUETP8M1_cfi --conditions auto:run1_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot Realistic8TeVCollision : FAILED - time: date Wed May 22 18:03:55 2019-date Wed May 22 18:02:13 2019 s - exit: 20736
cmsDriver.py RelVal -s HLT:Fake2,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_Fake2 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --processName=HLTRECO --filein file:RelVal_Raw_Fake2_DATA.root --fileout file:RelVal_Raw_Fake2_DATA_HLT_RECO.root : FAILED - time: date Wed May 22 18:06:17 2019-date Wed May 22 18:02:22 2019 s - exit: 20736
cmsDriver.py RelVal -s HLT:PRef,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_PRef --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_PRef_DATA.root --fileout file:RelVal_Raw_PRef_DATA_HLT_RECO.root : FAILED - time: date Wed May 22 18:07:11 2019-date Wed May 22 18:02:27 2019 s - exit: 20736
cmsDriver.py RelVal -s HLT:HIon,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_HIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018_pp_on_AA --processName=HLTRECO --filein file:RelVal_Raw_HIon_DATA.root --fileout file:RelVal_Raw_HIon_DATA_HLT_RECO.root : FAILED - time: date Wed May 22 18:08:42 2019-date Wed May 22 18:02:29 2019 s - exit: 20736
cmsDriver.py RelVal -s HLT:GRun,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_GRun --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_GRun_MC.root --fileout file:RelVal_Raw_GRun_MC_HLT_RECO.root : FAILED - time: date Wed May 22 18:12:20 2019-date Wed May 22 18:02:30 2019 s - exit: 20736
cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc_l1stage1 --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_25ns : FAILED - time: date Wed May 22 18:04:35 2019-date Wed May 22 18:02:32 2019 s - exit: 20736
cmsDriver.py RelVal -s HLT:PRef,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_PRef --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_PRef_MC.root --fileout file:RelVal_Raw_PRef_MC_HLT_RECO.root : FAILED - time: date Wed May 22 18:08:36 2019-date Wed May 22 18:02:36 2019 s - exit: 20736
cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_2016 : FAILED - time: date Wed May 22 18:04:33 2019-date Wed May 22 18:02:38 2019 s - exit: 20736
cmsDriver.py RelVal -s HLT:PIon,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_PIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_PIon_DATA.root --fileout file:RelVal_Raw_PIon_DATA_HLT_RECO.root : FAILED - time: date Wed May 22 18:07:11 2019-date Wed May 22 18:02:42 2019 s - exit: 20736
cmsDriver.py RelVal -s HLT:Fake,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run1_mc_Fake --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --processName=HLTRECO --filein file:RelVal_Raw_Fake_MC.root --fileout file:RelVal_Raw_Fake_MC_HLT_RECO.root : FAILED - time: date Wed May 22 18:08:04 2019-date Wed May 22 18:02:43 2019 s - exit: 20736
cmsDriver.py RelVal -s HLT:HIon,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_HIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018_pp_on_AA --processName=HLTRECO --filein file:RelVal_Raw_HIon_MC.root --fileout file:RelVal_Raw_HIon_MC_HLT_RECO.root : FAILED - time: date Wed May 22 18:10:15 2019-date Wed May 22 18:02:47 2019 s - exit: 20736
cmsDriver.py RelVal -s HLT:Fake2,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_Fake2 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --processName=HLTRECO --filein file:RelVal_Raw_Fake2_MC.root --fileout file:RelVal_Raw_Fake2_MC_HLT_RECO.root : FAILED - time: date Wed May 22 18:08:20 2019-date Wed May 22 18:02:49 2019 s - exit: 20736
cmsDriver.py RelVal -s HLT:Fake1,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_Fake1 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_25ns --processName=HLTRECO --filein file:RelVal_Raw_Fake1_MC.root --fileout file:RelVal_Raw_Fake1_MC_HLT_RECO.root : FAILED - time: date Wed May 22 18:09:32 2019-date Wed May 22 18:02:50 2019 s - exit: 20736
cmsDriver.py RelVal -s HLT:Fake,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run1_data_Fake --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --processName=HLTRECO --filein file:RelVal_Raw_Fake_DATA.root --fileout file:RelVal_Raw_Fake_DATA_HLT_RECO.root : FAILED - time: date Wed May 22 18:06:58 2019-date Wed May 22 18:03:00 2019 s - exit: 20736
cmsDriver.py RelVal -s HLT:GRun,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_GRun --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_GRun_DATA.root --fileout file:RelVal_Raw_GRun_DATA_HLT_RECO.root : FAILED - time: date Wed May 22 18:11:53 2019-date Wed May 22 18:03:12 2019 s - exit: 35584
cmsDriver.py RelVal -s HLT:Fake1,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_Fake1 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_25ns --processName=HLTRECO --filein file:RelVal_Raw_Fake1_DATA.root --fileout file:RelVal_Raw_Fake1_DATA_HLT_RECO.root : FAILED - time: date Wed May 22 18:07:09 2019-date Wed May 22 18:03:15 2019 s - exit: 20736
cmsDriver.py RelVal -s HLT:PIon,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_PIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_PIon_MC.root --fileout file:RelVal_Raw_PIon_MC_HLT_RECO.root : FAILED - time: date Wed May 22 18:10:58 2019-date Wed May 22 18:03:58 2019 s - exit: 20736

cmsbuild · 2019-05-22T16:15:11Z

Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped)

slava77 · 2019-05-22T16:50:21Z

@Sam-Harper
did this work in the local setup? (using the runTheMatrix.py -l limited ) ?

the errors are like

----- Begin Fatal Exception 22-May-2019 17:40:50 CEST-----------------------
An exception of category 'ScheduleExecutionFailure' occurred while
   [0] Calling beginJob
Exception Message:
Unrunnable schedule
Module run order problem found: 
particleFlow consumes gedGsfElectrons, gedGsfElectrons consumes fixedGridRhoFastjetAll, fixedGridRhoFastjetAll consumes particleFlow
 Running in the threaded framework would lead to indeterminate results.
 Please change order of modules in mentioned Path(s) to avoid inconsistent module ordering.

Sam-Harper · 2019-05-22T17:19:11Z

ah stupid thing on my part, mixing my local testing. Fixed

slava77 · 2019-05-22T17:54:28Z

@cmsbuild please test

@smuzaffar is there a way to make the bot to notice force-pushed updates?

slava77 · 2019-05-22T18:23:18Z

@smuzaffar is there a way to make the bot to notice force-pushed updates?

worse yet, it's not reacting to my please test either

slava77 · 2019-05-22T18:32:53Z

ah, it looks like the problem is on github side
https://www.githubstatus.com/

slava77 · 2019-06-12T17:56:15Z

the CB fits well, but returns roughly the same 1.01.

I'm taking #26892 (comment)
as an OK to move on with this PR.

slava77 · 2019-06-12T18:27:54Z

there is one more oddity:

In re-miniAOD of 2016 wf 136.7611 I see some changes in OOT variables

I do not see EGRegressionModifier* in the config of this workflow.
I guess the diff is from PF SC corrections entering somewhere in the OOT reco chain.
So, this perhaps can be blamed on the choice of the GT for this workflow (auto:run2_data_relval is used , while a more appropriate should be something with payloads more consistent with what was used in the legacy 80X production).
@Sam-Harper @christopheralanwest what do you think?

Sam-Harper · 2019-06-12T20:17:54Z

Thanks Slava, I agree we go as is. It does look like there is some interplay with PF calibrations from the OOT photons and I'll do some more tests tomorrow on this. But I think we can understand this in parallel and to be honest some relvals to debug would be useful as its something that cant be seen in the AOD applications.

slava77 · 2019-06-12T21:13:12Z

+1

for #26892 02920af (in a combination with #27089 a74a716)

code changes are in line with the PR description and the follow up review
jenkins tests pass and comparisons with the baseline show acceptable differences
- all data workflows are affected, even though normally only 2017 should have been affected. This is a feature of the GT PR affecting at this point only the PF SC corrections. This PR correctly applies the 2017 egamma corrections just to the 2017 parts.
  - For Run-1 case it is treated as an acceptable "damage" (or, rather, somewhat random change for a setup that has been inconsistent since a long time).
  - For Run-2 2016 and 2018 the appropriately consistent calibrations are to be introduced later in the UL preparation. Otherwise, this data period is in a state somewhat similar to the Run-1 case.
- the OOT photon changes in the 80X reminiAOD workflow is likely a feature of the workflow using (since a while) an GT inconsistent with the 80X legacy rereco
additionally, based on the local tests with 2017 MC with more events
- supercluster corrected energy error for jet data (esp in the barrel) looks bad, but it's apparently not used
- the slight overshoot in response for photons with low r9 in the barrel is apparently acceptable (and in part could be a feature of tests done without PU, while the reference sample tests [not perfectly matching the UL setup] show that there is no problem)

fabiocos · 2019-06-13T07:26:30Z

+operations

changes to Eras are coherent with the purpose of the PR

cmsbuild · 2019-06-13T07:27:08Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

franzoni · 2019-06-13T07:31:41Z

thanks a lot @slava77 @Sam-Harper @christopheralanwest
for the clarity of documentation along this PR.

For completeness and future reference,
can we add equally clear statements about the impact (or better: lack thereof) on other scenarios such as run3, phase2, PbPb?
Tkx.

fabiocos · 2019-06-13T07:47:32Z

+1

Sam-Harper · 2019-06-13T08:01:45Z

so I hadnt noticed the non-2017 changes. There is nothing in this PR than can effect anything other than 2017! And there is nothing in the Alca changes that should effect anything but 2017. Looking into this now!

So Giovanni, the statement is there should be zero effect on run3, phase2, pbpb (unless they use a 2017 modifier)

Sam-Harper · 2019-06-13T08:30:38Z

Ah okay hadnt had my coffee. So its just data that is effected on other years as they share the same global tag which is just the 2017 corrections. Will see what we can do about that with some iovs to fix that.

fabiocos · 2019-06-13T08:52:27Z

@Sam-Harper @christopheralanwest in order to finalize the 10_6_1 preparation, we will need a backport of this #26892 and of #27089 to the 10_6_X queue. In the meanwhile this code will run in the IBs

Sam-Harper · 2019-06-13T08:54:38Z

okay expect it early in the afternoon

Sam-Harper · 2019-06-14T10:29:42Z

so just to follow up on the scale shift at in the photons at low r9.

I did the same pho 35 no pileup test. I confirm Slavas results (unsurprisingly). I also rule out an issue with the workflow in which I apply the regression on top of AOD (gave identical results as it should).

I also did it with and without the PFCluster/SuperCluster corrections (and confirmed that things where different each time so I'm fully confident that the old corrections where being applied when I thought they were).

In both cases the photon energy was almost identical (1000 dipho events) as there is a small weak dependence on the PF cluster corrections. Still negligible. There is room for error in the test but I'm confident at 2 sigma that its correct.

So my conclusion: the scale shift we see in the tests is real. However its not present in 1050. It is not due to the new PF cluster or supercluster corrections in 1061

There are two possibilities, neither change the action which is : proceed as is.

its an artifact due to rho being zero causing the regression to give a scale shift (plausable) and will be fine in the pileup samples
something has fundamentally changed in the release causing this. In this case, we would need to retrain on 1061 samples. The timescale for this is too long and we'll just reapply at analysis/ reminiAOD level

slava77 · 2019-06-14T16:54:47Z

On 6/14/19 3:29 AM, Sam Harper wrote: There are two possibilities, neither change the action which is : proceed as is. 1. its an artifact due to rho being zero causing the regression to give a scale shift (plausable) and will be fine in the pileup samples 2. something has fundamentally changed in the release causing this. In this case, we would need to retrain on 1061 samples. The timescale for this is too long and we'll just reapply at analysis/ reminiAOD level

It sounds like it would be good to rerun this with pileup.

cmsbuild added this to the CMSSW_11_0_X milestone May 22, 2019

cmsbuild added code-checks-pending comparison-pending orp-pending pending-signatures reconstruction-pending tests-pending labels May 22, 2019

cmsbuild added code-checks-approved and removed code-checks-pending labels May 22, 2019

cmsbuild added tests-started and removed tests-pending labels May 22, 2019

cmsbuild added comparison-notrun tests-rejected and removed comparison-pending tests-started labels May 22, 2019

Sam-Harper force-pushed the EGRegModifierUpdates_1060 branch from 6fc793e to 804575c Compare May 22, 2019 17:18

cmsbuild removed code-checks-approved comparison-notrun labels May 22, 2019

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Jun 12, 2019

cmsbuild added fully-signed operations-approved and removed operations-pending pending-signatures labels Jun 13, 2019

cmsbuild added orp-approved and removed orp-pending labels Jun 13, 2019

cmsbuild merged commit dc71ac5 into cms-sw:master Jun 13, 2019

Sam-Harper mentioned this pull request Jun 13, 2019

Adapting EGRegression Modifiers to 106X and adding V3 modifier: 106X #27203

Merged

christopheralanwest mentioned this pull request Jun 13, 2019

Update global tags to include new EGamma regression [10_6_X] #27205

Merged

cmsbuild mentioned this pull request Jun 13, 2019

Avoid shared_ptr in RecoParticleFlow/PFProducer and remove PFSCEnergyCalibration #27154

Merged

christopheralanwest mentioned this pull request Jul 4, 2019

Add 11_0_X global tags for 2021 and later MC scenarios #27441

Merged

Sam-Harper mentioned this pull request Jul 30, 2019

Backporting EG UL regression code to 1030 #27642

Closed

christopheralanwest mentioned this pull request Aug 3, 2019

enabling UL regression for 2018 (and likely Run3) #27644

Merged

slava77 mentioned this pull request Oct 15, 2019

Backport: Light by light modifier PR 103X #27898

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adapting EGRegression Modifiers to 106X and adding V3 modifier #26892

Adapting EGRegression Modifiers to 106X and adding V3 modifier #26892

Sam-Harper commented May 22, 2019

cmsbuild commented May 22, 2019

cmsbuild commented May 22, 2019

cmsbuild commented May 22, 2019

slava77 commented May 22, 2019

cmsbuild commented May 22, 2019 •

edited

cmsbuild commented May 22, 2019

cmsbuild commented May 22, 2019

slava77 commented May 22, 2019

Sam-Harper commented May 22, 2019

slava77 commented May 22, 2019

slava77 commented May 22, 2019

slava77 commented May 22, 2019

slava77 commented Jun 12, 2019

slava77 commented Jun 12, 2019

Sam-Harper commented Jun 12, 2019

slava77 commented Jun 12, 2019 •

edited

fabiocos commented Jun 13, 2019

cmsbuild commented Jun 13, 2019

franzoni commented Jun 13, 2019

fabiocos commented Jun 13, 2019

Sam-Harper commented Jun 13, 2019

Sam-Harper commented Jun 13, 2019

fabiocos commented Jun 13, 2019

Sam-Harper commented Jun 13, 2019

Sam-Harper commented Jun 14, 2019

slava77 commented Jun 14, 2019 via email

Adapting EGRegression Modifiers to 106X and adding V3 modifier #26892

Adapting EGRegression Modifiers to 106X and adding V3 modifier #26892

Conversation

Sam-Harper commented May 22, 2019

PR description:

cmsbuild commented May 22, 2019

cmsbuild commented May 22, 2019

cmsbuild commented May 22, 2019

slava77 commented May 22, 2019

cmsbuild commented May 22, 2019 • edited

cmsbuild commented May 22, 2019

cmsbuild commented May 22, 2019

slava77 commented May 22, 2019

Sam-Harper commented May 22, 2019

slava77 commented May 22, 2019

slava77 commented May 22, 2019

slava77 commented May 22, 2019

slava77 commented Jun 12, 2019

slava77 commented Jun 12, 2019

Sam-Harper commented Jun 12, 2019

slava77 commented Jun 12, 2019 • edited

fabiocos commented Jun 13, 2019

cmsbuild commented Jun 13, 2019

franzoni commented Jun 13, 2019

fabiocos commented Jun 13, 2019

Sam-Harper commented Jun 13, 2019

Sam-Harper commented Jun 13, 2019

fabiocos commented Jun 13, 2019

Sam-Harper commented Jun 13, 2019

Sam-Harper commented Jun 14, 2019

slava77 commented Jun 14, 2019 via email

cmsbuild commented May 22, 2019 •

edited

slava77 commented Jun 12, 2019 •

edited