HGCal Global Reco (using "Sim Particle Flow" in endcaps) #15598

lgray · 2016-08-24T22:31:38Z

This PR implements "sim particle flow" that uses the GEANT information to associate reconstructed tracks to reconstructed clusters in the HGCal endcap.

Likewise, HGCal cluster (which are SimClusters introduced in pre10) are introduced to the reconstruction sequence and allowed to seed electrons, photons, etc.
For example below, ak4 jets over the whole fiducial volume:

Another example, ecalDrivenGsfElectrons (there's a bit of a fake problem right now in endcap, to fix):

I tested this using a ttbar workflow for the "LocalReco" scenarios, where I modified the last step to include the full global reconstruction instead.

@sethzenz @kpedro88

lgray · 2016-08-24T22:31:52Z

@cmsbuild please test

cmsbuild · 2016-08-24T22:38:34Z

The tests are being triggered in jenkins.

cmsbuild · 2016-08-24T22:38:35Z

A new Pull Request was created by @lgray (Lindsey Gray) for CMSSW_8_1_X.

It involves the following packages:

RecoEcal/Configuration
RecoEcal/EgammaClusterAlgos
RecoEcal/EgammaClusterProducers
RecoEgamma/Configuration
RecoEgamma/EgammaElectronAlgos
RecoEgamma/EgammaElectronProducers
RecoEgamma/EgammaIsolationAlgos
RecoEgamma/EgammaPhotonProducers
RecoEgamma/EgammaTools
RecoParticleFlow/Configuration
RecoParticleFlow/PFClusterProducer
RecoParticleFlow/PFClusterTools
RecoParticleFlow/PFProducer
RecoParticleFlow/PFSimProducer

@cmsbuild, @cvuosalo, @slava77, @davidlange6 can you please review it and eventually sign? Thanks.
@mmarionncern, @Sam-Harper, @rafaellopesdesa, @argiro, @cbernet, @bachtis this is something you requested to watch as well.
@slava77, @smuzaffar you are the release manager for this.

cms-bot commands are list here #13028

slava77 · 2016-08-24T22:41:30Z

Looks simple from the perspective of signature collection.
It wasn't clear during the meeting that all affected packages are in the reco category.

lgray · 2016-08-24T22:47:00Z

Yes, that was lucky.

cmsbuild · 2016-08-24T22:50:25Z

Pull request #15598 was updated. @cvuosalo, @fabozzi, @cmsbuild, @srimanob, @slava77, @hengne, @davidlange6 can you please check and sign again.

lgray · 2016-08-24T22:50:38Z

@cmsbuild please test

cmsbuild · 2016-08-24T22:50:44Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/14720/console

cmsbuild · 2016-08-25T01:10:23Z

-1

Tested at: db4024c

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-15598/14720/summary.html

I found follow errors while testing this PR

Failed tests: RelVals AddOn

RelVals:

When I ran the RelVals I found an error in the following worklfows:
5.1 step1

runTheMatrix-results/5.1_TTbar+TTbarFS+HARVESTFS/step1_TTbar+TTbarFS+HARVESTFS.log

135.4 step1

runTheMatrix-results/135.4_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS/step1_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS.log

101.0 step1

runTheMatrix-results/101.0_SingleElectronE120EHCAL+SingleElectronE120EHCAL/step1_SingleElectronE120EHCAL+SingleElectronE120EHCAL.log

1000.0 step2

runTheMatrix-results/1000.0_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT/step2_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT.log

1001.0 step2

runTheMatrix-results/1001.0_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4+ALCAHARVD5/step2_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4+ALCAHARVD5.log

1003.0 step2

runTheMatrix-results/1003.0_RunMinBias2012A+RunMinBias2012A+RECODDQM+HARVESTDDQM/step2_RunMinBias2012A+RunMinBias2012A+RECODDQM+HARVESTDDQM.log

1306.0 step3

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step3_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log

10021.0 step3

runTheMatrix-results/10021.0_TenMuE_0_200+TenMuE_0_200_pythia8_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017/step3_TenMuE_0_200+TenMuE_0_200_pythia8_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017.log

1330.0 step3

runTheMatrix-results/1330.0_ZMM_13+ZMM_13+DIGIUP15+RECOUP15+HARVESTUP15/step3_ZMM_13+ZMM_13+DIGIUP15+RECOUP15+HARVESTUP15.log

136.731 step3

runTheMatrix-results/136.731_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT+HARVESTDR2/step3_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT+HARVESTDR2.log

9.0 step3

runTheMatrix-results/9.0_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST/step3_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST.log

25.0 step3

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT/step3_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT.log

10424.0 step3

runTheMatrix-results/10424.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2023D1_GenSimFull+DigiFull_2023D1+RecoFullGlobal_2023D1+HARVESTFullGlobal_2023D1/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2023D1_GenSimFull+DigiFull_2023D1+RecoFullGlobal_2023D1+HARVESTFullGlobal_2023D1.log

10024.0 step3

runTheMatrix-results/10024.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017.log

50202.0 step3

runTheMatrix-results/50202.0_TTbar_13+TTbar_13+DIGIUP15_PU50+RECOUP15_PU50+HARVESTUP15_PU50/step3_TTbar_13+TTbar_13+DIGIUP15_PU50+RECOUP15_PU50+HARVESTUP15_PU50.log

11224.0 step3

runTheMatrix-results/11224.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2023D3_GenSimFull+DigiFull_2023D3+RecoFullGlobal_2023D3+HARVESTFullGlobal_2023D3/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2023D3_GenSimFull+DigiFull_2023D3+RecoFullGlobal_2023D3+HARVESTFullGlobal_2023D3.log

25202.0 step3

runTheMatrix-results/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25/step3_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25.log

AddOn:

I found errors in the following addon tests:

cmsDriver.py TTbar_8TeV_TuneCUETP8M1_cfi --conditions auto:run1_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,HLT:@Fake,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot Realistic8TeVCollision : FAILED - time: date Thu Aug 25 02:55:21 2016-date Thu Aug 25 02:55:09 2016 s - exit: 256
cmsDriver.py RelVal -s HLT:PRef,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_PRef --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_PRef_DATA.root --fileout file:RelVal_Raw_PRef_DATA_HLT_RECO.root : FAILED - time: date Thu Aug 25 03:09:26 2016-date Thu Aug 25 02:55:17 2016 s - exit: 256
cmsDriver.py RelVal -s HLT:HIon,RAW2DIGI,L1Reco,RECO --data --scenario=HeavyIons -n 10 --conditions auto:run2_data_HIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016,Run2_HI --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_HIon_DATA.root --fileout file:RelVal_Raw_HIon_DATA_HLT_RECO.root : FAILED - time: date Thu Aug 25 02:59:18 2016-date Thu Aug 25 02:55:19 2016 s - exit: 256
cmsDriver.py RelVal -s HLT:GRun,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_GRun --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_GRun_MC.root --fileout file:RelVal_Raw_GRun_MC_HLT_RECO.root : FAILED - time: date Thu Aug 25 03:07:03 2016-date Thu Aug 25 02:55:22 2016 s - exit: 256
cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,HLT:@relval25ns,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_25ns --magField 38T_PostLS1 : FAILED - time: date Thu Aug 25 02:55:31 2016-date Thu Aug 25 02:55:23 2016 s - exit: 256
cmsDriver.py RelVal -s HLT:PRef,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_PRef --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_PRef_MC.root --fileout file:RelVal_Raw_PRef_MC_HLT_RECO.root : FAILED - time: date Thu Aug 25 03:06:29 2016-date Thu Aug 25 02:55:28 2016 s - exit: 256
cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,HLT:@relval2016,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_2016 --magField 38T_PostLS1 : FAILED - time: date Thu Aug 25 02:55:37 2016-date Thu Aug 25 02:55:29 2016 s - exit: 256
cmsDriver.py RelVal -s HLT:PIon,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_PIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_PIon_DATA.root --fileout file:RelVal_Raw_PIon_DATA_HLT_RECO.root : FAILED - time: date Thu Aug 25 02:58:28 2016-date Thu Aug 25 02:55:34 2016 s - exit: 256
cmsDriver.py RelVal -s HLT:Fake,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run1_mc_Fake --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --processName=HLTRECO --filein file:RelVal_Raw_Fake_MC.root --fileout file:RelVal_Raw_Fake_MC_HLT_RECO.root : FAILED - time: date Thu Aug 25 03:04:21 2016-date Thu Aug 25 02:55:35 2016 s - exit: 256
cmsDriver.py RelVal -s HLT:HIon,RAW2DIGI,L1Reco,RECO --mc --scenario=HeavyIons -n 10 --conditions auto:run2_mc_HIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016,Run2_HI --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_HIon_MC.root --fileout file:RelVal_Raw_HIon_MC_HLT_RECO.root : FAILED - time: date Thu Aug 25 03:06:14 2016-date Thu Aug 25 02:55:40 2016 s - exit: 256
cmsDriver.py RelVal -s HLT:Fake1,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_Fake1 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_25ns --processName=HLTRECO --filein file:RelVal_Raw_Fake1_MC.root --fileout file:RelVal_Raw_Fake1_MC_HLT_RECO.root : FAILED - time: date Thu Aug 25 03:05:42 2016-date Thu Aug 25 02:55:45 2016 s - exit: 256
cmsDriver.py RelVal -s HLT:Fake,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run1_data_Fake --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --processName=HLTRECO --filein file:RelVal_Raw_Fake_DATA.root --fileout file:RelVal_Raw_Fake_DATA_HLT_RECO.root : FAILED - time: date Thu Aug 25 02:56:44 2016-date Thu Aug 25 02:55:49 2016 s - exit: 256
cmsDriver.py RelVal -s HLT:GRun,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_GRun --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_25ns --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_GRun_DATA.root --fileout file:RelVal_Raw_GRun_DATA_HLT_RECO.root : FAILED - time: date Thu Aug 25 03:02:09 2016-date Thu Aug 25 02:55:54 2016 s - exit: 256
cmsDriver.py RelVal -s HLT:Fake1,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_Fake1 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_25ns --processName=HLTRECO --filein file:RelVal_Raw_Fake1_DATA.root --fileout file:RelVal_Raw_Fake1_DATA_HLT_RECO.root : FAILED - time: date Thu Aug 25 02:56:50 2016-date Thu Aug 25 02:55:58 2016 s - exit: 256
cmsDriver.py RelVal -s HLT:PIon,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_PIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_PIon_MC.root --fileout file:RelVal_Raw_PIon_MC_HLT_RECO.root : FAILED - time: date Thu Aug 25 03:07:30 2016-date Thu Aug 25 02:56:01 2016 s - exit: 256

lgray · 2016-08-25T01:17:22Z

huh... ok... I'll look into it...

On Wed, Aug 24, 2016 at 8:10 PM, cmsbuild notifications@github.com wrote:

-1

Tested at: db4024c
db4024c

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-
request-integration/PR-15598/14720/summary.html

I found follow errors while testing this PR

Failed tests: RelVals AddOn

RelVals:

When I ran the RelVals I found an error in the following worklfows:
5.1 step1

runTheMatrix-results/5.1_TTbar+TTbarFS+HARVESTFS/step1_TTbar+TTbarFS+HARVESTFS.log

135.4 step1

runTheMatrix-results/135.4_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS/step1_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS.log

101.0 step1

runTheMatrix-results/101.0_SingleElectronE120EHCAL+SingleElectronE120EHCAL/step1_SingleElectronE120EHCAL+SingleElectronE120EHCAL.log

1000.0 step2

runTheMatrix-results/1000.0_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT/step2_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT.log

1001.0 step2

runTheMatrix-results/1001.0_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4+ALCAHARVD5/step2_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4+ALCAHARVD5.log

1003.0 step2

runTheMatrix-results/1003.0_RunMinBias2012A+RunMinBias2012A+RECODDQM+HARVESTDDQM/step2_RunMinBias2012A+RunMinBias2012A+RECODDQM+HARVESTDDQM.log

1306.0 step3

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step3_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log

10021.0 step3

runTheMatrix-results/10021.0_TenMuE_0_200+TenMuE_0_200_pythia8_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017/step3_TenMuE_0_200+TenMuE_0_200_pythia8_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017.log

1330.0 step3

runTheMatrix-results/1330.0_ZMM_13+ZMM_13+DIGIUP15+RECOUP15+HARVESTUP15/step3_ZMM_13+ZMM_13+DIGIUP15+RECOUP15+HARVESTUP15.log

136.731 step3

runTheMatrix-results/136.731_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT+HARVESTDR2/step3_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT+HARVESTDR2.log

9.0 step3

runTheMatrix-results/9.0_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST/step3_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST.log

25.0 step3

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT/step3_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT.log

10424.0 step3

runTheMatrix-results/10424.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2023D1_GenSimFull+DigiFull_2023D1+RecoFullGlobal_2023D1+HARVESTFullGlobal_2023D1/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2023D1_GenSimFull+DigiFull_2023D1+RecoFullGlobal_2023D1+HARVESTFullGlobal_2023D1.log

10024.0 step3

runTheMatrix-results/10024.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017.log

50202.0 step3

runTheMatrix-results/50202.0_TTbar_13+TTbar_13+DIGIUP15_PU50+RECOUP15_PU50+HARVESTUP15_PU50/step3_TTbar_13+TTbar_13+DIGIUP15_PU50+RECOUP15_PU50+HARVESTUP15_PU50.log

11224.0 step3

runTheMatrix-results/11224.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2023D3_GenSimFull+DigiFull_2023D3+RecoFullGlobal_2023D3+HARVESTFullGlobal_2023D3/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2023D3_GenSimFull+DigiFull_2023D3+RecoFullGlobal_2023D3+HARVESTFullGlobal_2023D3.log

25202.0 step3

runTheMatrix-results/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25/step3_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25.log

AddOn:

I found errors in the following addon tests:

cmsDriver.py TTbar_8TeV_TuneCUETP8M1_cfi --conditions auto:run1_mc --fast
-n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s
GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,HLT:@Fake
https://github.com/fake,VALIDATION --customise=HLTrigger/
Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO
--beamspot Realistic8TeVCollision : FAILED - time: date Thu Aug 25 02:55:21
2016-date Thu Aug 25 02:55:09 2016 s - exit: 256
cmsDriver.py RelVal -s HLT:PRef,RAW2DIGI,L1Reco,RECO --data --scenario=pp
-n 10 --conditions auto:run2_data_PRef --relval 9000,50 --datatier
"RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/
Configuration/CustomConfigs.L1THLT --era Run2_2016 --magField 38T_PostLS1
--processName=HLTRECO --filein file:RelVal_Raw_PRef_DATA.root --fileout
file:RelVal_Raw_PRef_DATA_HLT_RECO.root : FAILED - time: date Thu Aug 25
03:09:26 2016-date Thu Aug 25 02:55:17 2016 s - exit: 256
cmsDriver.py RelVal -s HLT:HIon,RAW2DIGI,L1Reco,RECO --data
--scenario=HeavyIons -n 10 --conditions auto:run2_data_HIon --relval
9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT
--customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era
Run2_2016,Run2_HI --magField 38T_PostLS1 --processName=HLTRECO --filein
file:RelVal_Raw_HIon_DATA.root --fileout file:RelVal_Raw_HIon_DATA_HLT_RECO.root
: FAILED - time: date Thu Aug 25 02:59:18 2016-date Thu Aug 25 02:55:19
2016 s - exit: 256
cmsDriver.py RelVal -s HLT:GRun,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n
10 --conditions auto:run2_mc_GRun --relval 9000,50 --datatier
"RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/
Configuration/CustomConfigs.L1THLT --era Run2_2016 --magField 38T_PostLS1
--processName=HLTRECO --filein file:RelVal_Raw_GRun_MC.root --fileout
file:RelVal_Raw_GRun_MC_HLT_RECO.root : FAILED - time: date Thu Aug 25
03:07:03 2016-date Thu Aug 25 02:55:22 2016 s - exit: 256
cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc --fast
-n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s
GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,
EI,HLT:@relval25ns,VALIDATION --customise=HLTrigger/
Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO
--beamspot NominalCollision2015 --era Run2_25ns --magField 38T_PostLS1 :
FAILED - time: date Thu Aug 25 02:55:31 2016-date Thu Aug 25 02:55:23 2016
s - exit: 256
cmsDriver.py RelVal -s HLT:PRef,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n
10 --conditions auto:run2_mc_PRef --relval 9000,50 --datatier
"RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/
Configuration/CustomConfigs.L1THLT --era Run2_2016 --magField 38T_PostLS1
--processName=HLTRECO --filein file:RelVal_Raw_PRef_MC.root --fileout
file:RelVal_Raw_PRef_MC_HLT_RECO.root : FAILED - time: date Thu Aug 25
03:06:29 2016-date Thu Aug 25 02:55:28 2016 s - exit: 256
cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc --fast
-n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s
GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,
EI,HLT:@relval2016,VALIDATION --customise=HLTrigger/
Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO
--beamspot NominalCollision2015 --era Run2_2016 --magField 38T_PostLS1 :
FAILED - time: date Thu Aug 25 02:55:37 2016-date Thu Aug 25 02:55:29 2016
s - exit: 256
cmsDriver.py RelVal -s HLT:PIon,RAW2DIGI,L1Reco,RECO --data --scenario=pp
-n 10 --conditions auto:run2_data_PIon --relval 9000,50 --datatier
"RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/
Configuration/CustomConfigs.L1THLT --era Run2_2016 --magField 38T_PostLS1
--processName=HLTRECO --filein file:RelVal_Raw_PIon_DATA.root --fileout
file:RelVal_Raw_PIon_DATA_HLT_RECO.root : FAILED - time: date Thu Aug 25
02:58:28 2016-date Thu Aug 25 02:55:34 2016 s - exit: 256
cmsDriver.py RelVal -s HLT:Fake,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n
10 --conditions auto:run1_mc_Fake --relval 9000,50 --datatier
"RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/
Configuration/CustomConfigs.L1THLT --processName=HLTRECO --filein
file:RelVal_Raw_Fake_MC.root --fileout file:RelVal_Raw_Fake_MC_HLT_RECO.root
: FAILED - time: date Thu Aug 25 03:04:21 2016-date Thu Aug 25 02:55:35
2016 s - exit: 256
cmsDriver.py RelVal -s HLT:HIon,RAW2DIGI,L1Reco,RECO --mc
--scenario=HeavyIons -n 10 --conditions auto:run2_mc_HIon --relval 9000,50
--datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/
Configuration/CustomConfigs.L1THLT --era Run2_2016,Run2_HI --magField
38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_HIon_MC.root
--fileout file:RelVal_Raw_HIon_MC_HLT_RECO.root : FAILED - time: date Thu
Aug 25 03:06:14 2016-date Thu Aug 25 02:55:40 2016 s - exit: 256
cmsDriver.py RelVal -s HLT:Fake1,RAW2DIGI,L1Reco,RECO --mc --scenario=pp
-n 10 --conditions auto:run2_mc_Fake1 --relval 9000,50 --datatier
"RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/
Configuration/CustomConfigs.L1THLT --era Run2_25ns --processName=HLTRECO
--filein file:RelVal_Raw_Fake1_MC.root --fileout
file:RelVal_Raw_Fake1_MC_HLT_RECO.root : FAILED - time: date Thu Aug 25
03:05:42 2016-date Thu Aug 25 02:55:45 2016 s - exit: 256
cmsDriver.py RelVal -s HLT:Fake,RAW2DIGI,L1Reco,RECO --data --scenario=pp
-n 10 --conditions auto:run1_data_Fake --relval 9000,50 --datatier
"RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/
Configuration/CustomConfigs.L1THLT --processName=HLTRECO --filein
file:RelVal_Raw_Fake_DATA.root --fileout file:RelVal_Raw_Fake_DATA_HLT_RECO.root
: FAILED - time: date Thu Aug 25 02:56:44 2016-date Thu Aug 25 02:55:49
2016 s - exit: 256
cmsDriver.py RelVal -s HLT:GRun,RAW2DIGI,L1Reco,RECO --data --scenario=pp
-n 10 --conditions auto:run2_data_GRun --relval 9000,50 --datatier
"RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/
Configuration/CustomConfigs.L1THLT --era Run2_25ns --magField 38T_PostLS1
--processName=HLTRECO --filein file:RelVal_Raw_GRun_DATA.root --fileout
file:RelVal_Raw_GRun_DATA_HLT_RECO.root : FAILED - time: date Thu Aug 25
03:02:09 2016-date Thu Aug 25 02:55:54 2016 s - exit: 256
cmsDriver.py RelVal -s HLT:Fake1,RAW2DIGI,L1Reco,RECO --data --scenario=pp
-n 10 --conditions auto:run2_data_Fake1 --relval 9000,50 --datatier
"RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/
Configuration/CustomConfigs.L1THLT --era Run2_25ns --processName=HLTRECO
--filein file:RelVal_Raw_Fake1_DATA.root --fileout
file:RelVal_Raw_Fake1_DATA_HLT_RECO.root : FAILED - time: date Thu Aug 25
02:56:50 2016-date Thu Aug 25 02:55:58 2016 s - exit: 256
cmsDriver.py RelVal -s HLT:PIon,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n
10 --conditions auto:run2_mc_PIon --relval 9000,50 --datatier
"RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/
Configuration/CustomConfigs.L1THLT --era Run2_2016 --magField 38T_PostLS1
--processName=HLTRECO --filein file:RelVal_Raw_PIon_MC.root --fileout
file:RelVal_Raw_PIon_MC_HLT_RECO.root : FAILED - time: date Thu Aug 25
03:07:30 2016-date Thu Aug 25 02:56:01 2016 s - exit: 256

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#15598 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABBMOX0rjw0np_woHCUmLs4u7XS6NSciks5qjOuCgaJpZM4JshQH
.

cmsbuild · 2016-08-29T16:33:57Z

Pull request #15598 was updated. @cvuosalo, @fabozzi, @cmsbuild, @srimanob, @slava77, @hengne, @davidlange6 can you please check and sign again.

lgray · 2016-08-29T16:34:22Z

@cmsbuild please test

cmsbuild · 2016-08-29T16:34:41Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/14809/console

cmsbuild · 2016-08-29T17:54:17Z

+1
Tested at: 4c64ac9
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-15598/14809/summary.html

cmsbuild · 2016-08-29T18:27:34Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-15598/14809/summary.html

slava77 · 2016-08-30T15:50:08Z

Looking at PU200 with MT4 (the nominal workflow configuration) , equivalent of 11424, RSS on 30 events goes up to 16GB (it's 10GB in the middle of the job). Memory profile on my 200 event job with MT16 suggests there is no real memory leak, but that's just a feeling based on observed numbers.
The workflow runs at ~20mins/event, about a factor of 2.5-3 increase from this PR.

@fabozzi something to follow up when defining the relval workflows so that they actually can complete.

lgray · 2016-08-30T18:52:06Z

@slava77 can you pin the 2.5-3x increase to any particular module? Likewise for the memory increase?

lgray · 2016-08-30T18:55:40Z

It's likely coming from the (currently large) amount of electron fakes and corresponding GSF tracks being seeded. If necessary, I can work to make some quick requirements on the electron seed clusters to reject obvious fakes. Would prefer in another PR.

slava77 · 2016-08-30T21:41:08Z

The time increase by x2.5-3 is including I/O and other non-threadable stalls (TimeEvent numbers), if I consider pure module time totals (TimeModule numbers) I get a factor of 30 increase.
That's somewhat "natural" given that we have switched from just local reco to full reco with validation included.
The time for common modules didn't change much.
Dropping the output module (writes a lot of tracking particles etc stuff, at about 100MB compressed per event), the remaining modules time changes from 6.6 s/evt to 1007 s/evt.

Of these ~1000 seconds per event

~ 300 s is validation (32 [mix] + 156 [recoMuonV* ] + 90 [tpTo* muon MTV]). Here the muon validation modules ran out of control
700 s is reco-related: (265 [allConversions] + 108 [ecalDrivenElectronSeeds] + 54 [pfTrackElec] + 170 [iterative tracking] + 100 [everything else])

slava77 · 2016-08-30T23:03:08Z

Here are some plots, comparing 2016 TTbar PU35 with 2023 D3 TTbar PU35 (11424.0)
Some are partly for amusement [a more meaningful comparison should be done vs D1]

calo towers are not made for HGCAL

hbhereco time is some random numbers

energy looks kind of funny too (many zeroes)

ECAL barrel looks fairly similar (self-validation)

EGamma pf candidates are fairly similar

... considering that they are in the barrel only

particleFlow charged hadrons are somewhat similar (some differences are from increased acceptance in eta)

pf electrons are about there

pf muons rate is down quite a bit

mostly from the endcaps ... I'm guessing it's the non-isolated muons that don't pass selections as well anymore

there are many more pf photons now .. probably from the fakes or maybe from OOT

even more neutral hadrons (are the scales off in HGCAL?)

electronSeeds goes to the story of more fakes

at the jet level, things look ~OK

@lgray please take a look and see if anything stands out as missing or broken that can be fixed in this PR.

lgray · 2016-08-30T23:39:21Z

The photons and neutral hadrons going nuts is from the "perfect" reconstruction (especially the huge increase at low pT. I may increase the energy cut to keep neutral candidates in the simulated PF producer, realistically we cannot reconstruct them all!

Otherwise this looks more or less like I expect. Next step: add in BH and control electron fakes.

slava77 · 2016-08-30T23:50:12Z

+1

for #15598 4c64ac9

local reco in D3 2023 scenario is replaced with global reco, in line with the description.
jenkins tests pass and comparisons with baseline show no differences (due to change in the workflow directory, comparisons in 11224.0 didn't run
details from local tests with D3 scenario can be found earlier in this thread

lgray · 2016-08-30T23:54:14Z

@fabozzi, @srimanob, @hengne please sign off

kpedro88 · 2016-08-31T13:28:08Z

@slava77 just a few comments on your plots:

HGCal has never been added to the CaloTowers algorithm. I think it is currently not intended to be added there.
HBHE reco in 2023D3 is still using the generic "upgrade reco" from 62XSLHC. Once the HCAL Phase1 reco development is finished, we can move to that (and delete the generic HCAL upgrade classes from CMSSW entirely).
The larger-than-average number of zero-energy HBHE rechits needs some investigation.

slava77 · 2016-08-31T14:02:10Z

On 8/31/16 6:28 AM, Kevin Pedro wrote:

@slava77 https://github.com/slava77 just a few comments on your plots:

HGCal has never been added to the CaloTowers algorithm. I think it
is currently not intended to be added there.

HBHE reco in 2023D3 is still using the generic "upgrade reco" from
62XSLHC. Once the HCAL Phase1 reco development is finished, we can
move to that (and delete the generic HCAL upgrade classes from CMSSW
entirely).

The larger-than-average number of zero-energy HBHE rechits needs
some investigation.

if phase-2 HB is multi-depth, I'd assume that more zeroes are from there.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#15598 (comment), or
mute the thread
https://github.com/notifications/unsubscribe-auth/AEdcbjCxY4hv8wFBaP1w-9ywT9aP3cLZks5qlYFpgaJpZM4JshQH.

lgray added 3 commits August 23, 2016 14:47

Allow CMS Phase2 with HGCal to run without crashing

bf48c79

working on integrating sim PF and barrel PF together

90a044a

minor reorganization to simplify propagation of merged PF collections

3f656c6

cmsbuild added this to the Next CMSSW_8_1_X milestone Aug 24, 2016

cmsbuild added reconstruction-pending pending-signatures tests-pending orp-pending comparison-pending labels Aug 24, 2016

cmsbuild added tests-started and removed tests-pending labels Aug 24, 2016

update HGCal workflows to global reco

db4024c

cmsbuild added tests-pending pdmv-pending and removed tests-started labels Aug 24, 2016

cmsbuild added tests-started and removed tests-pending labels Aug 24, 2016

cmsbuild added tests-rejected and removed tests-started labels Aug 25, 2016

get matrix tests running again

5cb4310

code review 2

4c64ac9

cmsbuild added tests-pending comparison-pending and removed comparison-available tests-approved labels Aug 29, 2016

cmsbuild added tests-started and removed tests-pending labels Aug 29, 2016

cmsbuild added tests-approved and removed tests-started labels Aug 29, 2016

cmsbuild added comparison-available and removed comparison-pending labels Aug 29, 2016

slava77 mentioned this pull request Aug 30, 2016

updates for HIPM in 2016 and full reco in 2023 cms-sw/cms-bot#737

Merged

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Aug 30, 2016

davidlange6 merged commit d676382 into cms-sw:CMSSW_8_1_X Aug 31, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HGCal Global Reco (using "Sim Particle Flow" in endcaps) #15598

HGCal Global Reco (using "Sim Particle Flow" in endcaps) #15598

lgray commented Aug 24, 2016 •

edited

Loading

lgray commented Aug 24, 2016

cmsbuild commented Aug 24, 2016

cmsbuild commented Aug 24, 2016

slava77 commented Aug 24, 2016

lgray commented Aug 24, 2016

cmsbuild commented Aug 24, 2016

lgray commented Aug 24, 2016

cmsbuild commented Aug 24, 2016 •

edited

Loading

cmsbuild commented Aug 25, 2016

lgray commented Aug 25, 2016

cmsbuild commented Aug 29, 2016

lgray commented Aug 29, 2016

cmsbuild commented Aug 29, 2016 •

edited

Loading

cmsbuild commented Aug 29, 2016

cmsbuild commented Aug 29, 2016

slava77 commented Aug 30, 2016

lgray commented Aug 30, 2016 •

edited

Loading

lgray commented Aug 30, 2016

slava77 commented Aug 30, 2016

slava77 commented Aug 30, 2016

lgray commented Aug 30, 2016

slava77 commented Aug 30, 2016

lgray commented Aug 30, 2016

kpedro88 commented Aug 31, 2016

slava77 commented Aug 31, 2016

HGCal Global Reco (using "Sim Particle Flow" in endcaps) #15598

HGCal Global Reco (using "Sim Particle Flow" in endcaps) #15598

Conversation

lgray commented Aug 24, 2016 • edited Loading

lgray commented Aug 24, 2016

cmsbuild commented Aug 24, 2016

cmsbuild commented Aug 24, 2016

slava77 commented Aug 24, 2016

lgray commented Aug 24, 2016

cmsbuild commented Aug 24, 2016

lgray commented Aug 24, 2016

cmsbuild commented Aug 24, 2016 • edited Loading

cmsbuild commented Aug 25, 2016

lgray commented Aug 25, 2016

cmsbuild commented Aug 29, 2016

lgray commented Aug 29, 2016

cmsbuild commented Aug 29, 2016 • edited Loading

cmsbuild commented Aug 29, 2016

cmsbuild commented Aug 29, 2016

slava77 commented Aug 30, 2016

lgray commented Aug 30, 2016 • edited Loading

lgray commented Aug 30, 2016

slava77 commented Aug 30, 2016

slava77 commented Aug 30, 2016

lgray commented Aug 30, 2016

slava77 commented Aug 30, 2016

lgray commented Aug 30, 2016

kpedro88 commented Aug 31, 2016

slava77 commented Aug 31, 2016

lgray commented Aug 24, 2016 •

edited

Loading

cmsbuild commented Aug 24, 2016 •

edited

Loading

cmsbuild commented Aug 29, 2016 •

edited

Loading

lgray commented Aug 30, 2016 •

edited

Loading