enable PF HEM mitigation for HI 2018 and add Run2_2018_pp_on_AA_noHCALmitigation era #24777

slava77 · 2018-10-02T09:42:45Z

current default era for 2018 HI processing Run2_2018_pp_on_AA now has the PF bad HCAL mitigation enabled
to perform tests for comparisons without the mitigation, a new era Run2_2018_pp_on_AA_noHCALmitigation is introduced

@franzoni this is a follow up to your request by email to have two eras for the 2018 HI.

@mandrenguyen

…2018_pp_on_AA

cmsbuild · 2018-10-02T09:43:02Z

The code-checks are being triggered in jenkins.

slava77 · 2018-10-02T09:43:24Z

@cmsbuild please test

cmsbuild · 2018-10-02T09:45:08Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-24777/6722

cmsbuild · 2018-10-02T09:45:27Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/30845/console Started: 2018/10/02 11:45

cmsbuild · 2018-10-02T09:45:28Z

A new Pull Request was created by @slava77 (Slava Krutelyov) for master.

It involves the following packages:

Configuration/Eras
Configuration/StandardSequences

@cmsbuild, @franzoni, @fabiocos, @davidlange6 can you please review it and eventually sign? Thanks.
@makortel, @felicepantaleo, @GiacomoSguazzoni, @rovere, @VinInn, @Martin-Grunewald, @dgulhan this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

cmsbuild · 2018-10-02T09:57:23Z

-1

Tested at: 4a7faa1

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-24777/30845/summary.html

I found follow errors while testing this PR

Failed tests: UnitTests RelVals AddOn

Unit Tests:

I found errors in the following unit tests:

---> test runtestPhysicsToolsNanoAOD had ERRORS
---> test TestConfigDP had ERRORS

RelVals:

When I ran the RelVals I found an error in the following worklfows:
5.1 step1

runTheMatrix-results/5.1_TTbar+TTbarFS+HARVESTFS/step1_TTbar+TTbarFS+HARVESTFS.log

7.3 step1

runTheMatrix-results/7.3_CosmicsSPLoose_UP17+CosmicsSPLoose_UP17+DIGICOS_UP17+RECOCOS_UP17+ALCACOS_UP17+HARVESTCOS_UP17/step1_CosmicsSPLoose_UP17+CosmicsSPLoose_UP17+DIGICOS_UP17+RECOCOS_UP17+ALCACOS_UP17+HARVESTCOS_UP17.log

4.53 step2

runTheMatrix-results/4.53_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT/step2_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT.log

135.4 step1

runTheMatrix-results/135.4_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS/step1_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS.log

8.0 step1

runTheMatrix-results/8.0_BeamHalo+BeamHalo+DIGICOS+RECOCOS+ALCABH+HARVESTCOS/step1_BeamHalo+BeamHalo+DIGICOS+RECOCOS+ALCABH+HARVESTCOS.log

136.731 step2

runTheMatrix-results/136.731_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2/step2_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2.log

9.0 step1

runTheMatrix-results/9.0_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST/step1_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST.log

136.7611 step2

runTheMatrix-results/136.7611_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016_HIPM+HARVESTDR2_REMINIAOD_data2016_HIPM/step2_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016_HIPM+HARVESTDR2_REMINIAOD_data2016_HIPM.log

25.0 step1

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT/step1_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT.log

136.788 step2

runTheMatrix-results/136.788_RunSinglePh2017B+RunSinglePh2017B+HLTDR2_2017+RECODR2_2017reHLT_skimSinglePh_Prompt+HARVEST2017/step2_RunSinglePh2017B+RunSinglePh2017B+HLTDR2_2017+RECODR2_2017reHLT_skimSinglePh_Prompt+HARVEST2017.log

136.8311 step2

runTheMatrix-results/136.8311_RunJetHT2017F_reminiaod+RunJetHT2017F_reminiaod+REMINIAOD_data2017+HARVEST2017_REMINIAOD_data2017/step2_RunJetHT2017F_reminiaod+RunJetHT2017F_reminiaod+REMINIAOD_data2017+HARVEST2017_REMINIAOD_data2017.log

136.85 step2

runTheMatrix-results/136.85_RunEGamma2018A+RunEGamma2018A+HLTDR2_2018+RECODR2_2018reHLT_skimEGamma_Prompt_L1TEgDQM+HARVEST2018_L1TEgDQM/step2_RunEGamma2018A+RunEGamma2018A+HLTDR2_2018+RECODR2_2018reHLT_skimEGamma_Prompt_L1TEgDQM+HARVEST2018_L1TEgDQM.log

150.0 step1

runTheMatrix-results/150.0_HydjetQ_B12_5020GeV_2018+HydjetQ_B12_5020GeV_2018+DIGIHI2018+RECOHI2018+HARVESTHI2018/step1_HydjetQ_B12_5020GeV_2018+HydjetQ_B12_5020GeV_2018+DIGIHI2018+RECOHI2018+HARVESTHI2018.log

158.0 step1

runTheMatrix-results/158.0_HydjetQ_B12_5020GeV_2018_ppReco+HydjetQ_B12_5020GeV_2018_ppReco+DIGIHI2018PPRECO+RECOHI2018PPRECO+ALCARECOHI2018PPRECO+HARVESTHI2018PPRECO/step1_HydjetQ_B12_5020GeV_2018_ppReco+HydjetQ_B12_5020GeV_2018_ppReco+DIGIHI2018PPRECO+RECOHI2018PPRECO+ALCARECOHI2018PPRECO+HARVESTHI2018PPRECO.log

1306.0 step1

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step1_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log

1325.7 step2

runTheMatrix-results/1325.7_TTbar_13_94Xv2NanoAODINPUT+TTbar_13_94Xv2NanoAODINPUT+NANOEDMMC2017_94XMiniAODv2+HARVESTNANOAODMC2017_94XMiniAODv2/step2_TTbar_13_94Xv2NanoAODINPUT+TTbar_13_94Xv2NanoAODINPUT+NANOEDMMC2017_94XMiniAODv2+HARVESTNANOAODMC2017_94XMiniAODv2.log

1330.0 step1

runTheMatrix-results/1330.0_ZMM_13+ZMM_13+DIGIUP15+RECOUP15_L1TMuDQM+HARVESTUP15_L1TMuDQM/step1_ZMM_13+ZMM_13+DIGIUP15+RECOUP15_L1TMuDQM+HARVESTUP15_L1TMuDQM.log

25202.0 step1

runTheMatrix-results/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25/step1_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25.log

10024.0 step1

runTheMatrix-results/10024.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017/step1_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017.log

10042.0 step1

runTheMatrix-results/10042.0_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017/step1_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017.log

10224.0 step1

runTheMatrix-results/10224.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017PU_GenSimFull+DigiFullPU_2017PU+RecoFullPU_2017PU+HARVESTFullPU_2017PU/step1_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017PU_GenSimFull+DigiFullPU_2017PU+RecoFullPU_2017PU+HARVESTFullPU_2017PU.log

4.22 step2

runTheMatrix-results/4.22_RunCosmics2011A+RunCosmics2011A+RECOCOSD+ALCACOSD+SKIMCOSD+HARVESTDC/step2_RunCosmics2011A+RunCosmics2011A+RECOCOSD+ALCACOSD+SKIMCOSD+HARVESTDC.log

10824.0 step1

runTheMatrix-results/10824.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+HARVESTFull_2018+ALCAFull_2018+NanoFull_2018/step1_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+HARVESTFull_2018+ALCAFull_2018+NanoFull_2018.log

11624.0 step1

runTheMatrix-results/11624.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2019_GenSimFull+DigiFull_2019+RecoFull_2019+HARVESTFull_2019+ALCAFull_2019/step1_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2019_GenSimFull+DigiFull_2019+RecoFull_2019+HARVESTFull_2019+ALCAFull_2019.log

20034.0 step1

runTheMatrix-results/20034.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D17_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D17+RecoFullGlobal_2023D17+HARVESTFullGlobal_2023D17/step1_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D17_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D17+RecoFullGlobal_2023D17+HARVESTFullGlobal_2023D17.log

20434.0 step1

runTheMatrix-results/20434.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D19_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D19+RecoFullGlobal_2023D19+HARVESTFullGlobal_2023D19/step1_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D19_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D19+RecoFullGlobal_2023D19+HARVESTFullGlobal_2023D19.log

21234.0 step1

runTheMatrix-results/21234.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D21_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D21+RecoFullGlobal_2023D21+HARVESTFullGlobal_2023D21/step1_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D21_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D21+RecoFullGlobal_2023D21+HARVESTFullGlobal_2023D21.log

250202.181 step1

runTheMatrix-results/250202.181_TTbar_13UP18+TTbar_13UP18+PREMIXUP18_PU25+DIGIPRMXLOCALUP18_PU25+RECOPRMXUP18_PU25+HARVESTUP18_PU25/step1_TTbar_13UP18+TTbar_13UP18+PREMIXUP18_PU25+DIGIPRMXLOCALUP18_PU25+RECOPRMXUP18_PU25+HARVESTUP18_PU25.log

101.0 step1

runTheMatrix-results/101.0_SingleElectronE120EHCAL+SingleElectronE120EHCAL/step1_SingleElectronE120EHCAL+SingleElectronE120EHCAL.log

140.53 step2

runTheMatrix-results/140.53_RunHI2011+RunHI2011+RECOHID11+HARVESTDHI/step2_RunHI2011+RunHI2011+RECOHID11+HARVESTDHI.log

1000.0 step2

runTheMatrix-results/1000.0_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT/step2_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT.log

1001.0 step2

runTheMatrix-results/1001.0_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVDSIPIXELCALRUN1+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4+ALCAHARVD5/step2_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVDSIPIXELCALRUN1+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4+ALCAHARVD5.log

AddOn:

I found errors in the following addon tests:

cmsDriver.py TTbar_8TeV_TuneCUETP8M1_cfi --conditions auto:run1_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot Realistic8TeVCollision : FAILED - time: date Tue Oct 2 11:55:38 2018-date Tue Oct 2 11:55:37 2018 s - exit: 256
cmsDriver.py RelVal -s L1REPACK:Full --data --scenario=pp -n 10 --conditions auto:run2_hlt_Fake2 --relval 9000,50 --datatier "RAW" --eventcontent RAW --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_2016 --fileout file:RelVal_Raw_Fake2_DATA.root --filein /store/data/Run2016B/JetHT/RAW/v1/000/272/762/00000/C666CDE2-E013-E611-B15A-02163E011DBE.root : FAILED - time: date Tue Oct 2 11:55:53 2018-date Tue Oct 2 11:55:46 2018 s - exit: 256
cmsRun /build/cmsbld/jenkins/workspace/ib-any-integration/CMSSW_10_3_X_2018-10-01-2300/src/HLTrigger/Configuration/test/OnLine_HLT_Fake2.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Tue Oct 2 11:55:53 2018-date Tue Oct 2 11:55:46 2018 s - exit: 21504
cmsDriver.py RelVal -s HLT:Fake2,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_Fake2 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --processName=HLTRECO --filein file:RelVal_Raw_Fake2_DATA.root --fileout file:RelVal_Raw_Fake2_DATA_HLT_RECO.root : FAILED - time: date Tue Oct 2 11:55:53 2018-date Tue Oct 2 11:55:46 2018 s - exit: 256
cmsDriver.py RelVal -s L1REPACK:Full --data --scenario=pp -n 10 --conditions auto:run2_hlt_PRef --relval 9000,50 --datatier "RAW" --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_2018 --eventcontent RAW --fileout file:RelVal_Raw_PRef_DATA.root --filein /store/data/Run2017A/HLTPhysics4/RAW/v1/000/295/606/00000/36DE5E0A-3645-E711-8FA1-02163E01A43B.root : FAILED - time: date Tue Oct 2 11:55:59 2018-date Tue Oct 2 11:55:49 2018 s - exit: 256
cmsRun /build/cmsbld/jenkins/workspace/ib-any-integration/CMSSW_10_3_X_2018-10-01-2300/src/HLTrigger/Configuration/test/OnLine_HLT_PRef.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Tue Oct 2 11:55:59 2018-date Tue Oct 2 11:55:49 2018 s - exit: 21504
cmsDriver.py RelVal -s HLT:PRef,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_PRef --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_PRef_DATA.root --fileout file:RelVal_Raw_PRef_DATA_HLT_RECO.root : FAILED - time: date Tue Oct 2 11:55:59 2018-date Tue Oct 2 11:55:49 2018 s - exit: 256
cmsDriver.py RelVal -s L1REPACK:Full2015Data --data --scenario=HeavyIons -n 10 --conditions auto:run2_hlt_HIon --relval 9000,50 --datatier "RAW" --eventcontent RAW --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_2016,Run2_HI --fileout file:RelVal_Raw_HIon_DATA.root --filein /store/hidata/HIRun2015/HIHardProbes/RAW-RECO/HighPtJet-PromptReco-v1/000/263/689/00000/1802CD9A-DDB8-E511-9CF9-02163E0138CA.root : FAILED - time: date Tue Oct 2 11:55:59 2018-date Tue Oct 2 11:55:51 2018 s - exit: 256
cmsRun /build/cmsbld/jenkins/workspace/ib-any-integration/CMSSW_10_3_X_2018-10-01-2300/src/HLTrigger/Configuration/test/OnLine_HLT_HIon.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Tue Oct 2 11:55:59 2018-date Tue Oct 2 11:55:51 2018 s - exit: 21504
cmsDriver.py RelVal -s HLT:HIon,RAW2DIGI,L1Reco,RECO --data --scenario=HeavyIons -n 10 --conditions auto:run2_data_HIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016,Run2_HI --processName=HLTRECO --filein file:RelVal_Raw_HIon_DATA.root --fileout file:RelVal_Raw_HIon_DATA_HLT_RECO.root : FAILED - time: date Tue Oct 2 11:55:59 2018-date Tue Oct 2 11:55:51 2018 s - exit: 256
cmsDriver.py TTbar_Tauola_13TeV_TuneCUETP8M1_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=pp -n 10 --conditions auto:run2_mc_GRun --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_2018 --fileout file:RelVal_Raw_GRun_MC.root : FAILED - time: date Tue Oct 2 11:56:12 2018-date Tue Oct 2 11:55:55 2018 s - exit: 256
cmsRun /build/cmsbld/jenkins/workspace/ib-any-integration/CMSSW_10_3_X_2018-10-01-2300/src/HLTrigger/Configuration/test/OnLine_HLT_GRun.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Tue Oct 2 11:56:12 2018-date Tue Oct 2 11:55:55 2018 s - exit: 21504
cmsDriver.py RelVal -s HLT:GRun,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_GRun --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_GRun_MC.root --fileout file:RelVal_Raw_GRun_MC_HLT_RECO.root : FAILED - time: date Tue Oct 2 11:56:12 2018-date Tue Oct 2 11:55:55 2018 s - exit: 256
cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc_l1stage1 --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_25ns : FAILED - time: date Tue Oct 2 11:55:57 2018-date Tue Oct 2 11:55:56 2018 s - exit: 256
cmsDriver.py TTbar_Tauola_8TeV_TuneCUETP8M1_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=pp -n 10 --conditions auto:run1_mc_Fake --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --fileout file:RelVal_Raw_Fake_MC.root : FAILED - time: date Tue Oct 2 11:56:16 2018-date Tue Oct 2 11:55:57 2018 s - exit: 23040
cmsRun /build/cmsbld/jenkins/workspace/ib-any-integration/CMSSW_10_3_X_2018-10-01-2300/src/HLTrigger/Configuration/test/OnLine_HLT_Fake.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Tue Oct 2 11:56:16 2018-date Tue Oct 2 11:55:57 2018 s - exit: 21504
cmsDriver.py RelVal -s HLT:Fake,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run1_mc_Fake --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --processName=HLTRECO --filein file:RelVal_Raw_Fake_MC.root --fileout file:RelVal_Raw_Fake_MC_HLT_RECO.root : FAILED - time: date Tue Oct 2 11:56:16 2018-date Tue Oct 2 11:55:57 2018 s - exit: 23040
cmsDriver.py TTbar_Tauola_13TeV_TuneCUETP8M1_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=pp -n 10 --conditions auto:run2_mc_PRef --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_2018 --fileout file:RelVal_Raw_PRef_MC.root : FAILED - time: date Tue Oct 2 11:56:12 2018-date Tue Oct 2 11:56:02 2018 s - exit: 256
cmsRun /build/cmsbld/jenkins/workspace/ib-any-integration/CMSSW_10_3_X_2018-10-01-2300/src/HLTrigger/Configuration/test/OnLine_HLT_PRef.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Tue Oct 2 11:56:12 2018-date Tue Oct 2 11:56:02 2018 s - exit: 21504
cmsDriver.py RelVal -s HLT:PRef,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_PRef --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_PRef_MC.root --fileout file:RelVal_Raw_PRef_MC_HLT_RECO.root : FAILED - time: date Tue Oct 2 11:56:12 2018-date Tue Oct 2 11:56:02 2018 s - exit: 256
cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_2016 : FAILED - time: date Tue Oct 2 11:56:05 2018-date Tue Oct 2 11:56:04 2018 s - exit: 256
cmsDriver.py RelVal -s L1REPACK:Full --data --scenario=pp -n 10 --conditions auto:run2_hlt_2018v36 --relval 9000,50 --datatier "RAW" --eventcontent RAW --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_2018 --fileout file:RelVal_Raw_2018v36_DATA.root --filein /store/data/Run2017A/HLTPhysics4/RAW/v1/000/295/606/00000/36DE5E0A-3645-E711-8FA1-02163E01A43B.root : FAILED - time: date Tue Oct 2 11:56:21 2018-date Tue Oct 2 11:56:06 2018 s - exit: 256
cmsRun /build/cmsbld/jenkins/workspace/ib-any-integration/CMSSW_10_3_X_2018-10-01-2300/src/HLTrigger/Configuration/test/OnLine_HLT_2018v36.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Tue Oct 2 11:56:21 2018-date Tue Oct 2 11:56:06 2018 s - exit: 21504
cmsDriver.py RelVal -s HLT:2018v36,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_2018v36 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_2018v36_DATA.root --fileout file:RelVal_Raw_2018v36_DATA_HLT_RECO.root : FAILED - time: date Tue Oct 2 11:56:21 2018-date Tue Oct 2 11:56:06 2018 s - exit: 256
cmsDriver.py RelVal -s L1REPACK:Full --data --scenario=pp -n 10 --conditions auto:run2_hlt_PIon --relval 9000,50 --datatier "RAW" --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_2018 --eventcontent RAW --fileout file:RelVal_Raw_PIon_DATA.root --filein /store/data/Run2017A/HLTPhysics4/RAW/v1/000/295/606/00000/36DE5E0A-3645-E711-8FA1-02163E01A43B.root : FAILED - time: date Tue Oct 2 11:56:15 2018-date Tue Oct 2 11:56:07 2018 s - exit: 256
cmsRun /build/cmsbld/jenkins/workspace/ib-any-integration/CMSSW_10_3_X_2018-10-01-2300/src/HLTrigger/Configuration/test/OnLine_HLT_PIon.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Tue Oct 2 11:56:15 2018-date Tue Oct 2 11:56:07 2018 s - exit: 21504
cmsDriver.py RelVal -s HLT:PIon,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_PIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_PIon_DATA.root --fileout file:RelVal_Raw_PIon_DATA_HLT_RECO.root : FAILED - time: date Tue Oct 2 11:56:15 2018-date Tue Oct 2 11:56:07 2018 s - exit: 256
cmsDriver.py TTbar_Tauola_13TeV_TuneCUETP8M1_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=pp -n 10 --conditions auto:run2_mc_2018v36 --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_2018 --fileout file:RelVal_Raw_2018v36_MC.root : FAILED - time: date Tue Oct 2 11:56:28 2018-date Tue Oct 2 11:56:12 2018 s - exit: 256
cmsRun /build/cmsbld/jenkins/workspace/ib-any-integration/CMSSW_10_3_X_2018-10-01-2300/src/HLTrigger/Configuration/test/OnLine_HLT_2018v36.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Tue Oct 2 11:56:28 2018-date Tue Oct 2 11:56:12 2018 s - exit: 21504
cmsDriver.py RelVal -s HLT:2018v36,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_2018v36 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_2018v36_MC.root --fileout file:RelVal_Raw_2018v36_MC_HLT_RECO.root : FAILED - time: date Tue Oct 2 11:56:28 2018-date Tue Oct 2 11:56:12 2018 s - exit: 256
cmsDriver.py TTbar_Tauola_13TeV_TuneCUETP8M1_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=HeavyIons -n 10 --conditions auto:run2_mc_HIon --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_HI --fileout file:RelVal_Raw_HIon_MC.root : FAILED - time: date Tue Oct 2 11:56:24 2018-date Tue Oct 2 11:56:17 2018 s - exit: 256
cmsRun /build/cmsbld/jenkins/workspace/ib-any-integration/CMSSW_10_3_X_2018-10-01-2300/src/HLTrigger/Configuration/test/OnLine_HLT_HIon.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Tue Oct 2 11:56:24 2018-date Tue Oct 2 11:56:17 2018 s - exit: 21504
cmsDriver.py RelVal -s HLT:HIon,RAW2DIGI,L1Reco,RECO --mc --scenario=HeavyIons -n 10 --conditions auto:run2_mc_HIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_HI --processName=HLTRECO --filein file:RelVal_Raw_HIon_MC.root --fileout file:RelVal_Raw_HIon_MC_HLT_RECO.root : FAILED - time: date Tue Oct 2 11:56:24 2018-date Tue Oct 2 11:56:17 2018 s - exit: 256
cmsDriver.py TTbar_Tauola_13TeV_TuneCUETP8M1_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=pp -n 10 --conditions auto:run2_mc_Fake2 --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_2016 --fileout file:RelVal_Raw_Fake2_MC.root : FAILED - time: date Tue Oct 2 11:56:24 2018-date Tue Oct 2 11:56:19 2018 s - exit: 256
cmsRun /build/cmsbld/jenkins/workspace/ib-any-integration/CMSSW_10_3_X_2018-10-01-2300/src/HLTrigger/Configuration/test/OnLine_HLT_Fake2.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Tue Oct 2 11:56:24 2018-date Tue Oct 2 11:56:19 2018 s - exit: 21504
cmsDriver.py RelVal -s HLT:Fake2,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_Fake2 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --processName=HLTRECO --filein file:RelVal_Raw_Fake2_MC.root --fileout file:RelVal_Raw_Fake2_MC_HLT_RECO.root : FAILED - time: date Tue Oct 2 11:56:24 2018-date Tue Oct 2 11:56:19 2018 s - exit: 256
cmsDriver.py TTbar_Tauola_13TeV_TuneCUETP8M1_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=pp -n 10 --conditions auto:run2_mc_Fake1 --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_25ns --fileout file:RelVal_Raw_Fake1_MC.root : FAILED - time: date Tue Oct 2 11:56:28 2018-date Tue Oct 2 11:56:23 2018 s - exit: 256
cmsRun /build/cmsbld/jenkins/workspace/ib-any-integration/CMSSW_10_3_X_2018-10-01-2300/src/HLTrigger/Configuration/test/OnLine_HLT_Fake1.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Tue Oct 2 11:56:28 2018-date Tue Oct 2 11:56:23 2018 s - exit: 21504
cmsDriver.py RelVal -s HLT:Fake1,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_Fake1 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_25ns --processName=HLTRECO --filein file:RelVal_Raw_Fake1_MC.root --fileout file:RelVal_Raw_Fake1_MC_HLT_RECO.root : FAILED - time: date Tue Oct 2 11:56:28 2018-date Tue Oct 2 11:56:23 2018 s - exit: 256
cmsDriver.py RelVal -s L1REPACK:GT1 --data --scenario=pp -n 10 --conditions auto:run1_hlt_Fake --relval 9000,50 --datatier "RAW" --eventcontent RAW --customise=HLTrigger/Configuration/CustomConfigs.L1T --fileout file:RelVal_Raw_Fake_DATA.root --filein /store/data/Run2012A/MuEG/RAW/v1/000/191/718/14932935-E289-E111-830C-5404A6388697.root : FAILED - time: date Tue Oct 2 11:56:42 2018-date Tue Oct 2 11:56:28 2018 s - exit: 23040
cmsRun /build/cmsbld/jenkins/workspace/ib-any-integration/CMSSW_10_3_X_2018-10-01-2300/src/HLTrigger/Configuration/test/OnLine_HLT_Fake.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Tue Oct 2 11:56:42 2018-date Tue Oct 2 11:56:28 2018 s - exit: 21504
cmsDriver.py RelVal -s HLT:Fake,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run1_data_Fake --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --processName=HLTRECO --filein file:RelVal_Raw_Fake_DATA.root --fileout file:RelVal_Raw_Fake_DATA_HLT_RECO.root : FAILED - time: date Tue Oct 2 11:56:42 2018-date Tue Oct 2 11:56:28 2018 s - exit: 23040
cmsDriver.py RelVal -s L1REPACK:Full --data --scenario=pp -n 10 --conditions auto:run2_hlt_GRun --relval 9000,50 --datatier "RAW" --eventcontent RAW --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_2018 --fileout file:RelVal_Raw_GRun_DATA.root --filein /store/data/Run2017A/HLTPhysics4/RAW/v1/000/295/606/00000/36DE5E0A-3645-E711-8FA1-02163E01A43B.root : FAILED - time: date Tue Oct 2 11:56:44 2018-date Tue Oct 2 11:56:31 2018 s - exit: 256
cmsRun /build/cmsbld/jenkins/workspace/ib-any-integration/CMSSW_10_3_X_2018-10-01-2300/src/HLTrigger/Configuration/test/OnLine_HLT_GRun.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Tue Oct 2 11:56:44 2018-date Tue Oct 2 11:56:31 2018 s - exit: 21504
cmsDriver.py RelVal -s HLT:GRun,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_GRun --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_GRun_DATA.root --fileout file:RelVal_Raw_GRun_DATA_HLT_RECO.root : FAILED - time: date Tue Oct 2 11:56:44 2018-date Tue Oct 2 11:56:31 2018 s - exit: 256
cmsDriver.py RelVal -s L1REPACK:GCTGT --data --scenario=pp -n 10 --conditions auto:run2_hlt_Fake1 --relval 9000,50 --datatier "RAW" --eventcontent RAW --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_25ns --fileout file:RelVal_Raw_Fake1_DATA.root --filein /store/data/Run2015D/MuonEG/RAW/v1/000/256/677/00000/80950A90-745D-E511-92FD-02163E011C5D.root : FAILED - time: date Tue Oct 2 11:56:38 2018-date Tue Oct 2 11:56:33 2018 s - exit: 256
cmsRun /build/cmsbld/jenkins/workspace/ib-any-integration/CMSSW_10_3_X_2018-10-01-2300/src/HLTrigger/Configuration/test/OnLine_HLT_Fake1.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Tue Oct 2 11:56:38 2018-date Tue Oct 2 11:56:33 2018 s - exit: 21504
cmsDriver.py RelVal -s HLT:Fake1,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_Fake1 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_25ns --processName=HLTRECO --filein file:RelVal_Raw_Fake1_DATA.root --fileout file:RelVal_Raw_Fake1_DATA_HLT_RECO.root : FAILED - time: date Tue Oct 2 11:56:38 2018-date Tue Oct 2 11:56:33 2018 s - exit: 256
cmsDriver.py TTbar_Tauola_13TeV_TuneCUETP8M1_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=pp -n 10 --conditions auto:run2_mc_PIon --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_2018 --fileout file:RelVal_Raw_PIon_MC.root : FAILED - time: date Tue Oct 2 11:56:44 2018-date Tue Oct 2 11:56:37 2018 s - exit: 256
cmsRun /build/cmsbld/jenkins/workspace/ib-any-integration/CMSSW_10_3_X_2018-10-01-2300/src/HLTrigger/Configuration/test/OnLine_HLT_PIon.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Tue Oct 2 11:56:44 2018-date Tue Oct 2 11:56:37 2018 s - exit: 21504
cmsDriver.py RelVal -s HLT:PIon,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_PIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_PIon_MC.root --fileout file:RelVal_Raw_PIon_MC_HLT_RECO.root : FAILED - time: date Tue Oct 2 11:56:44 2018-date Tue Oct 2 11:56:37 2018 s - exit: 256

cmsbuild · 2018-10-02T09:57:25Z

Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped)

slava77 · 2018-10-02T10:54:26Z

@cmsbuild please test

I forgot to push the fix before submitting

cmsbuild · 2018-10-02T10:54:38Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/30846/console Started: 2018/10/02 12:55

slava77 · 2018-10-02T11:00:15Z

@smuzaffar

I notice that the code-checks were not triggered here after I updated the PR from 4a7faa1 to the current head with a --force push.

It looks like I found a way to bypass the checks ;)

smuzaffar · 2018-10-02T11:40:35Z

yes this is a known issue. We rely of the commit timestamp so if you had a commit in past and you force push that commit then cms-bot does not recognize it and does not reset the signatures.

I will check if we can make use of PR request pushed_at time for this purpose.

mandrenguyen · 2018-10-02T12:51:20Z

@cfmcginn @mverwe @FHead @stepobr We're expected by PPD to compare these two eras and verify that the mitigation behaves reasonably in PbPb.

cmsbuild · 2018-10-02T13:07:13Z

+1
Tested at: 3d2d8ca
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-24777/30846/summary.html

cmsbuild · 2018-10-02T13:07:16Z

Comparison job queued.

cmsbuild · 2018-10-02T14:33:34Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-24777/30846/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 342 differences found in the comparisons
DQMHistoTests: Total files compared: 32
DQMHistoTests: Total histograms compared: 3162800
DQMHistoTests: Total failures: 142
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3162461
DQMHistoTests: Total skipped: 197
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 31 files compared)
Checked 134 log files, 14 edm output root files, 32 DQM output files

slava77 · 2018-10-02T14:43:30Z

Reco comparison results: 342 differences found in the comparisons

these are all in 158.0, as expected.
Changes start from PF candidates and propagate downstream, as expected.
Differences are pretty small, similar to what was observed in tests with HEM OK; these changes are due to a few bad cells present in HCAL at large.

fabiocos · 2018-10-03T15:38:10Z

+operations

the DQM differences are also concentrated in wf 158.0

fabiocos · 2018-10-03T15:38:15Z

+1

cmsbuild · 2018-10-03T15:38:31Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

add Run2_2018_pp_on_AA_noHCALmitigation derived explicitly from Run2_…

3d2d8ca

…2018_pp_on_AA

cmsbuild added this to the CMSSW_10_3_X milestone Oct 2, 2018

cmsbuild added code-checks-pending comparison-pending operations-pending orp-pending pending-signatures tests-pending labels Oct 2, 2018

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 2, 2018

cmsbuild added tests-started and removed tests-pending labels Oct 2, 2018

cmsbuild added comparison-notrun tests-rejected and removed comparison-pending tests-started labels Oct 2, 2018

slava77 force-pushed the CMSSW_10_3_0_pre5/sign1047/ppONaa-pfMitigation branch from 4a7faa1 to 3d2d8ca Compare October 2, 2018 10:53

cmsbuild added comparison-pending tests-pending and removed comparison-notrun tests-rejected labels Oct 2, 2018

cmsbuild added tests-started and removed tests-pending labels Oct 2, 2018

cmsbuild added tests-approved and removed tests-started labels Oct 2, 2018

cmsbuild added comparison-available and removed comparison-pending labels Oct 2, 2018

cmsbuild added fully-signed operations-approved orp-approved and removed operations-pending orp-pending pending-signatures labels Oct 3, 2018

cmsbuild merged commit f850c1c into cms-sw:master Oct 3, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

enable PF HEM mitigation for HI 2018 and add Run2_2018_pp_on_AA_noHCALmitigation era #24777

enable PF HEM mitigation for HI 2018 and add Run2_2018_pp_on_AA_noHCALmitigation era #24777

slava77 commented Oct 2, 2018

cmsbuild commented Oct 2, 2018

slava77 commented Oct 2, 2018

cmsbuild commented Oct 2, 2018

cmsbuild commented Oct 2, 2018 •

edited

cmsbuild commented Oct 2, 2018

cmsbuild commented Oct 2, 2018

cmsbuild commented Oct 2, 2018

slava77 commented Oct 2, 2018

cmsbuild commented Oct 2, 2018 •

edited

slava77 commented Oct 2, 2018

smuzaffar commented Oct 2, 2018 •

edited

mandrenguyen commented Oct 2, 2018

cmsbuild commented Oct 2, 2018

cmsbuild commented Oct 2, 2018

cmsbuild commented Oct 2, 2018

slava77 commented Oct 2, 2018

fabiocos commented Oct 3, 2018

fabiocos commented Oct 3, 2018

cmsbuild commented Oct 3, 2018

enable PF HEM mitigation for HI 2018 and add Run2_2018_pp_on_AA_noHCALmitigation era #24777

enable PF HEM mitigation for HI 2018 and add Run2_2018_pp_on_AA_noHCALmitigation era #24777

Conversation

slava77 commented Oct 2, 2018

cmsbuild commented Oct 2, 2018

slava77 commented Oct 2, 2018

cmsbuild commented Oct 2, 2018

cmsbuild commented Oct 2, 2018 • edited

cmsbuild commented Oct 2, 2018

cmsbuild commented Oct 2, 2018

cmsbuild commented Oct 2, 2018

slava77 commented Oct 2, 2018

cmsbuild commented Oct 2, 2018 • edited

slava77 commented Oct 2, 2018

smuzaffar commented Oct 2, 2018 • edited

mandrenguyen commented Oct 2, 2018

cmsbuild commented Oct 2, 2018

cmsbuild commented Oct 2, 2018

cmsbuild commented Oct 2, 2018

slava77 commented Oct 2, 2018

fabiocos commented Oct 3, 2018

fabiocos commented Oct 3, 2018

cmsbuild commented Oct 3, 2018

cmsbuild commented Oct 2, 2018 •

edited

cmsbuild commented Oct 2, 2018 •

edited

smuzaffar commented Oct 2, 2018 •

edited