CTPPS: bug fix #2 in run ranges for FEDs channels mapping (9_2_X) #18734

forthommel · 2017-05-15T09:51:49Z

Forward-port of #18732 onto 9_2_X.

Proper handling of the begin/end conditions through the usage of a edm::contains (instead of a home-brewed check).

runTheMatrix.py -l limited on 9316056 yielded 17 17 16 14 7 1 1 1 tests passed, 1 0 0 0 0 0 0 0 failed (with the usual step1 DAS error on 4.22)

cmsbuild · 2017-05-15T09:52:08Z

A new Pull Request was created by @forthommel (Laurent Forthomme) for master.

It involves the following packages:

CondFormats/CTPPSReadoutObjects

@ghellwig, @arunhep, @cerminar, @cmsbuild, @franzoni, @ggovi, @mmusich, @davidlange6 can you please review it and eventually sign? Thanks.
@ghellwig, @tocheng this is something you requested to watch as well.
@davidlange6 you are the release manager for this.

cms-bot commands are listed here

franzoni · 2017-05-15T17:23:11Z

please test

cmsbuild · 2017-05-15T17:23:42Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/19844/console Started: 2017/05/15 21:51

forthommel · 2017-05-15T19:52:33Z

Thanks a lot, @franzoni! It seems that @cmsbuild is a bit tired tonight :)

cmsbuild · 2017-05-15T20:44:56Z

-1

Tested at: 11b5f3f

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-18734/19844/summary.html

I found follow errors while testing this PR

Failed tests: RelVals AddOn

RelVals:

When I ran the RelVals I found an error in the following worklfows:
101.0 step1

runTheMatrix-results/101.0_SingleElectronE120EHCAL+SingleElectronE120EHCAL/step1_SingleElectronE120EHCAL+SingleElectronE120EHCAL.log

8.0 step3

runTheMatrix-results/8.0_BeamHalo+BeamHalo+DIGICOS+RECOCOS+ALCABH+HARVESTCOS/step3_BeamHalo+BeamHalo+DIGICOS+RECOCOS+ALCABH+HARVESTCOS.log

1306.0 step3

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step3_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log

1330.0 step3

runTheMatrix-results/1330.0_ZMM_13+ZMM_13+DIGIUP15+RECOUP15+HARVESTUP15/step3_ZMM_13+ZMM_13+DIGIUP15+RECOUP15+HARVESTUP15.log

10042.0 step3

runTheMatrix-results/10042.0_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+ALCAFull_2017+HARVESTFull_2017/step3_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+ALCAFull_2017+HARVESTFull_2017.log

9.0 step3

runTheMatrix-results/9.0_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST/step3_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST.log

25.0 step3

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT/step3_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT.log

50202.0 step3

runTheMatrix-results/50202.0_TTbar_13+TTbar_13+DIGIUP15_PU50+RECOUP15_PU50+HARVESTUP15_PU50/step3_TTbar_13+TTbar_13+DIGIUP15_PU50+RECOUP15_PU50+HARVESTUP15_PU50.log

10824.0 step3

runTheMatrix-results/10824.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+ALCAFull_2018+HARVESTFull_2018/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+ALCAFull_2018+HARVESTFull_2018.log

10024.0 step3

runTheMatrix-results/10024.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+ALCAFull_2017+HARVESTFull_2017/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+ALCAFull_2017+HARVESTFull_2017.log

21234.0 step3

runTheMatrix-results/21234.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D4_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D4+RecoFullGlobal_2023D4+HARVESTFullGlobal_2023D4/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D4_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D4+RecoFullGlobal_2023D4+HARVESTFullGlobal_2023D4.log

24434.0 step3

runTheMatrix-results/24434.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D12_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D12+RecoFullGlobal_2023D12+HARVESTFullGlobal_2023D12/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D12_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D12+RecoFullGlobal_2023D12+HARVESTFullGlobal_2023D12.log

26634.0 step3

runTheMatrix-results/26634.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D15_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D15+RecoFullGlobal_2023D15+HARVESTFullGlobal_2023D15/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D15_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D15+RecoFullGlobal_2023D15+HARVESTFullGlobal_2023D15.log

23234.0 step3

runTheMatrix-results/23234.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D8_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D8+RecoFullGlobal_2023D8+HARVESTFullGlobal_2023D8/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D8_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D8+RecoFullGlobal_2023D8+HARVESTFullGlobal_2023D8.log

25202.0 step3

runTheMatrix-results/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25/step3_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25.log

AddOn:

I found errors in the following addon tests:

cmsDriver.py RelVal -s HLT:GRun,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_GRun --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2017 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_GRun_MC.root --fileout file:RelVal_Raw_GRun_MC_HLT_RECO.root : FAILED - time: date Mon May 15 22:39:17 2017-date Mon May 15 22:26:20 2017 s - exit: 16640
cmsDriver.py RelVal -s HLT:PRef,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_PRef --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2017 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_PRef_MC.root --fileout file:RelVal_Raw_PRef_MC_HLT_RECO.root : FAILED - time: date Mon May 15 22:39:01 2017-date Mon May 15 22:26:23 2017 s - exit: 16640
cmsDriver.py RelVal -s HLT:GRun2016,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_GRun2016 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_GRun2016_MC.root --fileout file:RelVal_Raw_GRun2016_MC_HLT_RECO.root : FAILED - time: date Mon May 15 22:42:02 2017-date Mon May 15 22:26:32 2017 s - exit: 16640
cmsDriver.py RelVal -s HLT:Fake,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run1_mc_Fake --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --processName=HLTRECO --filein file:RelVal_Raw_Fake_MC.root --fileout file:RelVal_Raw_Fake_MC_HLT_RECO.root : FAILED - time: date Mon May 15 22:37:44 2017-date Mon May 15 22:26:38 2017 s - exit: 16640
cmsDriver.py RelVal -s HLT:HIon,RAW2DIGI,L1Reco,RECO --mc --scenario=HeavyIons -n 10 --conditions auto:run2_mc_HIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016,Run2_HI --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_HIon_MC.root --fileout file:RelVal_Raw_HIon_MC_HLT_RECO.root : FAILED - time: date Mon May 15 22:38:34 2017-date Mon May 15 22:26:40 2017 s - exit: 16640
cmsDriver.py RelVal -s HLT:Fake2,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_Fake2 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --processName=HLTRECO --filein file:RelVal_Raw_Fake2_MC.root --fileout file:RelVal_Raw_Fake2_MC_HLT_RECO.root : FAILED - time: date Mon May 15 22:38:53 2017-date Mon May 15 22:26:41 2017 s - exit: 16640
cmsDriver.py RelVal -s HLT:Fake1,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_Fake1 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_25ns --processName=HLTRECO --filein file:RelVal_Raw_Fake1_MC.root --fileout file:RelVal_Raw_Fake1_MC_HLT_RECO.root : FAILED - time: date Mon May 15 22:38:17 2017-date Mon May 15 22:26:45 2017 s - exit: 16640
cmsDriver.py RelVal -s HLT:PIon,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_PIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2017 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_PIon_MC.root --fileout file:RelVal_Raw_PIon_MC_HLT_RECO.root : FAILED - time: date Mon May 15 22:39:26 2017-date Mon May 15 22:26:57 2017 s - exit: 16640

cmsbuild · 2017-05-15T20:44:59Z

Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped)

cmsbuild · 2017-05-15T21:33:00Z

Pull request #18734 was updated. @ghellwig, @arunhep, @cerminar, @cmsbuild, @franzoni, @ggovi, @mmusich, @davidlange6 can you please check and sign again.

forthommel · 2017-05-15T21:43:03Z

Fixed again this (in)famous 1:min → edm::EventID(1, 0, 1) conversion (instead of (1, 0, 0) leading to an unknown mapping/out of range) with 549fbf0.

…figuration file

cmsbuild · 2017-05-15T22:19:38Z

Pull request #18734 was updated. @ghellwig, @arunhep, @cerminar, @cmsbuild, @franzoni, @ggovi, @mmusich, @davidlange6 can you please check and sign again.

slava77 · 2017-05-15T22:30:18Z

@cmsbuild please test

cmsbuild · 2017-05-15T22:30:35Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/19857/console Started: 2017/05/16 00:32

cmsbuild · 2017-05-15T23:52:36Z

+1
Tested at: 9316056
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-18734/19857/summary.html

cmsbuild · 2017-05-15T23:52:39Z

Comparison job queued.

cmsbuild · 2017-05-16T02:17:16Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-18734/19857/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 1690 differences found in the comparisons
DQMHistoTests: Total files compared: 24
DQMHistoTests: Total histograms compared: 1829428
DQMHistoTests: Total failures: 27275
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 1801973
DQMHistoTests: Total skipped: 180
DQMHistoTests: Total Missing objects: 0
Checked 98 log files, 14 edm output root files, 24 DQM output files

franzoni · 2017-05-16T12:09:36Z

hello @forthommel
can you please try to run w/ this PR
on the configuration and input data which generated the failure you're trying to fix ( https://hypernews.cern.ch/HyperNews/CMS/get/prep-ops/3940/1/1/1/1/1.html )

Thanks!

forthommel · 2017-05-16T12:32:02Z

Hi @franzoni,
I tested the 8_0_X version of this PR with 1000 events of the PSet.py/pkl attached to this HN thread, and I did not manage to reproduce the crash encountered before the patch!

ggovi · 2017-05-16T15:43:48Z

+1

forthommel · 2017-05-16T16:03:11Z

@franzoni Unfortunately the transition from 8_0_X to 9_2_X is not trivial enough for this configuration to run out of the box (the first error encountered is related to the missing _task attribute of the Schedule object, but I suspect there will be more hidden there...)

forthommel added 2 commits May 15, 2017 11:40

Replacing the homebrewed validity ranges check by a edm::contains

c2e5fcd

Replacing the boost shared pointers by the STL ones

11b5f3f

cmsbuild added this to the CMSSW_9_2_X milestone May 15, 2017

cmsbuild added alca-pending comparison-pending db-pending orp-pending pending-signatures tests-pending labels May 15, 2017

cmsbuild added tests-started and removed tests-pending labels May 15, 2017

cmsbuild added comparison-notrun tests-rejected and removed comparison-pending tests-started labels May 15, 2017

Treating the event id 1:min as a special case

549fbf0

cmsbuild added comparison-pending tests-pending and removed comparison-notrun tests-rejected labels May 15, 2017

Using the amended validity range instead of the one stored in the con…

9316056

…figuration file

cmsbuild added tests-started and removed tests-pending labels May 15, 2017

cmsbuild added tests-approved and removed tests-started labels May 15, 2017

cmsbuild added comparison-available and removed comparison-pending labels May 16, 2017

cmsbuild added db-approved and removed db-pending labels May 16, 2017

davidlange6 merged commit cfa9831 into cms-sw:master May 16, 2017

forthommel deleted the ctpps-runranges_patch2-9_2_X branch May 16, 2017 20:18

forthommel mentioned this pull request May 17, 2017

CTPPS: Small memory footprint reduction for the channels mapping handler #18815

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CTPPS: bug fix #2 in run ranges for FEDs channels mapping (9_2_X) #18734

CTPPS: bug fix #2 in run ranges for FEDs channels mapping (9_2_X) #18734

forthommel commented May 15, 2017 •

edited

cmsbuild commented May 15, 2017

franzoni commented May 15, 2017

cmsbuild commented May 15, 2017 •

edited

forthommel commented May 15, 2017

cmsbuild commented May 15, 2017

cmsbuild commented May 15, 2017

cmsbuild commented May 15, 2017

forthommel commented May 15, 2017

cmsbuild commented May 15, 2017

slava77 commented May 15, 2017

cmsbuild commented May 15, 2017 •

edited

cmsbuild commented May 15, 2017

cmsbuild commented May 15, 2017

cmsbuild commented May 16, 2017

franzoni commented May 16, 2017

forthommel commented May 16, 2017 •

edited

ggovi commented May 16, 2017

forthommel commented May 16, 2017

CTPPS: bug fix #2 in run ranges for FEDs channels mapping (9_2_X) #18734

CTPPS: bug fix #2 in run ranges for FEDs channels mapping (9_2_X) #18734

Conversation

forthommel commented May 15, 2017 • edited

cmsbuild commented May 15, 2017

franzoni commented May 15, 2017

cmsbuild commented May 15, 2017 • edited

forthommel commented May 15, 2017

cmsbuild commented May 15, 2017

cmsbuild commented May 15, 2017

cmsbuild commented May 15, 2017

forthommel commented May 15, 2017

cmsbuild commented May 15, 2017

slava77 commented May 15, 2017

cmsbuild commented May 15, 2017 • edited

cmsbuild commented May 15, 2017

cmsbuild commented May 15, 2017

cmsbuild commented May 16, 2017

franzoni commented May 16, 2017

forthommel commented May 16, 2017 • edited

ggovi commented May 16, 2017

forthommel commented May 16, 2017

forthommel commented May 15, 2017 •

edited

cmsbuild commented May 15, 2017 •

edited

cmsbuild commented May 15, 2017 •

edited

forthommel commented May 16, 2017 •

edited