Switch AddOn tests to use Run 2 eras #11690

mark-grimes · 2015-10-08T13:30:33Z

This changes the AddOn tests to use the Run 2 eras instead of the postLS1 customisations. Basically all that's changed is --customise SLHCUpgradeSimulations/Configuration/postLS1Customs.customisePostLS1 goes to --era Run2_25ns; ...customisePostLS1_50ns to --era Run2_50ns; and ...customisePostLS1_HI to --era Run2_HI.
#11466 is a PR to change over the standard runTheMatrix workflows. The latest diff of the configs produced by the two different methods is at http://grimes.web.cern.ch/grimes/eraProcessDiffs/processDiffs-7_6_X_2015-10-07-1710-PR11466.html

cmsbuild · 2015-10-08T13:31:12Z

A new Pull Request was created by @mark-grimes (Mark Grimes) for CMSSW_7_6_X.

Switch AddOn tests to use Run 2 eras

It involves the following packages:

Utilities/ReleaseScripts

@cmsbuild, @smuzaffar, @Dr15Jones can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @wddgit this is something you requested to watch as well.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.
If you are a L2 or a release manager you can ask for tests by saying 'please test' in the first line of a comment.
@Degano you are the release manager for this.
You can merge this pull request by typing 'merge' in the first line of your comment.

Martin-Grunewald · 2015-10-08T13:32:21Z

please test

cmsbuild · 2015-10-08T13:36:18Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/8694/console

cmsbuild · 2015-10-08T15:04:21Z

-1

Tested at: 54ca6cc
I found errors in the following addon tests:

cmsDriver.py RelVal -s L1REPACK:GCTGT --data --scenario=pp -n 10 --conditions auto:run2_hlt_50nsGRun --relval 9000,50 --datatier "RAW" --eventcontent RAW --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_50ns --magField 38T_PostLS1 --fileout file:RelVal_Raw_50nsGRun_DATA.root --filein /store/data/Run2012A/MuEG/RAW/v1/000/191/718/14932935-E289-E111-830C-5404A6388697.root : FAILED - time: date Thu Oct 8 16:52:34 2015-date Thu Oct 8 16:49:27 2015 s - exit: 16640
cmsDriver.py TTbar_Tauola_13TeV_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=pp -n 10 --conditions auto:run2_mc_50nsGRun --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_50ns --magField 38T_PostLS1 --fileout file:RelVal_Raw_50nsGRun_MC.root : FAILED - time: date Thu Oct 8 16:53:05 2015-date Thu Oct 8 16:49:31 2015 s - exit: 34304
cmsDriver.py RelVal -s L1REPACK:GCTGT --data --scenario=HeavyIons -n 10 --conditions auto:run2_hlt_HIon --relval 9000,50 --datatier "RAW" --eventcontent RAW --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_HI --magField 38T_PostLS1 --fileout file:RelVal_Raw_HIon_DATA.root --filein /store/hidata/HIRun2011/HIHighPt/RAW/v1/000/182/838/F20AAF66-F71C-E111-9704-BCAEC532971D.root : FAILED - time: date Thu Oct 8 16:50:58 2015-date Thu Oct 8 16:49:35 2015 s - exit: 16640
cmsDriver.py TTbar_Tauola_13TeV_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=pp -n 10 --conditions auto:run2_mc_GRun --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_25ns --magField 38T_PostLS1 --fileout file:RelVal_Raw_GRun_MC.root : FAILED - time: date Thu Oct 8 16:53:51 2015-date Thu Oct 8 16:50:26 2015 s - exit: 34304
cmsDriver.py RelVal -s L1REPACK:GCTGT --data --scenario=pp -n 10 --conditions auto:run2_hlt_PIon --relval 9000,50 --datatier "RAW" --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_25ns --magField 38T_PostLS1 --eventcontent RAW --fileout file:RelVal_Raw_PIon_DATA.root --filein /store/data/Run2012A/MuEG/RAW/v1/000/191/718/14932935-E289-E111-830C-5404A6388697.root : FAILED - time: date Thu Oct 8 16:55:01 2015-date Thu Oct 8 16:53:10 2015 s - exit: 16640
cmsDriver.py TTbar_Tauola_13TeV_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=HeavyIons -n 10 --conditions auto:run2_mc_HIon --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_HI --magField 38T_PostLS1 --fileout file:RelVal_Raw_HIon_MC.root : FAILED - time: date Thu Oct 8 16:57:00 2015-date Thu Oct 8 16:55:06 2015 s - exit: 34304
cmsDriver.py RelVal -s L1REPACK:GCTGT --data --scenario=pp -n 10 --conditions auto:run2_hlt_GRun --relval 9000,50 --datatier "RAW" --eventcontent RAW --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_25ns --magField 38T_PostLS1 --fileout file:RelVal_Raw_GRun_DATA.root --filein /store/data/Run2012A/MuEG/RAW/v1/000/191/718/14932935-E289-E111-830C-5404A6388697.root : FAILED - time: date Thu Oct 8 17:00:24 2015-date Thu Oct 8 16:57:14 2015 s - exit: 16640
cmsDriver.py TTbar_Tauola_13TeV_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=pp -n 10 --conditions auto:run2_mc_PIon --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_25ns --magField 38T_PostLS1 --fileout file:RelVal_Raw_PIon_MC.root : FAILED - time: date Thu Oct 8 17:02:33 2015-date Thu Oct 8 17:00:09 2015 s - exit: 34304

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-11690/8694/summary.html

Martin-Grunewald · 2015-10-08T15:22:13Z

I suggest to look at the std GEN-SIM-DIGI-L1 job "hlt_mc_GRun"
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-11690/8694/addOnTests/hlt_mc_GRun/

which has as error:


# Conditions read from  CMS_CONDITIONS  via FrontierProd 
disableME1a = 0
----- Begin Fatal Exception 08-Oct-2015 16:51:19 CEST-----------------------
An exception of category 'NoRecord' occurred while
   [0] Processing run: 1
   [1] Calling global beginRun for module L1TStage1Layer2Producer/'simCaloStage1Digis'
Exception Message:
No "L1TCaloConfigRcd" record found in the EventSetup for synchronization value
Run: 1 LuminosityBlock: 0 Event: 0 Time: 1
 Please add an ESSource or ESProducer that delivers such a record.
----- End Fatal Exception -------------------------------------------------
terminate called after throwing an instance of 'cms::Exception'
  what():  An exception of category 'Assert' occurred.
Exception Message:
Geant4 is still alive, master thread state must be set to EndRun before Destruct

cmsbuild · 2015-10-08T15:49:38Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-11690/8694/summary.html

mark-grimes · 2015-10-09T05:48:11Z

I've submitted #11707 to fix this. Closing until that is merged, then I'll reopen. This wasn't noticed before because the L1 step was always run with L1Reco.

mark-grimes · 2015-10-16T07:21:38Z

Now that #11817 has been merged, this should be fine.

mark-grimes · 2015-10-16T07:21:51Z

please test

cmsbuild · 2015-10-16T07:27:10Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/8945/console

cmsbuild · 2015-10-19T06:31:18Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_7_6_X IBs (but tests are reportedly failing). This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @Degano, @smuzaffar

Martin-Grunewald · 2015-10-20T04:54:47Z

Can you add a dummy era which does nothing? Called Null or None?
This way I can use in my script --era=$Era also for Run1 workflows

Thanks!

mark-grimes · 2015-10-20T05:01:47Z

Technically it's trivial, unless you want something to guarantee that no one later adds any commands to it. There was an agreement to discuss new eras in the release planning meeting, I'll raise it this afternoon.

Martin-Grunewald · 2015-10-20T06:26:47Z

OK, maybe then we should have an era called "Run1" which is (currently) empty and presumably nobody will touch it...

mark-grimes · 2015-10-20T14:30:26Z

Another option is that I fix the parsing scripts so that --era="" is the same as not specifying an era at all. It currently doesn't work but I should be able to fix that fairly easily, and doesn't risk people adding commands to it.

Martin-Grunewald · 2015-10-20T14:32:19Z

ah yes!

davidlange6 · 2015-10-20T14:39:31Z

I’m in favor of an era=“Run1” so that we can later change the default

On Oct 20, 2015, at 4:30 PM, Mark Grimes notifications@github.com wrote:

Another option is that I fix the parsing scripts so that --era="" is the same as not specifying an era at all. It currently doesn't work but I should be able to fix that fairly easily, and doesn't risk people adding commands to it.

—
Reply to this email directly or view it on GitHub.

mark-grimes · 2015-10-21T11:32:45Z

@Martin-Grunewald - Run1 era added with #12023 for 8_0_X. Let me know if you need it in other releases.

Martin-Grunewald · 2015-10-21T11:34:12Z

76X also , would be great!
Thanks!

davidlange6 · 2015-10-26T10:24:50Z

@cmsbuild, please test

cmsbuild · 2015-10-26T10:26:55Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/9191/console

cmsbuild · 2015-10-26T10:26:56Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_7_6_X IBs after it passes the integration tests. This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @Degano, @smuzaffar

cmsbuild · 2015-10-26T12:33:50Z

+1
Tested at: 54ca6cc
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-11690/9191/summary.html

cmsbuild · 2015-10-26T12:36:40Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_7_6_X IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @Degano, @smuzaffar

davidlange6 · 2015-10-26T13:33:52Z

+1

Switch AddOn tests to use Run 2 eras

cmsbuild · 2015-10-26T16:33:58Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-11690/9191/summary.html

Switch AddOn tests to use Run 2 eras instead of postLS1 customisations

54ca6cc

cmsbuild added this to the Next CMSSW_7_6_X milestone Oct 8, 2015

cmsbuild added core-pending pending-signatures tests-pending orp-pending comparison-pending labels Oct 8, 2015

mark-grimes mentioned this pull request Oct 8, 2015

Switch runTheMatrix tests to use Run 2 eras #11466

Merged

cmsbuild added tests-started and removed tests-pending labels Oct 8, 2015

cmsbuild added tests-rejected and removed tests-started labels Oct 8, 2015

cmsbuild added comparison-available and removed comparison-pending labels Oct 8, 2015

mark-grimes mentioned this pull request Oct 9, 2015

Add caloConfig and caloConfigSource to L1 step #11707

Merged

mark-grimes closed this Oct 9, 2015

mark-grimes mentioned this pull request Oct 15, 2015

Make the L1REPACK step work with the Run 2 eras #11817

Merged

mark-grimes reopened this Oct 16, 2015

cmsbuild removed comparison-available tests-rejected labels Oct 16, 2015

cmsbuild added tests-rejected and removed tests-started labels Oct 19, 2015

cmsbuild mentioned this pull request Oct 19, 2015

Switch AddOn tests to use Run 2 eras #11936

Merged

cmsbuild added tests-pending tests-started and removed tests-rejected tests-pending labels Oct 26, 2015

cmsbuild added tests-approved and removed tests-started labels Oct 26, 2015

cmsbuild added orp-approved and removed orp-pending labels Oct 26, 2015

cmsbuild added a commit that referenced this pull request Oct 26, 2015

Merge pull request #11690 from mark-grimes/switchAddOnTestsToEras

91d96e5

Switch AddOn tests to use Run 2 eras

cmsbuild merged commit 91d96e5 into cms-sw:CMSSW_7_6_X Oct 26, 2015

mark-grimes deleted the switchAddOnTestsToEras branch October 26, 2015 13:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Switch AddOn tests to use Run 2 eras #11690

Switch AddOn tests to use Run 2 eras #11690

mark-grimes commented Oct 8, 2015

cmsbuild commented Oct 8, 2015

Martin-Grunewald commented Oct 8, 2015

cmsbuild commented Oct 8, 2015

cmsbuild commented Oct 8, 2015

Martin-Grunewald commented Oct 8, 2015

cmsbuild commented Oct 8, 2015

mark-grimes commented Oct 9, 2015

mark-grimes commented Oct 16, 2015

mark-grimes commented Oct 16, 2015

cmsbuild commented Oct 16, 2015

cmsbuild commented Oct 19, 2015

Martin-Grunewald commented Oct 20, 2015

mark-grimes commented Oct 20, 2015

Martin-Grunewald commented Oct 20, 2015

mark-grimes commented Oct 20, 2015

Martin-Grunewald commented Oct 20, 2015

davidlange6 commented Oct 20, 2015

mark-grimes commented Oct 21, 2015

Martin-Grunewald commented Oct 21, 2015

davidlange6 commented Oct 26, 2015

cmsbuild commented Oct 26, 2015

cmsbuild commented Oct 26, 2015

cmsbuild commented Oct 26, 2015

cmsbuild commented Oct 26, 2015

davidlange6 commented Oct 26, 2015

cmsbuild commented Oct 26, 2015

Switch AddOn tests to use Run 2 eras #11690

Switch AddOn tests to use Run 2 eras #11690

Conversation

mark-grimes commented Oct 8, 2015

cmsbuild commented Oct 8, 2015

Martin-Grunewald commented Oct 8, 2015

cmsbuild commented Oct 8, 2015

cmsbuild commented Oct 8, 2015

Martin-Grunewald commented Oct 8, 2015

cmsbuild commented Oct 8, 2015

mark-grimes commented Oct 9, 2015

mark-grimes commented Oct 16, 2015

mark-grimes commented Oct 16, 2015

cmsbuild commented Oct 16, 2015

cmsbuild commented Oct 19, 2015

Martin-Grunewald commented Oct 20, 2015

mark-grimes commented Oct 20, 2015

Martin-Grunewald commented Oct 20, 2015

mark-grimes commented Oct 20, 2015

Martin-Grunewald commented Oct 20, 2015

davidlange6 commented Oct 20, 2015

mark-grimes commented Oct 21, 2015

Martin-Grunewald commented Oct 21, 2015

davidlange6 commented Oct 26, 2015

cmsbuild commented Oct 26, 2015

cmsbuild commented Oct 26, 2015

cmsbuild commented Oct 26, 2015

cmsbuild commented Oct 26, 2015

davidlange6 commented Oct 26, 2015

cmsbuild commented Oct 26, 2015