relval matrix updates #13797

hengne · 2016-03-22T10:49:25Z

relval updates:

add phase1 ZMM_13 pythia8 workflow
update gensim input for phase1
finer job splitting for phase1 digi and reco steps
update mixing/premixing inputs for 2015 workflows
JpsiMuMu_Pt-15 workflow update to JpsiMuMu_Pt-8 with Pythia8

… workflows

…MSSW_8_1_0_pre1_relval_fixJson_phase1_rebase

hengne · 2016-03-22T10:49:48Z

please test

cmsbuild · 2016-03-22T10:49:49Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/12005/console

cmsbuild · 2016-03-22T10:49:50Z

A new Pull Request was created by @hengne (Hengne Li) for CMSSW_8_1_X.

It involves the following packages:

Configuration/PyReleaseValidation

@boudoul, @franzoni, @fabozzi, @cmsbuild, @srimanob, @hengne, @davidlange6 can you please review it and eventually sign? Thanks.
@ghellwig, @Martin-Grunewald this is something you requested to watch as well.
@slava77, @Degano, @smuzaffar you are the release manager for this.

cms-bot commands are list here #13028

davidlange6 · 2016-03-22T16:49:06Z

Configuration/PyReleaseValidation/python/MatrixInjector.py

+            wmsplit['DigiFull']=5
+            wmsplit['RecoFull']=5
+            wmsplit['DigiFullPU']=1
+            wmsplit['RecoFullPU']=1


these really effect only the very high pileup workflows?

first two lines are for noPU workflows 5 lumis/job, last two lines are for PU workflows 1 lumi/job. I just take the same configs as for 2015 workflows.

cmsbuild · 2016-03-22T17:57:07Z

-1

Tested at: 819c129
I found errors in the following unit tests:

---> test runtestTqafTopEventSelection had ERRORS

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13797/12005/summary.html

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
87b7530
8580f6e
bb9ac08
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13797/12005/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13797/12005/git-merge-result

hengne · 2016-03-23T11:01:02Z

There are the following errors:
1.) in unitTest, there is mainly the database connection error :

####### Case1: Connect to an invalid database.
CORAL/Services/XMLLookupService Error Could not open "dblookup.xml" for reading
# Expected Error: No physical Update connection for "invalid" is available. ( CORAL : "ReplicaCatalogue::replicasForConnection" from "CORAL/Services/ConnectionService" )
####### Case2: Connect to a valid database. Transaction not started.

2.) Matrix Test, there are two DAS errors.

All seem to be temporary connection errors to me. I will test it again.

hengne · 2016-03-23T11:01:07Z

please test

cmsbuild · 2016-03-23T11:01:27Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/12019/console

cmsbuild · 2016-03-23T14:53:02Z

-1
Tested at: 819c129
When I ran the RelVals I found an error in the following worklfows:
25202.0 step3

runTheMatrix-results/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25/step3_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25.log
----- Begin Fatal Exception 23-Mar-2016 14:53:19 CET-----------------------
An exception of category 'Configuration' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=MixingModule label='mix'
Exception Message:
RootEmbeddedFileSequence no input files specified for secondary input source.
----- End Fatal Exception -------------------------------------------------

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13797/12021/summary.html

hengne · 2016-03-23T15:56:55Z

for the same das_client.py HTTPError i will wait until this issue has been solved to test again.

boudoul · 2016-03-23T16:10:30Z

Just for my own education, since this is the step3 which is failing because of query to the pile up dataset, , why do we have the pile up option in the step3 (reco step) ? I naively though the pile up option was needed only for the digi step ? (non urgent it's really for my own curiosity)

hengne · 2016-03-23T16:14:06Z

me too... i don't really understand... the mixing should be done at the DIGI step. but would the RAW2DIGI in the reco step also makes such a das_client query?

davidlange6 · 2016-03-23T16:24:26Z

the validation step effectively redoes all the mixing.. (apparently implemented as a second das query to get the files)

On Mar 23, 2016, at 5:14 PM, Hengne Li notifications@github.com wrote:

me too... i don't really understand... the mixing should be done at the DIGI step. but would the RAW2DIGI in the reco step also makes such a das_client query?

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub

boudoul · 2016-03-23T16:27:50Z

ah it's the validation step , ok thanks!

hengne · 2016-03-24T08:46:44Z

please test

cmsbuild · 2016-03-24T08:47:07Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/12038/console

cmsbuild · 2016-03-24T10:17:56Z

+1
Tested at: 819c129
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13797/12038/summary.html

hengne · 2016-03-24T10:19:48Z

+1

cmsbuild · 2016-03-24T10:20:15Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_8_1_X IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @slava77, @davidlange6, @Degano, @smuzaffar

cmsbuild · 2016-03-24T11:55:05Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13797/12038/summary.html

davidlange6 · 2016-03-28T10:21:59Z

+1

hengne added 12 commits March 16, 2016 11:56

update phase1 ZMM workflow to pythia8 and 13TeV

2208957

add commentted lines for pu50 and pu70 for upgrade workflows

04be57a

update premixing input

7c5e0a5

update JpsiMuMu_Pt-15 to JpsiMuMu_Pt-8 Pythia8

e439cdb

update gensim input for phase1

7b93ffb

temp remove t2_ch_cern request

fdc375f

upgrade ZMM_13TeV PD name change to ZMM_13 for consistency with other…

b1b24da

… workflows

add finner job splitting for phase 1 workflows

072ed16

update fastsim premixing no signal input

cad94a5

Merge remote-tracking branch 'official-cmssw/CMSSW_8_1_X' into from-C…

87b7530

…MSSW_8_1_0_pre1_relval_fixJson_phase1_rebase

relval memory back to 3k GB, PU back to 35

6865c43

add back T2_CH_CERN requirement

819c129

cmsbuild added this to the Next CMSSW_8_1_X milestone Mar 22, 2016

cmsbuild added pending-signatures tests-pending orp-pending comparison-pending pdmv-pending labels Mar 22, 2016

cmsbuild added tests-started and removed tests-pending labels Mar 22, 2016

davidlange6 reviewed Mar 22, 2016
View reviewed changes

cmsbuild added tests-rejected and removed tests-started labels Mar 22, 2016

cmsbuild added tests-rejected and removed tests-started labels Mar 23, 2016

cmsbuild added tests-pending tests-started and removed tests-rejected tests-pending labels Mar 24, 2016

cmsbuild added tests-approved and removed tests-started labels Mar 24, 2016

cmsbuild added fully-signed pdmv-approved and removed pdmv-pending pending-signatures labels Mar 24, 2016

cmsbuild added comparison-available and removed comparison-pending labels Mar 24, 2016

cmsbuild added orp-approved and removed orp-pending labels Mar 28, 2016

cmsbuild merged commit 4671860 into cms-sw:CMSSW_8_1_X Mar 28, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

relval matrix updates #13797

relval matrix updates #13797

hengne commented Mar 22, 2016

hengne commented Mar 22, 2016

cmsbuild commented Mar 22, 2016

cmsbuild commented Mar 22, 2016

davidlange6 Mar 22, 2016

hengne Mar 22, 2016

cmsbuild commented Mar 22, 2016

hengne commented Mar 23, 2016

hengne commented Mar 23, 2016

cmsbuild commented Mar 23, 2016

cmsbuild commented Mar 23, 2016

hengne commented Mar 23, 2016

boudoul commented Mar 23, 2016

hengne commented Mar 23, 2016

davidlange6 commented Mar 23, 2016

boudoul commented Mar 23, 2016

hengne commented Mar 24, 2016

cmsbuild commented Mar 24, 2016

cmsbuild commented Mar 24, 2016

hengne commented Mar 24, 2016

cmsbuild commented Mar 24, 2016

cmsbuild commented Mar 24, 2016

davidlange6 commented Mar 28, 2016

relval matrix updates #13797

relval matrix updates #13797

Conversation

hengne commented Mar 22, 2016

hengne commented Mar 22, 2016

cmsbuild commented Mar 22, 2016

cmsbuild commented Mar 22, 2016

davidlange6 Mar 22, 2016

Choose a reason for hiding this comment

hengne Mar 22, 2016

Choose a reason for hiding this comment

cmsbuild commented Mar 22, 2016

hengne commented Mar 23, 2016

hengne commented Mar 23, 2016

cmsbuild commented Mar 23, 2016

cmsbuild commented Mar 23, 2016

hengne commented Mar 23, 2016

boudoul commented Mar 23, 2016

hengne commented Mar 23, 2016

davidlange6 commented Mar 23, 2016

boudoul commented Mar 23, 2016

hengne commented Mar 24, 2016

cmsbuild commented Mar 24, 2016

cmsbuild commented Mar 24, 2016

hengne commented Mar 24, 2016

cmsbuild commented Mar 24, 2016

cmsbuild commented Mar 24, 2016

davidlange6 commented Mar 28, 2016