adding Prod2017 wfs #22393

prebello · 2018-02-28T18:03:06Z

This PR aims to add new Prod2017 wfs. Following #22388

FYI @fabozzi

cmsbuild · 2018-02-28T18:03:25Z

The code-checks are being triggered in jenkins.

cmsbuild · 2018-02-28T18:05:39Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-22393/3617

cmsbuild · 2018-02-28T18:05:55Z

A new Pull Request was created by @prebello (Patricia Rebello Teles) for master.

It involves the following packages:

Configuration/PyReleaseValidation

@GurpreetSinghChahal, @cmsbuild, @prebello, @kpedro88, @fabozzi can you please review it and eventually sign? Thanks.
@makortel, @felicepantaleo, @ebrondol, @ghellwig, @Martin-Grunewald this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

prebello · 2018-02-28T23:29:27Z

please test workflow 1301.17,1302.17,1303.17

cmsbuild · 2018-02-28T23:29:42Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/26391/console Started: 2018/03/01 01:03

cmsbuild · 2018-03-01T01:56:53Z

+1
Tested at: e239469
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22393/26391/summary.html

cmsbuild · 2018-03-01T01:56:56Z

Comparison job queued.

cmsbuild · 2018-03-01T03:25:13Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22393/26391/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/build/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-22393/1301.17_ProdMinBias_13UP17+ProdMinBias_13UP17+DIGIUP17PROD1+RECOPRODUP17
/build/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-22393/1302.17_ProdTTbar_13UP17+ProdTTbar_13UP17+DIGIUP17PROD1+RECOPRODUP17
/build/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-22393/1303.17_ProdQCD_Pt_3000_3500_13UP17+ProdQCD_Pt_3000_3500_13UP17+DIGIUP17PROD1+RECOPRODUP17

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 29
DQMHistoTests: Total histograms compared: 2479021
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2478844
DQMHistoTests: Total skipped: 176
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 1.05000000005 KiB( 22 files compared)
Checked 118 log files, 9 edm output root files, 29 DQM output files

fabozzi · 2018-03-01T08:50:28Z

Configuration/PyReleaseValidation/python/relval_steps.py

@@ -1635,6 +1644,8 @@ def gen2018HiMix(fragment,howMuch):
 steps['RECOPROD1']=merge([{ '-s' : 'RAW2DIGI,L1Reco,RECO,RECOSIM,EI,PAT', '--datatier' : 'AODSIM,MINIAODSIM', '--eventcontent' : 'AODSIM,MINIAODSIM'},step3Defaults])
 #steps['RECOPRODUP15']=merge([{ '-s':'RAW2DIGI,L1Reco,RECO,EI,DQM:DQMOfflinePOGMC','--datatier':'AODSIM,DQMIO','--eventcontent':'AODSIM,DQM'},step3Up2015Defaults])
 steps['RECOPRODUP15']=merge([{ '-s':'RAW2DIGI,L1Reco,RECO,RECOSIM,EI,PAT,DQM:DQMOfflinePOGMC','--datatier':'AODSIM,MINIAODSIM,DQMIO','--eventcontent':'AODSIM,MINIAODSIM,DQM'},step3Up2015Defaults])
+## for 2017 PROD
+steps['RECOPRODUP17']=merge([{ '--era' :'Run2_2017','-s':'RAW2DIGI,L1Reco,RECO,RECOSIM,EI,PAT','--conditions': 'auto:phase1_2017_realistic'},steps['RECOPRODUP15']])


Since you are not running DQM step, you have to remove DQM/DQMIO from eventcontent/datatier

cmsbuild · 2018-03-03T02:06:27Z

Pull request #22393 was updated. @GurpreetSinghChahal, @cmsbuild, @prebello, @kpedro88, @fabozzi can you please check and sign again.

cmsbuild · 2018-03-03T06:29:35Z

+1
Tested at: 8acf388
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22393/26475/summary.html

cmsbuild · 2018-03-03T06:29:38Z

Comparison job queued.

cmsbuild · 2018-03-03T07:56:00Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22393/26475/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/build/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-22393/1301.17_ProdMinBias_13UP17+ProdMinBias_13UP17+DIGIUP17PROD1+RECOPRODUP17

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 29
DQMHistoTests: Total histograms compared: 2479021
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2478844
DQMHistoTests: Total skipped: 176
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 1.00000000011 KiB( 21 files compared)
Checked 118 log files, 9 edm output root files, 29 DQM output files

fabozzi · 2018-03-03T10:23:41Z

+1

kpedro88 · 2018-03-03T16:34:44Z

+1

cmsbuild · 2018-03-03T16:35:01Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

fabiocos

For my information, is DQMOffline no more needed for 15 samples?

prebello · 2018-03-05T16:04:07Z

Hi @fabiocos, 3 wfs were added. Maybe I misunderstood your question wrt 15 samples.
As MC production, so not used as relvals, so DQM is not needed.

fabiocos · 2018-03-05T16:34:14Z

Configuration/PyReleaseValidation/python/relval_steps.py

@@ -1634,7 +1643,9 @@ def gen2018HiMix(fragment,howMuch):
 steps['RECODBG']=merge([{'--eventcontent':'RECODEBUG,MINIAODSIM,DQM'},steps['RECO']])
 steps['RECOPROD1']=merge([{ '-s' : 'RAW2DIGI,L1Reco,RECO,RECOSIM,EI,PAT', '--datatier' : 'AODSIM,MINIAODSIM', '--eventcontent' : 'AODSIM,MINIAODSIM'},step3Defaults])
 #steps['RECOPRODUP15']=merge([{ '-s':'RAW2DIGI,L1Reco,RECO,EI,DQM:DQMOfflinePOGMC','--datatier':'AODSIM,DQMIO','--eventcontent':'AODSIM,DQM'},step3Up2015Defaults])
-steps['RECOPRODUP15']=merge([{ '-s':'RAW2DIGI,L1Reco,RECO,RECOSIM,EI,PAT,DQM:DQMOfflinePOGMC','--datatier':'AODSIM,MINIAODSIM,DQMIO','--eventcontent':'AODSIM,MINIAODSIM,DQM'},step3Up2015Defaults])
+steps['RECOPRODUP15']=merge([{ '-s':'RAW2DIGI,L1Reco,RECO,RECOSIM,EI,PAT','--datatier':'AODSIM,MINIAODSIM','--eventcontent':'AODSIM,MINIAODSIM'},step3Up2015Defaults])


I was referring to this change

steps['RECOPROD1'] (Run1 case) also excluded DQM:DQMOfflinePOGMC
Prod wfs are only for testing MC production, so DQM step is not needed. @fabozzi can explain why DQM offline was included before.

fabiocos · 2018-03-05T16:37:14Z

+1

fabozzi · 2018-03-05T17:25:11Z

@prebello @fabiocos
In 'prod' relvals we implement the same sequence as in MC production.
We just produce them as technical test for previous MC eras (run1, 2016, and now 2017).

Since in MC production we do not run any DQM sequence, we have cut
out DQM:DQMOfflinePOGMC in all 'prod' relvals (DQM:DQMOfflinePOGMC was likely a remnant of DQM sequence that was run long long time ago in MC production).

fabiocos · 2018-03-05T21:13:04Z

@fabozzi @prebello thank you for the clarification

adding Prod2017 wfs

e239469

cmsbuild added this to the CMSSW_10_1_X milestone Feb 28, 2018

cmsbuild added code-checks-pending comparison-pending orp-pending pdmv-pending pending-signatures tests-pending upgrade-pending labels Feb 28, 2018

cmsbuild added code-checks-approved and removed code-checks-pending labels Feb 28, 2018

cmsbuild added tests-started and removed tests-pending labels Feb 28, 2018

cmsbuild added tests-approved and removed tests-started labels Mar 1, 2018

cmsbuild added comparison-available and removed comparison-pending labels Mar 1, 2018

fabozzi reviewed Mar 1, 2018

View reviewed changes

removing DQM step

8acf388

cmsbuild removed code-checks-approved comparison-available tests-approved labels Mar 3, 2018

cmsbuild added the code-checks-approved label Mar 3, 2018

cmsbuild added tests-started and removed tests-pending labels Mar 3, 2018

prebello mentioned this pull request Mar 3, 2018

adding Prod2017 wfs #22388

Closed

cmsbuild added tests-approved and removed tests-started labels Mar 3, 2018

cmsbuild added comparison-available and removed comparison-pending labels Mar 3, 2018

cmsbuild added pdmv-approved and removed pdmv-pending labels Mar 3, 2018

cmsbuild added fully-signed upgrade-approved and removed pending-signatures upgrade-pending labels Mar 3, 2018

fabiocos reviewed Mar 4, 2018

View reviewed changes

fabiocos reviewed Mar 5, 2018

View reviewed changes

cmsbuild added orp-approved and removed orp-pending labels Mar 5, 2018

cmsbuild merged commit bfe12c4 into cms-sw:master Mar 5, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

adding Prod2017 wfs #22393

adding Prod2017 wfs #22393

prebello commented Feb 28, 2018

cmsbuild commented Feb 28, 2018

cmsbuild commented Feb 28, 2018

cmsbuild commented Feb 28, 2018

prebello commented Feb 28, 2018

cmsbuild commented Feb 28, 2018 •

edited

cmsbuild commented Mar 1, 2018

cmsbuild commented Mar 1, 2018

cmsbuild commented Mar 1, 2018

fabozzi Mar 1, 2018

cmsbuild commented Mar 3, 2018

cmsbuild commented Mar 3, 2018

cmsbuild commented Mar 3, 2018

cmsbuild commented Mar 3, 2018

fabozzi commented Mar 3, 2018

kpedro88 commented Mar 3, 2018

cmsbuild commented Mar 3, 2018

fabiocos left a comment

prebello commented Mar 5, 2018

fabiocos Mar 5, 2018

prebello Mar 5, 2018 •

edited

fabiocos commented Mar 5, 2018

fabozzi commented Mar 5, 2018

fabiocos commented Mar 5, 2018

adding Prod2017 wfs #22393

adding Prod2017 wfs #22393

Conversation

prebello commented Feb 28, 2018

cmsbuild commented Feb 28, 2018

cmsbuild commented Feb 28, 2018

cmsbuild commented Feb 28, 2018

prebello commented Feb 28, 2018

cmsbuild commented Feb 28, 2018 • edited

cmsbuild commented Mar 1, 2018

cmsbuild commented Mar 1, 2018

cmsbuild commented Mar 1, 2018

fabozzi Mar 1, 2018

Choose a reason for hiding this comment

cmsbuild commented Mar 3, 2018

cmsbuild commented Mar 3, 2018

cmsbuild commented Mar 3, 2018

cmsbuild commented Mar 3, 2018

fabozzi commented Mar 3, 2018

kpedro88 commented Mar 3, 2018

cmsbuild commented Mar 3, 2018

fabiocos left a comment

Choose a reason for hiding this comment

prebello commented Mar 5, 2018

fabiocos Mar 5, 2018

Choose a reason for hiding this comment

prebello Mar 5, 2018 • edited

Choose a reason for hiding this comment

fabiocos commented Mar 5, 2018

fabozzi commented Mar 5, 2018

fabiocos commented Mar 5, 2018

cmsbuild commented Feb 28, 2018 •

edited

prebello Mar 5, 2018 •

edited